The pooling operation used in convolutional neural networks is a big mistake, and the fact that it works so well is a disaster.
I got fed up with academia and decided I would rather be a carpenter.
I have always been convinced that the only way to get artificial intelligence to work is to do the computation in a way similar to the human brain. That is the goal I have been pursuing. We are making progress, though we still have lots to learn about how the brain actually works.
All you need is lots and lots of data and lots of information about what the right answer is, and you'll be able to train a big neural net to do what you want.
I had a stormy graduate career, where every week we would have a shouting match. I kept doing deals where I would say, 'Okay, let me do neural nets for another six months, and I will prove to you they work.' At the end of the six months, I would say, 'Yeah, but I am almost there. Give me another six months.'
A deep-learning system doesn't have any explanatory power.
In A.I., the holy grail was how do you generate internal representations.
Deep learning is already working in Google search and in image search; it allows you to image-search a term like 'hug.' It's used to getting you Smart Replies to your Gmail. It's in speech and vision. It will soon be used in machine translation, I believe.
In the long run, curiosity-driven research just works better... Real breakthroughs come from people focusing on what they're excited about.
In a sensibly organised society, if you improve productivity, there is room for everybody to benefit.
Humans are still much better than computers at recognizing speech.
Now that neural nets work, industry and government have started calling neural nets AI. And the people in AI who spent all their life mocking neural nets and saying they'd never do anything are now happy to call them AI and try and get some of the money.
Machines can do things cheaper and better. We're very used to that in banking, for example. ATM machines are better than tellers if you want a simple transaction. They're faster, they're less trouble, they're more reliable, so they put tellers out of work.
Computers will understand sarcasm before Americans do.
We now think of internal representation as great big vectors, and we do not think of logic as the paradigm for how to get things to work. We just think you can have these great big neural nets that learn, and so, instead of programming, you are just going to get them to learn everything.
The role of radiologists will evolve from doing perceptual things that could probably be done by a highly trained pigeon to doing far more cognitive things.
My father was an entomologist who believed in continental drift. In the early '50s, that was regarded as nonsense. It was in the mid-'50s that it came back. Someone had thought of it 30 or 40 years earlier named Alfred Wegener, and he never got to see it come back.
Once your computer is pretending to be a neural net, you get it to be able to do a particular task by just showing it a whole lot of examples.
The question is, can we make neural networks that are 1,000 times bigger? And how can we do that with existing computation?
I am betting on Google's team to be the epicenter of future breakthroughs.
