Why is machine learning experiencing such explosive growth today?

(I answered this question on Quora but thought it was worth recording my thoughts here for any email subscribers.)

Machine learning has been around since the 1950's, but only in the last decade or two has it really taken off.

Why is machine learning experiencing such explosive growth today? Why not 60 years ago? Why not 20 years ago?

Data and computation.

The biggest unintended consequence of the shift from desktop to the web is that we now automatically collect vast amounts of data on just about everything, and we can ship improvements instantaneously. Those are the waves that push machine learning forward.

That sounds simple but it's hard to overstate the impact. If you were a brilliant AI researcher working on Microsoft Word in 1990, the only data you had was what you could collect in the lab. If you discovered a breakthrough, it would take years to ship it. Ten years later, the same researcher working at Google had access to a vast repository of search queries, clicks, page views, web pages, and links, and if she found an algorithmic improvement to boost click throughs by 5%, she could ship it instantly.

The internet companies (and folks doing scientific computing) were the first to apply machine learning to huge amounts of data, which is why technologies like MapReduce and BigTable were invented at places like Google, but we're seeing the same techniques move into every other application area: genetics (where the cost of sequencing DNA is falling faster than Moore's law), health (with electronic health records), energy, finance, security, and even the army. It's pervasive enough that "Big Data" conferences like Strata have thousands of attendees.

Likewise, computation is cheap and plentiful. Witness the rebirth of neural networks in the past few years. Has the backpropagation algorithm fundamentally changed since it was invented in 1974? Nope. But we have a million times more CPU power. (We also have algorithmic advances, like pretraining, Hessian-free optimization, etc. but the biggest change is Moore's law.)

Researchers were excited about machine learning as far back as the 1950's, and many of the implications were clear even then. But to achieve that vision required the rise and fall of giants: Intel to build the microprocessor, Apple to pioneer the PC, Microsoft to put a computer on every desktop, Cisco to network the network, AOL to bring the internet to the rest of us, Netscape to invent the web browser, Amazon to bring commerce online, Google to organize the world's information, and Facebook to digitize our friendships.

Ultimately, the reason machine learning works today is not due to any algorithmic breakthrough, but a decades-long macro-trend to digitize and network data, which is just now bearing fruit.