One Model to Learn Them All and AI Is/Isn’t Taking Over the World (by Silly Rabbit)

One model to learn them all

The Google corporation recently shared this technical paper: “One model to learn them all” (less technical write up here by VentureBeat). While the model in and of itself is not transformational, the approach is a pretty big deal as it lays out a template for how to create a single machine learning model that can address multiple tasks well.

And in other Google machine learning news, Google and Carnegie Mellon University ran an experiment using ‘enormous data,’ taking an unprecedentedly huge collection of 300 million labeled images (rather than a more typical one million images) to test whether it’s possible to get more accurate image recognition not by tweaking the design of existing algorithms but by feeding them much, much more data. The answer, unsurprisingly, is yes, you get better-trained models using enormous data sets and having fifty powerful GPUs grind on the data for two months solid.

Semantic scholar

Semantic Scholar is a neat search engine for scientific papers, which has been gaining traction with Microsoft, Google and Baidu joining the Open Academic Search working group.

Quote by Oren Etzioni, the CEO of the Allen Institute for AI (AI2), who produce Semantic Scholar: “What if a cure for an intractable cancer is hidden within the tedious reports on thousands of clinical studies? In 20 years’ time, AI will be able to read — and more importantly, understand — scientific text. These AI readers will be able to connect the dots between disparate studies to identify novel hypotheses and to suggest experiments that would otherwise be missed. AI-based discovery engines will help find the answers to science’s thorniest problem.”

AI is/isn’t taking over the world

Depending who you ask, AI is either just about to take over the world or is embryonic and trivial in its achievements to date.

In the taking-over-the-world corner, we have this canonical article titled “How AI is taking over the global economy in one chart.” The absolute comparisons of R&D budget sizes in this article (and the oversimplified social conclusions) seem pretty dubious, but the point is most likely directionally correct on the relative size of R&D spending of ‘the big eight’ compared to smaller industrialized nations, as well as the fact that the ability to fund R&D is going to be very decisive for both companies and nations over the next few decades.

And in the embryonic-and-trivial corner, Evolutionary biologist Phil Madgwick points out that, “Artificial intelligence does not mimic natural intelligence, and it is not clear that there have been significant developments toward anything with rabbit-like intelligence, let alone human-like intelligence.”

My view: both of these things are simultaneously true in that while we are far from human-level machines, woe betide companies and countries which are currently under-investing in applied AI R&D.

ALIS

MOV37, a Fund of Funds (FoF) put out their thesis/manifesto for ALIS (Autonomous Learning Investment Strategies), which, as well as being a handy anthology of every known AI trope in the last 12 months, is also, in my opinion, a pretty accurate perspective on the next wave of AI-driven investing (except for the ‘two people and a laptop’ bit, which just doesn’t jibe with anything we’re seeing in any other machine learning field, per the ‘enormous data’ link above).

The real question this piece left me with is: who is going to decide which ALIS funds to invest in? Here in the Valley, ‘Deep tech’ investors are typically ex-tech entrepreneurs with deep engineering backgrounds, so they somewhat understand what they’re investing in. What’s unclear is how the majority of FoFs and allocators are going to arrange themselves to invest in ALIS machine learning strategies without any actual experience in developing ALIS-type machine learning strategies. Perhaps the FoF strategy will be more the Consumer-VC strategy of ‘just seed a bunch of small things with limited discrimination, let most die, and wait until a couple become scaled breakouts like Instagram/Pinterest/Snapchat and return the fund.’

Time will tell.

Kai Fu Lee, Commence!

And finally, as a genre, I really like commencement speeches. Speakers seem to push themselves to ‘tell their best truth’ as well as address the meaning of their achievements (while keeping it short and accessible).

Here is a great commencement speech to the Engineering School of Columbia University by legendary engineer Kai Fu Lee (of Apple, Microsoft and Google fame).

Enjoy!

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/15982/