A.I. researchers and A.I. practitioners live in different worlds.
This is the web version of Eye on A.I., Fortune’s weekly newsletter covering artificial intelligence and business. To get it delivered weekly to your in-box, sign up here.
For the past three years, two London-based investors have compiled an extremely comprehensive summary of the current “State of A.I.” It’s the work of Ian Hogarth, who founded the concert discovery site Songkick and is now a prominent angel investor, and Nathan Benaich, a venture capitalist whose firm Air Street Capital focuses on startups built around applications of artificial intelligence.
This year’s report runs to 177 detailed Powerpoint slides. It’s a great way to take the pulse of the whole field.
The report touches on so many areas that I won’t be able to do it justice. But I will highlight a few things that struck me.
One trend emerging from the report that I touched on in this newsletter back in December: a growing dichotomy between the priorities of A.I. researchers and those of A.I. practitioners who work in other kinds of businesses, such as healthcare and finance.
The research community wants to push the boundaries of what A.I. can do. Benaich calls this group the “big-model world.” And for good reason. Many of the A.I. systems that are currently at the bleeding edge are truly gargantuan. Training a model that has hundreds of billions of parameters—as OpenAI’s GPT-3 language model does—takes mind-boggling amounts of computing power and costs millions of dollars.
Benaich and Hogarth question whether that is sustainable. “We are rapidly approaching outrageous computational, economic and environmental costs to gain incrementally smaller improvements in model performance,” the two write. They note that many machine learning researchers feel that progress in the field has stagnated and that fundamentally different approaches may be needed to bring us much closer to artificial general intelligence—systems that can perform many different kinds of tasks at human or super-human levels.
With the exception of the world’s tech giants, most companies can’t afford to live in “big-model world.” If more sophisticated A.I. actually depends on building larger and larger models, then “only a small number of actors will be able to compete,” Hogarth tells me. And while the likes of Google, Microsoft and IBM hope to sell their big models, pre-trained and pre-built, to customers of their cloud computing services, many businesses are reluctant to adopt giant pre-trained A.I. software because they don’t have enough insight into how it’s trained and how it’ll perform. Companies fear that by adopting it they may be inadvertently stepping into an ethical, reputational or regulatory morass.
Most businesses live on a different A.I. planet. Benaich calls this “the task-specific A.I. world.” These folks are looking to build A.I. systems that perform one highly specialized task exceptionally well. In building this task-specific software, even startups can compete. Benaich, for instance, points to a young London company called PolyAI that he’s invested in. It has built a chat bot-like conversational A.I. system that outperforms many of the larger language models, such as Google’s BERT, but is a fraction of the size of most cutting-edge NLP systems. (PolyAI’s system took in 59 million parameters compared to BERT, which even in its lightweight version uses 110 million parameters.) This allows PolyAI’s software to be trained on just a dozen GPUs—the graphics processing chips that have become the workhorses of A.I. computing—in a single day.
Benaich and Hogarth also have a nice slide deep in the report showing that 25% of the fastest-growing Github projects in the second half of 2020 were for “machine-learning operations” (MLOps), or the engineering nitty-gritty that lets companies deploy, run and maintain A.I. software over the long haul. MLOps is now trending as a Google search term for the first time. This, Benaich and Hogarth write, “signals an industry shift from technology R&D (how to build models) to operations (how to run models).”
Here are some other key takeaways from “The State of A.I.”:
Benaich and Hogarth always make a few predictions for the coming year. Last year, they got four of six predictions right. Here are three of their eight for next year:
We’ll check in next year to see if they’re right. Meanwhile, here’s the rest of this week’s A.I. news.
Jeremy Kahn
@jeremyakahn
jeremy.kahn@fortune.com