Now that most people are familiar with generative AI and the work of neural networks and LLMs, what’s next? What is the market going to experience, and what is going to drive the industry forward?
There was a panel at our IIA event in Davos in January that wrestled with some of these questions. It included quite a good lineup of business representatives with Google, TCS and others in the mix.
Moderated by Daniela Rus, my colleague at the MIT CSAIL lab, these experts went over some of the main trends in AI that are likely to influence its use in the coming years.
Scaling Laws and Architectures
Most panelists agreed that scaling laws are still alive, and still relevant.
“People have gone through all the reaches of the internet and tucked in as many words and corpuses as possible, and are hitting a bit of that wall in terms of the training set that you can use,” said panelist Jack Hidary, CEO of SandboxAQ. “And of course, you have to spend billions of dollars … in other areas of AI, though, and when we generate data, synthetic data that comes from equations or sensors or other kinds of sources, we have not even close to hitting any kind of data wall. So I think that we can scale very dramatically.”
However, there’s a move from transformer models to other new kinds of builds for LLMs that will make them more distributable, more decentralized, and able to run on edge devices.
For example, Mathias Lechner, CTO of Liquid AI, went over some aspects of liquid networks that are smaller and leaner constructs, and the resource savings attached to them. I’ve written about liquid networks a lot, so you can go back and look at the technical details. Also, this is the common disclaimer: that I have consulted with Liquid AI and know the people working on this technology at CSAIL as well.
“The inference cost, the cost to actually operate these models, you know, needs to be taken into account,” he said. “It’s important: to train such a huge model, you know, running and operating this model (is more expensive) compared to a small model that’s much more efficient, that’s much more optimized.”
In the future of applications, panelists suggested we’ll have to look at engineering solutions.
“We will see more efficiency,” said Peter Sarlin of Silo. “We will see sort of improved performance. At the same time, I think we’re still sort of mostly focusing on general-purpose AI assistance and very specific sets of use cases, as was highlighted, and … we have a lot of new modalities that need to be tackled, and there’s tremendous value in that.”
Bilingual Talent
What about the people who are needed to usher in the next generation of AI?
We may not need language experts, but we will need people with very particular sets of skills.
Hidary talked about the juxtaposition of chemistry, biology and physics, and suggested that with quantum computing thrown into the mix, there will be a need for bilingual professionals who are trained in not just AI, but other disciplines like chemistry as well.
He also mentioned how quantitative AI is being run on GPUs, which is an important development in the hardware space.
Bringing Products to Market
The panel also talked about how to actually bring revenue with AI.
Harrick Vin said TCS is looking at a process of “augmenting people” through digital processes to improve the quality of their decisions. This is something, he said, that’s in process now.
Hidary talked about biopharma and chemistry, and the use of carbon dioxide for practical purposes.
Others discussed a more generalized use, where AI undergirds all sorts of capabilities across different verticals.
The Greater Context
It’s clear that we’re moving onto new models that will help with optimization and efficiency.
One thing that didn’t come up as much in this panel is job displacement. In a piece that I did recently, I looked at various common types of jobs that may be taken over soon by AI entities. It’s also obvious that agentic AI is the way of the future. But this panel talked about how companies will use those capabilities to their advantage, and that’s a worthy topic, to understand what things will look like relatively soon.
Read the full article here
