Applied Magic

Q: What do you think is done right in the design of today's LLMs and what changes do you think are required to build much more capable systems?

A: LLMs based on the transformer-architecture with self-attention have proven highly effective in capturing long-range dependencies within very large training sets and have allowed the creation of what can reasonably be termed proto-AGI systems. LLMs are scalable in that an essentially linear improvement in performance against various benchmarks is observed for increases in the size (number of parameters and amount of compute) of the model. Transfer learning through fine-tuning of foundational models with user data enables a very wide range of applications to be built without requiring training from scratch.

However, the brute force (hammer/nail) approach of LLMs has inevitable downsides. Foundational models are increasingly costly (in terms of money, time, energy and environmental damage) both to train and to serve. It is clear that though LLMs can simulate (and a simulation of a mind is a mind) some powerful aspects of natural and human intelligences (especially as related to language), natural intelligences are much more efficient (in time and energy) in learning and generalizing from small training sets. They don’t, for instance, have to memorize essentially the whole of the internet and child can learn to identify different types of cars from just a handful of examples .There may not be one weird trick that enables general intelligence, but studying natural systems, such as the mammalian central nervous system and human cognitive structures, as well as evolutionarily more distant systems such as the New Caledonian crow, octopuses and insect colonies, can provide ample inspiration.

True AGIs thus will likely have very different architectures and characteristics to current LLMs. Hybrid systems that incorporate various dynamically interacting specialized subsystem that deploy multiple paradigms may be required. For instance, subsystems that apply reinforcement learning (including continuous learning) could enable LLMs to adapt to new information and tasks without catastrophic forgetting, while symbolic logic components could enhance their ability to perform complex reasoning and solve problems requiring explicit knowledge manipulation. Rich world models and knowledge representation grounding, including common sense about the physical universe and social world, will also be needed. These systems should help to reduce to help reduce hallucinations, increase accuracy on mathematical and logical tasks and improve explainability and interpretability. It would be good to see continual progress of these kinds of hybrid systems on benchmarks such as François Chollet’s Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) as well as proxies for general intelligence such as Oxford University finals exams. Solutions to the Alignment Problem as well as removing bias and ensuring reliability and safety will be vital in creating Friendly AGI, but are likely to prove the hardest problems to crack because there are few obvious approaches to constraining absolutely the behavior of powerful, recursively self-improving intelligent systems with multiple goals and the ability to act freely in the physical and virtual worlds.

LLMs and transformers are not necessarily the One Architecture to rule them all. Alternative approaches to LLMs include evolutionary algorithms, neuromorphic chips, spiking neural networks, or quantum computing. Let a thousand philosophies bloom!