Recent advances in artificial intelligence, primarily driven by foundation models, have enabled impressive progress. However, achieving artificial general intelligence, which involves reaching human-level performance across various tasks, remains a significant challenge. A critical missing component is a formal description of what it would take for an autonomous system to self-improve towards increasingly creative and diverse discoveries without end—a “Cambrian explosion” of emergent capabilities i-e the creation of open-ended, ever-self-improving AI remains elusive., behaviors, and artifacts. This open-ended invention is how humans and society accumulate new knowledge and technology, making it essential for artificial superhuman intelligence.
DeepMind researchers propose a concrete formal definition of open-endedness in AI systems from the perspective of novelty and learnability. They illustrate a path towards achieving artificial superhuman intelligence (ASI) by developing open-ended systems built upon foundation models. These open-ended systems would be capable of making robust, relevant discoveries that are understandable and beneficial to humans. The researchers argue that such open-endedness, enabled by the combination of foundation models and open-ended algorithms, is an essential property for any ASI system to continuously expand its capabilities and knowledge in a way that can be utilized by humanity.
The researchers provide a formal definition of open-endedness from the perspective of an observer. An open-ended system produces a sequence of artifacts that are both novel and learnable. Novelty is defined as artifacts becoming increasingly unpredictable to the observer’s model over time. Learnability requires that conditioning on a longer history of past artifacts makes future artifacts more predictable. The observer uses a statistical model to predict future artifacts based on the history, judging the quality of predictions using a loss metric. Interestingness is represented by the observer’s choice of loss function, capturing which features they find useful to learn about. This formal definition quantifies the key intuition that an open-ended system endlessly generates artifacts that are both novel and meaningful to the observer.
The researchers argue that while continued scaling of foundation models trained on passive data may lead to further improvements, this approach alone is unlikely to achieve ASI. They posit that open-endedness, the ability to endlessly generate novel yet learnable artifacts, is an essential property of any ASI system. Foundation models provide a powerful base capability, but must be combined with open-ended algorithms to enable the kind of continual, experiential learning process required for true open-endedness. The researchers outline four overlapping paths towards developing open-ended foundation models, drawing inspiration from the scientific method of forming hypotheses, experimentation, and codifying new knowledge. This paradigm of actively compiling an online dataset through open-ended exploration may represent the fastest route to realizing ASI.
With the advent of powerful foundation models, they believe designing a truly general open-ended learning system may now be feasible. However, the immense capabilities of such open-ended AI systems also come with significant safety risks that go beyond existing concerns with foundation models alone. They emphasize that solutions to these safety challenges must be pursued hand-in-hand with developing open-endedness itself, as the solutions may depend on the specific design of the open-ended system. They outline key areas of risk related to how knowledge is created and transmitted in the human-AI interaction loop. Addressing these fundamental safety problems is not just about mitigating downsides, but ensuring the open-ended system meets minimum usability specifications that would make it beneficial for humanity.
In this study, researchers strongly state that the combination of foundation models and open-ended algorithms can provide a promising path towards achieving ASI. While extremely capable, foundation models alone are limited in their ability to discover truly new knowledge. By developing open-ended systems that can endlessly generate novel yet learnable artifacts, one may be able to realize ASI and drastically enhance scientific and technological progress. However, such powerful open-ended AI systems also raise novel safety concerns that must be carefully addressed through responsible development focused on ensuring the artifacts remain interpretable to humans. If these challenges can be overcome, open-ended foundation models could unlock tremendous benefits for society.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.
If you like our work, you will love our newsletter..
Don’t Forget to join our 44k+ ML SubReddit
Asjad is an intern consultant at Marktechpost. He is persuing B.Tech in mechanical engineering at the Indian Institute of Technology, Kharagpur. Asjad is a Machine learning and deep learning enthusiast who is always researching the applications of machine learning in healthcare.
