The race to build models of the world has begun, writes The Economist
Project Genie, an experimental Artificial Intelligence (AI) model released by Google in January, is an impressive technical feat. Give the tool a request, such as an image or a short snippet of text, and it will generate an interactive world for the user to explore.
If you type a simple query, the result is a realistic simulation. If, on the other hand, you start from a painting by Georges Seurat, you can take a Sunday stroll in the park in the artist's perfect style.
Project Genie may look like a video game, but its creators claim it's something much deeper. They call it a "world model," a crucial tool for helping AI systems understand the complex and unpredictable physical spaces where many of them will be placed to work in the future.
The company argues that a future where humanoid robots go to the store to buy ingredients before cooking dinner, or where autonomous cars drive on rural roads, would not be possible without models of the world.
The concept dates back to a 1943 book by Kenneth Craik, a Scottish psychologist, who suggested that organisms carry within their minds a “small-scale model” of the world, on which they test hypotheses before applying them to reality.
Having some understanding of how the world works is a necessary step before making plans to change it. Without one, every living being would be forced to live in a purely reactive manner, recoiling from pain, searching for food, and little else.
Giving AI systems this ability was a promising area of research back in the 1990s, before large language models (LLMs) captured the world's attention. Now, that attention has returned.
There are three main approaches being explored to build models of the world. A natural starting point is AI video generators. Generating coherent video depends on simulating a coherent world: if the laws of reality change from one frame to the next, the result would be meaningless.
Such rudimentary models of the world can fill in details beyond what they are given: give them a picture of a maze and they can draw a path through it; show them a picture of hands holding a jar and they will accurately model the movements needed to open it.
Project Genie is the culmination of this approach. Its usefulness becomes clear when imagined in combination with another AI, for example a sales robot, that is trying to learn how to act in the physical world.
The billions of hours of training data required for such a task would be much harder to collect from the real world than from a model that can simulate the environment. And if the simulations are accurate enough, the system can use this data to train itself.
However, even the most realistic video in the world can't capture every detail that a human would notice. For example, the broken refrigerator in the back of the store that's causing the fresh fish to rot isn't captured by the camera, nor is the smell that accompanies it.
Even objects that are not directly visible remain beyond it. If the content of a hallway in a store is generated, for example, the adjacent hallways do not exist for the model until the user enters them. This makes it more difficult to simulate complex environments or allow multiple users to move around in the same model.
Another approach to building models of the world aims, therefore, to create full three-dimensional environments rather than two-dimensional simulations. Fei-Fei Li, a computer scientist at Stanford University, is leading an approach she calls spatial intelligence.
According to her, models of the world must be interactive, multimodal (capable of interpreting different demands) and sustainable.
Video-based systems can overcome the first two hurdles, but struggle with the third. Project Genie, for example, runs for a maximum of 60 seconds before its simulations start to break down.
Dr Li's startup, World Labs, has built a world model called Marble, which can create digital versions of three-dimensional worlds that are internally consistent and complete.
This means that it is possible, for example, for several users to be inside the same world at the same time. Furthermore, spaces are not created from scratch every time the user looks around; rather, they are created in their entirety from the start.
World Labs is offering its product to architects, who can use it to imagine a space and explore it virtually before sending it to a 3D printer.
Yann LeCun, former lead AI scientist at Meta, thinks that models of the world can be built in a different, less direct way. For him, focusing on real spaces is a distraction.
Ultimately, many AIs will have to navigate virtual labyrinths like human resources systems or legal documents, not just physical spaces like stores. He believes that equipping AI with the tools to consistently model both types of environments is an important step toward making it useful.
According to him, an AI can use a large language model to interact with such a model of the world and help it perform tasks, whether in the real world or on a computer.
Kjo qasje, e quajtur Joint-Embedding Predictive Architecture (JEPA), do t’i lejonte një IA të simulonte veçori komplekse të botës reale. Modelet ekzistuese të botës përqendrohen në atë që do të ndodhë menjëherë, dhe jo në ngjarje që mund (ose mund të mos) ndodhin në një të ardhme më të largët.
Njerëzit mendojnë përpara gjatë gjithë kohës: duke vlerësuar motin përpara se të vendosin nëse do të dalin nga shtëpia me një çadër; duke marrë parasysh rrezikun e vonesës në një takim të rëndësishëm kur zgjedhin cilin tren të kapin; dhe kështu me radhë.
E rëndësishmja është se këto vendime mund të merren shpejt, pa qenë nevoja të vizualizohet çdo sekondë e ditës. Modelet aktuale të botës nuk kanë një mekanizëm të tillë parashikimi.
Dr LeCun ka eksploruar potencialin e një sistemi JEPA që nga viti 2022 dhe në nëntor 2025 ai u largua nga Meta për t’u marrë me këtë problem me kohë të plotë. Startup-i i tij, Advanced Machine Intelligence, planifikon t’i kthejë idetë e tij në realitet, duke filluar me një partneritet me Nabla, një startup në teknologjinë e shëndetit. Ai thotë se synimi është një sistem që përdor modelin e vet të botës për të përcaktuar “se cili rend veprimesh do të realizojë në mënyrë optimale një detyrë që unë i caktoj”.
Por çfarë nëse këto qasje të ndërlikuara janë të tepërta? Nëse sistemet ekzistuese të IA gjeneruese tashmë mund të bëjnë gjëra të dobishme në botën reale, ndoshta ato tashmë përmbajnë një lloj modeli të botës brenda tyre.
Kjo është pikëpamja e Ilya Sutskever, bashkëthemelues i OpenAI, dhe e shumë prej ish-kolegëve të tij që ende punojnë në laborator. Trajnimi i një modeli të madh gjuhësor, tha ai në vitin 2023, nuk është gjë tjetër veçse “të mësosh një model të botës”.
Kompresimi i gjithë informacionit të përmbajtur në internet në disa qindra gigabajt numrash është i mundur vetëm nëse një sistem “mëson” parimet themelore që qëndrojnë pas atij informacioni.
Një këndvështrim i ri fantastik
Ka disa tregues që ai mund të ketë të drejtë. Në vitin 2023, një model gjuhësor i trajnuar me një listë lëvizjesh në lojën Othello u tregua se pasqyronte gjendjen e tabelës brenda rrjetit të vet nervor, edhe pse nuk kishte parë kurrë një tabelë Othello dhe nuk i ishin mësuar rregullat e lojës.
Ishte një përfaqësim aq i detajuar sa studiuesit mundën të identifikonin pjesë specifike të rrjetit nervor që ruanin ngjyrën e gurëve individualë. Kjo do të thoshte se ata mund të bënin ndërhyrje të veçanta për të ndryshuar perceptimin e modelit për lojën, një nivel kontrolli i paprecedentë mbi llogaritjet e një modeli të madh gjuhësor.
Modelet më të mëdha gjuhësore ka të ngjarë të kenë modele edhe më komplekse të botës brenda tyre, nëse studiuesit do të arrinin t’i gjenin ato. Anthropic, një laborator IA, ka udhëhequr kërkime mbi “interpretimin” e modeleve të saj Claude, duke gjetur grupe neuronesh artificiale që korrespondojnë me gjithçka, nga ndjenjat e fajit deri tek ura Golden Gate.
And interfering with these structures, as in the example of Othello, causes corresponding changes in the subsequent behavior of these models.
This suggests that the systems are not simply stringing together words: they have a consistent understanding of the physical features of the real world, which they use to answer questions. This sounds a lot like what one would expect from an internal model of the world.
Not everyone agrees. The great linguistic models, argues Dr Li, are simply “wordsmiths in the dark.” The ability to use language to describe the world, she says, does not mean they have a grounded meaning in its reality.
Like a student who has only read about a foreign country, there is a piece of missing knowledge that cannot be filled by books alone, she says. Whatever approach proves most effective, one thing is clear: Artificial Intelligence is ready to visit the real world./monitor.al
