World View AI
True agentic AI capability won't happen while LLMs and word-based AI leads the charge. We need 'world view' models to allow action in space-time - beyond the confines of the 2D 'page'.
We should be planning for ‘world view’ AI. The best-next-word LLM-driven approach has - through its accessibility and patent capability - catapulted adoption of AI, however its limited scope means that it’s an initial phase of AI’s impact, and not the sole or optimal path for commerce or the customer.
In recent posts we’ve looked at the promise of ‘agentic’ commerce, the need for Sovereign AI and the definitions needed for the echt human and the authorised agent. It’s time to take a ‘world view’.
Limitations of LLMs
LLMs (large language models) are trained on, er, language. Every word, ever. Every combination. Every language. All modes (business, chat, speeches, essays, news - everything). As a result, an LLM can ‘pick the best words’, based on the best word combinations in its training base. To most humans this looks like a type of savant genius. After all, we are verbal creatures, and to see this compendious knowledge, recall, and authoring is like the fever dream of every helicopter parent!
An analogue would be a gifted pupil at school. Aces every academic subject, but lacks hand-eye coordination, can’t follow a map, can’t manipulate a scalpel, splice a gene, or build an IKEA bookcase, brew a craft beer, or splice a mainbrace. You get the picture - brains (or the appearance of braininess) only get you so far in a 3D world.
In retail, while the LLMs can write web content, analyse ad campaigns, parse data feeds, create images, they cannot talk to your WMS, work out how to get a large fridge into a 3rd floor flat with a narrow hallway and tight turns on the stairs, or talk to your energy provider or grocer of choice. For this, we need to look at “world view” AI that has an understanding of the rules of physics, niche and specialised behaviours, materials, and constraints.
Such localised and specialised knowledge corpora exist, and many have benefitted from machine learning to optimise them over many years (think weather prediction, crop management, traffic control, gene therapy), but these systems are not open access, easy to use by the layperson (nor perhaps should they be?), and there is a lack of interoperability. A case of many worlds rather than a holistic worldview.
So, let’s look at the ideas in World View AI, why it’s becoming important, who’s already doing it, and the ‘so what’ for those of us who are users rather than creator-owners of the AI.
LeCun’s critique of the LLM path
The recent resignation of Yann LeCun (the so-called “godfather of AI”) from Meta brought his critique of the direction of travel into the public domain: mainstream AI, embodied by large language models (LLMs), has hit its limits. LeCun argues LLMs are essentially sophisticated autocomplete machines—they excel at generating text and powering chatbots, but fundamentally, they don’t reason, adapt, or interact with the real world.
That this interaction is both necessary and advantageous can be seen from the results that Walmart, Amazon, and others achieve by linking insights from supply chain, demand signals, and retail systems. This article on Fortune covers AI in the supply chain. While we admire, we also realise that the tuning, training, deployment and integrations are proprietary.
Why Autocomplete Isn’t Agency
LLMs like GPT4 etc. have revolutionised how retailers deliver customer service, create content, and automate knowledge work. Shopify has a helpful and expansive run-through. However, these capabilities have limitations: they can’t simulate real environments, predict outcomes, or coordinate physical actions. This disconnect is increasingly apparent as retailers seek to optimise supply chains, automate store logistics, and create immersive customer experiences (to emulate and compete/collaborate with the Amazon and Walmart world).
Furthermore, for true agentic commerce (that’s more than an automation script for a web purchase) we need the agency to extend to the ‘real world’.
World Models: AI That Simulates Reality
Emerging AI research suggests that “world models” are the key to advancing from text-based intelligence to “situated intelligence.” Unlike LLMs, world models build richly detailed, internal representations of real or imagined environments. NVIDIA has a helpful overview of World models. They combine sensory inputs—vision, sensor data, and spatial mapping to forecast, plan, and act in dynamic settings. A recent, accessible version comes from Google’s DeepMind team with the release of Genie 3 (a tool that allows you to create realistic worlds with interactive prompts - feel the “wow”. This is as if decades of Google Maps and Google Earth, every game you’ve played and virtual worlds have all been handed to you in one go!
These systems underpin self-driving vehicles, robotics, logistics, and even virtual shopping spaces where customers interact with digital products as if in a tangible reality.
What About Digital Twins?
In case you think I’m referring to a digital representation of a single, exact reality, we should quickly divert to “Digital Twins”. Digital twins are specific, data-driven replicas of physical assets - like an exact retail store, warehouse, or entire supply chain. This allows you to run simulations on that exact setup. However, a world model can abstract, generalise and predict. Digital twins offer high-fidelity, real-time monitoring and optimisation for one store or supply chain (or nuclear reactor!) at a time.
The most advanced retail setups combine both, feeding live digital twin data into world model simulations to enable foresight, elasticity, and robust “what-if” scenario testing, as seen in the Walmart and Amazon examples above.
Domain Specialists vs Generalists
Retailers like Amazon have the data, computing power and smarts to apply machine learning to their own business - remembering that it covers selling, supply chain, warehousing, delivery and logistics and advertising - what a trove! This knowledge is made available to mere mortals via approved, Amazon-benefitting ways. An interesting blog post shows the extent of AI in use at Amazon warehouses, and then suggests that you become a client :) It’s not a generally applicable or open-access knowledge source.
At a further abstraction, a model that manages a supermarket’s logistics is useless for gene splicing or autonomous vehicles. The missing ingredient? Interoperability - standard protocols and modular architectures that let specialised AIs collaborate and share knowledge.
Novus (a no-code integration service) has set out a good approach to battle the “AI sprawl”, and this is reminiscent of the microservices/APIs/MACH-alliance approaches to ecommerce stacks. Incidentally, this week, Kelly Goetsch (of Pipe17, and one of the animators of the MACH Alliance when he was at CommerceTools) has launched the Commerce Operations Foundation that looks at how orders flow across systems in agentic commerce.
We can see the growth of deep, ML-powered knowledge corpora, the start of monolithic integrations (within Amazon, Walmart, and others), and limited service interfaces to these worlds. In parallel, there are movements to open standards for access and interchange of agents.
Movements Toward “World AI”
How are we progressing toward a “world AI”?
Three approaches of note:
- Constraining the world. through simulated, bounded environments where models can safely learn and experiment (the DeepMind example above). These deep, specific models can be used alone, but cry out to be connected to other capabilities…
- Creating general world models capable of understanding “open world” reality; LeCun’s new venture aims to build embodied models that learn and plan without human supervision. Within this article there’s a link to NVIDIA’s CEO, Jensen Huang, predicting that world models are the ‘next big thing’ after agentic AI. I’m not going to argue ;)
- Connecting smaller world models - retail AIs (plural) for supply chain, customer experience, and store operation can interact, synchronise, and adapt dynamically as a network of modular agents. Dexterity (a company that creates a ‘world view’ for robot systems) explains its approach in a blog post, and I’ve mentioned the Order Network Exchange above.
Let’s have a look at some glimpses of this future within the retail and D2C sectors today.
Examples in retail and D2C
Digital twin-driven store simulation - Carrefour’s use of Twinn Witness (formerly Lanner) modelling in their DC operations
Supply chain world models - FedEx uses predictive, world model-augmented systems to re-route shipments, balance inventory, and forecast disruptions. This article is from pre-history (2022) and shows that behind-the-scenes AI to increase capability has been extant way before the LLM boom…
Product development: Nike has used AI extensively across its supply chain, product development, and sustainability initiatives. This article gives a run-down of the main areas.
Autonomous fulfilment: Ocado and Amazon Robotics deploy world-model-guided robots to streamline picking and replenishment
Immersive shopping environments: IKEA’s 3D design tools (Kitchen Planner) and Gucci’s metaverse boutiques experiment (now over - release here) show some moves to bring physical and digital ‘space’ into the browsing and consideration processes.
Conversations for the next Board awayday
AI is on every agenda, but let’s take a view that’s not just focused on the basket and the marketing funnel - what’s our “world”?
Consider the several AI-powered initiatives we already have (IOT, shelf-edge systems, demand forecasting, logistics tracking, RFID information) and consider how these are connecting data, insights, and actions
Ask your main suppliers to suggest to you how they can enable and support information exchange and action triggers (without requiring a 3-year, multimillion-pound/dollar/euro “transformation programme”). Most will tout their expansive, finger-in-many-digital-pies credentials, but ask for a 15-minute presentation just for you. Ask them to record it as a webinar, with a deck, references, and examples. That way you can mull, synthesise and consider in advance of your board meeting (without having to sit through hours of meetings). #yourewelcome
Take note of the many emerging protocols on interoperability, and dedicate some time to monitoring them. I’ve mentioned some already, but in the foreground, we have Google's agent-to-agent protocol (A2A), Anthropic’s “Model Context Protocol” (MCP). Others will emerge in key domains, and baking in the question “are you open to [protocol x]” in supplier conversations will increase openness and interoperability.
Use your suppliers for a real-world understanding. NVIDIA has Cosmos, its “World Foundation Models” platform, and while the technically able will be able to access these via GitHub or HuggingFace, a board might like to be shown around. Also, their Omniverse… Contact your friendly local NVIDIA contact… Other giants (including Oracle, Salesforce’s eVerse AI training world) are available :)
Thinking
LLMs have brought AI to broad attention and use, and this AI, in the hands of customers, is causing ripples throughout retail. Equally, the same AI in the hands of retailers is improving, changing and challenging operations. However, this is the first stage.
As the customer considers their own agency and identity, so retailers and brands look past the marketing and conversion funnels to ‘world-views’ of their own operations (today), world-views with their partners (today and tomorrow) and soon-ish to world-views that the customer inhabits (when she’s not stuck on our websites buying things!). Boards need to have these worlds in mind, even as they hurry to exploit today’s more limited AI, and avoid getting painted into a corner, demoted to a ‘mere user’ of systems, or cut out of their customers’ whole lives.
If you own first-party data, you need to inhabit and participate in the ‘world’.
What do you think? What have I missed? Let me know!