The rise of agentic AI: designing for a new era
Pattern matching AI is giving way to a new era of agentic AI, powered by reasoning, goal-oriented behaviour and world models.
Apologies in advance to our potential AI overlords and bird fanciers, but it’s easy to view current AI as being more like a sophisticated parrot, mechanically reflecting what it has learned without genuine understanding. Some find this criticism extremely unfair, especially given the incredible progress we’ve made, and while being slightly reductive, there is some truth to this considering what’s coming. For genuine understanding to occur, AI models will need to have the ability to interact with the physical world and learned experiences. As Andy Clark, a prominent voice in embodied cognition, says, “minds are not simply in the head, but extend into the world.”
For designers and engineers, this all raises interesting questions. How can we design for something that lacks genuine understanding? Should we focus on highlighting these limitations to the user, or should we try to create an illusion of comprehension? Will users disengage if there is the slightest sense of distrust? The answer might lie in how we approach the next stage of AI development.
AI, unlike previous technological advancements, has highlighted the exciting yet utterly exhausting nature of constant change. Every day brings something new, demanding our attention. We might not always fully understand it, we, but continuously learning and trying to adapt to these changes will be our ongoing responsibility.
AI agents in action
Agentic AI represents a big step forward. These systems do not passively respond to commands but can set goals, make plans, and take action. They combine LLMs (Large Language Models) with other AI techniques, allowing them to not only understand and respond to user requests but also plan and execute complex tasks. Think of how an AI agent can write a marketing campaign and then, and this is the big difference with current models, decide where to launch it, monitor it, and tweak it based on what’s happening in the world.
The fundamental difference between earlier models and agentic AI lies in their capacity for autonomous action and learning. Current LLMs are essentially reactive, responding to immediate prompts based on pre-existing knowledge. They lack the tools, memory, and reasoning necessary for higher-level cognitive processes like critical thinking and problem-solving. Now, these agentic AI systems are more sophisticated. They can create goals, strategise, and use many tools to achieve complex tasks. They can learn from past interactions, user preferences, and even self-correct based on feedback.
This is AI with initiative and a real purpose. Designing for this type of system varies from what we currently design for because of advancements in reasoning capabilities and goal-oriented behaviour.
Advanced reasoning
Chatbots use pattern recognition in the way they work, yet these new agentic models have evolved to using a reasoning layer, which involves decision-making. They can analyse complex scenarios and choose the best course of action. Moving from a ‘pattern matching’ layer to the ‘reasoning’ layer is important, as it allows the AI system to understand and respond to our needs in a more nuanced and balanced way. It feels like it’s actually thinking, though it can take some time to do this, depending on the task. At a simple level, what we see are AI tools that can not only generate variations of a task but also understand the user’s goals and suggest improvements.
Goal oriented behaviour
AI agents, as we will call them, have objectives and can break down complex goals into smaller tasks, strategize, and execute these independently. They are more than LLMs, which are the basis of ChatGPT. They are systems that combine several key components, capable of making and acting upon decisions by thinking slow. Current LLMs think fast. This “thinking fast and slow,” popularised by Daniel Kahneman, is the biggest difference. Designers will need to use this goal-oriented behaviour to create experiences that assist users in getting things done easily, whether it’s planning a trip, designing a presentation, or finally tackling that long-brewing novel.
What is happening under the hood is very interesting and highlights the fluid and difficult nature of what we are trying to design for. An AI personal assistant might book your entire holiday based on a simple request like, “Give me ideas for a trip to Italy next summer”, but so many interesting things are going on.
Let’s break down how an AI agent might tackle a task.
Step 1: Goal setting and planning
I provide the AI agent with a request to “find the best place for a holiday location in Italy for June”.
The agent deconstructs this goal into sub-tasks, such as identifying my preferences, gather information on Italy in June, find suitable locations, and present options with relevant details.
Step 2: Analysing and reasoning
The agent analyses the request and past interactions to understand my preferences. It connects to external knowledge bases, APIs, and perhaps even use google_search to gather information on Italy in June.
It then uses its reasoning abilities to match my preferences with destinations, considering factors like weather, how I hate crowds, and local events. Finally, it presents the best options in a clear and informative way.
Step 3: Learning and monitoring
The agent monitors my feedback, clicks, questions, explicit comments, and learns from it. Feedback is stored in its memory and used to refine its understanding of my quirks and preferences. If I then ask a follow-up question like “Show me more about the Amalfi Coast” the agent adapts and provides me with more details.
The world model is next
Even with these advancements, something crucial is still missing: a deep understanding of the world. We must look ahead to where world models come in. These are internal representations of how the world works, allowing AI to predict, reason, and understand cause and effect. Some industry insiders, such as Meta’s Yann LeCun, suggest this might not happen for another 10 years, but others think it’s around the corner.
We humans build these world models for ourselves through “sensorimotor learning,” which means interacting with the physical world and learning from our experiences. We learn about gravity by dropping things, about solidity by bumping into them. Our understanding is grounded in our physical interactions with the world. In his 1942 book, philosopher Maurice Merleau-Ponty said that “the world is not what I think, but what I live through”, his point being that this embodied experience is fundamental to how we perceive and interact with the world.
LLMs primarily learn from text. They lack this real-world, embodied experience. It’s like trying to learn to ride a bike by reading a manual; without the wobbling and occasional falling on your backside, you’ll never understand how to do it properly.
To bridge this gap, sensorimotor learning is being integrated into AI systems by training AI agents in virtual worlds where they can interact with objects and experience the consequences of their actions. Further on, multimodal learning, where a diverse data set of images, sounds, and sensor readings, will create a better understanding of the world.
“We are creatures whose minds are inextricably interwoven with our bodies and the world around us.” This is central to understanding how intelligence emerges, both in humans and potentially in AI. This means moving beyond a mere imitation game to genuine understanding and action.
It’s a journey that takes us from AI that can write like a human, to AI that can plan and act on complex tasks and finally understand and act in the world like a human. It’s about moving, as Clark suggests, from brains in vats to minds that are truly embedded in the world. This AI train is moving faster and faster down the track, and we will need to hold on tight and stay informed. If we become complacent and allow ourselves to be mystified by the black box, we risk missing an opportunity to shape its development and ensure it really serves our needs.
