Most AI-powered robots effectively stop learning once they are deployed. Researchers at Skylark Labs, working with academics from Carnegie Mellon University and the University of California, Berkeley, have developed a new approach designed to enable robots to continue learning from successful experiences in the real world – without retraining their underlying AI models or forgetting what they already know.
The researchers call the new architecture Continual Field-Adaptive Models, or CFAMs, and describe the technology in a newly published research paper, Continual Field-Adaptive Models (CFAMs) for Post-Deployment Physical AI.
The central problem addressed by the research is becoming increasingly important as robots move from controlled laboratories and factories into less predictable environments.
Modern robot-learning models can require enormous quantities of training data. Once deployed, however, a robot inevitably encounters situations that differ from those represented in its training data – an object may be positioned differently, a load may be heavier, terrain may change or an obstacle may appear where one was not expected.
Retraining a large AI model every time this happens may be impractical, particularly when a robot is operating remotely with limited computing power or without a reliable connection to a data center.
Updating a model in the field also introduces another problem: catastrophic forgetting, in which learning new information causes an AI system to lose some of its previously acquired capabilities.
CFAM is designed to address both problems by separating stable, previously learned capabilities from a faster form of memory that can continue growing after deployment.
Giving robots a long-term and short-term memory
The researchers take inspiration from the complementary learning systems found in the human brain.
CFAM contains three relatively stable components that the researchers call the Sensor cortex, Reasoning cortex and Action cortex. Together, they allow the machine to perceive its surroundings, decide what needs to be done and execute the appropriate physical action.
These components are trained before deployment and subsequently frozen.
A separate fast-learning component, called the Capsule Field, stores new experiences encountered during operation.
The researchers compare this arrangement loosely with the relationship between the neocortex and hippocampus in the human brain: one provides relatively stable long-term knowledge, while the other allows new experiences to be encoded rapidly.
The individual units stored within the Capsule Field are called Competence Capsules.
Instead of changing millions or billions of parameters in the robot’s underlying AI model, CFAM can record a useful new experience as a capsule and retrieve that information when a similar situation occurs again.
Crucially, this process takes place on the robot itself and does not require conventional gradient-based retraining.
A robot that expands its abilities by working
One of the more interesting ideas behind CFAM is that a robot’s useful operating range can gradually expand simply through successful operation.
The researchers describe this as extending the robot’s “competence envelope”.
Imagine, for example, that a robot has learned to grasp an object in a particular position. If the same object subsequently appears farther away or at a different orientation, the robot does not necessarily need to learn the task again from scratch.
CFAM uses what the researchers call a Geometric Residual Transform, or GRT, to adapt the stored movement to the geometry of the new situation.
As the paper puts it, the skill is effectively “re-aimed, not re-learned”.
If the robot successfully performs the task near the edge of its existing competence, that successful experience can then be stored as another Competence Capsule.
The next time the robot encounters a similar situation, it has that experience available directly.
Over time, therefore, the range of situations covered by its stored skills can expand.
The approach is applicable to more than robot manipulation. The researchers tested CFAM across five different types of physical system: a manipulator, quadruped, humanoid, quadrotor and off-road vehicle.
For a quadruped, for example, the variation might involve encountering different terrain. For an autonomous vehicle or drone, the relevant geometry can involve routes, waypoints, terrain and obstacles rather than an object being manipulated.
Learning from less data
The researchers also report significant reductions in the amount of training data required before deployment.
Their experiments use an in-house Skylark Labs dataset containing more than 2.6 million trajectories across five physical platforms. The researchers say CFAM and the standard-policy baselines used for the physical-platform comparisons were trained on the same dataset.
According to the paper, CFAM reaches the operating point achieved by a standard policy trained on the full prior-training dataset while using only 40 percent of that data, equivalent to 2.5 times fewer prior-training trajectories.
The researchers then examined what happened after deployment as the system encountered variations that were close to, but outside, its previous training distribution.
Autonomously capturing verified successful cases increased action success from 74.0 percent to 87.9 percent – an improvement of 13.9 percentage points.
CFAM also showed substantially less forgetting in sequential simulation experiments.
The researchers report backward transfer of -0.5 percentage points, compared with -11.4 percentage points using LoRA, a widely used method for efficiently adapting AI models.
Physical AI beyond the data center
The ability to learn locally becomes particularly significant when robots operate somewhere a cloud connection or large computing infrastructure cannot be assumed.
The paper focuses heavily on mission-critical applications in defense, public safety and hazardous environments.
Examples include inspecting unstable mine shafts, handling suspected explosive devices, searching earthquake-damaged structures and operating inside radiation-contaminated facilities.
The researchers also point to much more remote possibilities, including deep-ocean operations and planetary exploration, where communications delays or the absence of suitable training data make conventional cloud-dependent approaches impractical.
Similar principles could ultimately be relevant to commercial physical AI applications. Robots deployed in factories, warehouses, construction sites, farms and other changing environments routinely encounter conditions that were not represented perfectly in their original training data.
A machine capable of retaining useful experience from those encounters could potentially become more capable over its operational lifetime rather than remaining frozen at approximately the level of competence it possessed when it left the laboratory.
Skylark Labs describes CFAM as an architecture for this kind of on-device continual learning, particularly where connectivity, computing resources and training data are constrained.
Not unlimited self-learning
There is, however, an important boundary to what the researchers have demonstrated.
CFAM is not intended to enable a robot to encounter a completely unfamiliar problem and autonomously invent an entirely new skill.
The system instead addresses what the researchers call near-out-of-distribution, or near-OOD, situations.
A robot that already knows how to grasp one type of object might extend that competence to a related object. A walking robot familiar with one surface might learn to handle another related type of terrain.
But genuinely new tasks for which the system possesses no relevant skill family – what the researchers call “open-world novelty” – are outside the scope of the current work.
That distinction makes CFAM less a system for robots that can spontaneously teach themselves anything and more an architecture for machines that can build upon what they already know.
For physical AI, that may prove to be the more immediately useful capability.
Today’s robot-learning systems can demonstrate increasingly sophisticated behavior in laboratories. The challenge is maintaining – and expanding – that competence once those machines encounter the unpredictable physical world.
CFAM proposes one way of bridging that gap: teach the robot a relatively small number of skills before deployment, allow it to adapt those skills to new situations, remember successful experiences locally, and do so without continually rewriting the AI system that provided its original capabilities.
About the researchers
The paper is authored by researchers affiliated with Skylark Labs, Carnegie Mellon University and the University of California, Berkeley.
- Amarjot Singh – Skylark Labs: Founder and CEO of Skylark Labs, whose research focuses on brain-inspired, data- and compute-efficient AI for physical and security applications.
- Tanmay R. Pancholi – Skylark Labs: Researcher at Skylark Labs and co-author of the CFAM research paper.
- Jainam Kothari – Skylark Labs: Researcher at Skylark Labs and co-author of the CFAM research paper.
- Shrirang Mahajan – Skylark Labs: Researcher at Skylark Labs working on the company’s CFAM physical-AI architecture and related continual-learning technologies.
- Ketan Bansal – Skylark Labs: Researcher at Skylark Labs and co-author of the CFAM research paper.
- Vince Nakayama – Skylark Labs: Researcher at Skylark Labs and co-author of the CFAM research paper.
- Zackory Erickson – Carnegie Mellon University: Assistant professor in CMU’s Robotics Institute whose research covers robot learning, mobile manipulation, sensing and physical human-robot interaction.
- Jeff Schneider – Carnegie Mellon University: Research professor in CMU’s Robotics Institute specializing in machine learning, autonomous systems, reinforcement learning and robotics.
- Giuseppe Loianno – University of California, Berkeley: Robotics researcher and founder of the Agile Robotics and Perception Lab, specializing in autonomous and agile robotic systems, particularly aerial robots.
- Alexandre M. Bayen – University of California, Berkeley: Liao-Cho Professor of Engineering at UC Berkeley, with research spanning control, optimization, autonomous systems and transportation.
