Teaching AI to Debug Might Beat Teaching It to Remember

This is a Plain English Papers summary of a research paper called Evolving Programmatic Skill Networks. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.

The problem: unbounded learning in open worlds

Imagine teaching a child to cook. They learn to boil water, then make pasta, then a simple sauce. But here's the trap: if they learn sauce poorly, does it destroy their ability to boil water? A human brain doesn't work that way. Neural networks trained on one task often forget previous tasks when trained on new ones. The problem gets worse as more tasks arrive, and in open-ended worlds like Minecraft where tasks never stop coming, this becomes a fundamental barrier to continuous learning.

This is the foundational tension that drives the Programmatic Skill Network (PSN) paper. Most AI agents face a harsh dilemma: they either memorize everything, creating bloated systems that don't generalize, or they learn continuously and suffer catastrophic forgetting. The trade-off seems inevitable. As an agent masters new skills, the old ones fade. But what if it didn't have to?

The paper tackles this by asking a provocative question: what if skills weren't parameters to be finetuned, but programs to be refined?

Why symbolic programs beat neural parameters

When you learn a recipe, you don't memorize neural activation patterns. You write down an algorithm: "heat oil, add onions, cook until translucent." This explicitness is powerful. If someone asks why the recipe works, you can walk them through each step. If it fails, you can debug it: "oh, the heat was too high." You can reuse the "saut[?] onion" step in ten different dishes.

Neural networks are the opposite. They're incomprehensible recipes. You can't ask why they work. When they fail, you're left guessing. And reusing learned knowledge means copying parameters, which often doesn't generalize to new contexts.

This design choice[?]that skills should be executable programs rather than learned parameters[?]cascades through everything that follows. If skills are code, then learning becomes refactoring. Failure becomes debugging. Knowledge reuse becomes writing functions. The entire framework changes.

The framework

The Programmatic Skill Network maintains a knowledge graph where each node is an executable skill (a program written by language models) and edges represent composition relationships. When the agent encounters a new task, it either executes an existing skill or synthesizes a new one. When a skill fails, it goes through REFLECT, a debugging process that traces through the execution, locates the broken step in the skill composition, and pinpoints which skill to fix.

The Programmatic Skill Network framework. The agent maintains a skill network where the hybrid planner selects or synthesizes skills; the PSN manager executes them. On failure, the skill optimizer performs fault localization, updates, and refactoring.

The framework rests on three pillars. REFLECT is structured fault localization. When a skill fails, it doesn't say "this is broken." It uses language models to trace through the execution path, identify which step in the composition chain failed, and isolate the culprit skill. This is debugging, not black-box trial-and-error. It tells the agent exactly what to fix rather than randomly mutating everything involved.

Progressive optimization with maturity-aware gating recognizes a critical insight: not all skills should learn the same way. A skill used 100 times and thoroughly validated should be locked down, resistant to change. A skill invented yesterday should be plastic, eager to adapt when discovered useful elsewhere. The system implements this by updating recently-created or uncertain skills frequently while updating stable, proven skills rarely. This prevents a newly-learned task from accidentally breaking a well-tested skill.

Canonical structural refactoring keeps the skill library clean. Over time, duplicate logic emerges. Specialized skills pile up. Refactoring extracts common sub-skills, merges equivalent programs, and collapses specializations back to parameterized versions. This happens only occasionally and is validated by rollback: if a refactoring breaks things, the agent reverts it.

These five types of refactoring emerge in practice:

Parametric coverage: A specialized skill is rewritten as a wrapper around a more general, parameterized skill.

Behavioral coverage: Duplicated logic inside a composite skill is replaced by a call to an existing reusable skill, preserving behavior while reducing redundancy.

Sibling specializations: Multiple specialized skills expose a missing higher-level abstraction that can be explicitly synthesized and reused.

Common subskill extraction: Repeated sub-operations across different skills are factored into a shared subskill, improving reuse and reducing duplication.

Duplication removal: Functionally equivalent skills are merged into a single canonical representation.

The payoff is that PSN automatically discovers and extracts abstractions that emerge from experience. When you've learned five cooking tasks and reinvented "saut[?] onions" in five ways, refactoring notices the pattern and writes a generic function. The skill library doesn't just grow larger; it becomes more structured.

How PSN avoids catastrophic forgetting

When you refactor code, you don't delete old functions. You reorganize how they're used. If the refactored version breaks something, you roll back. The skill library doesn't lose information; it reorganizes it. And because maturity gating locks down stable skills by default, new learning can't destroy them.

Skill retention rate under continual learning on Minecraft. PSN consistently preserves previously mastered skills, while Voyager (a neural network baseline) exhibits severe catastrophic forgetting as training progresses.

This graph shows the payoff. As training progresses, PSN's retention rate stays high. Voyager's crashes. The mechanism works: maturity gating keeps old skills frozen by default, preventing new learning from erasing them.

Cumulative success rate of tasks for PSN with and without maturity gating on Minecraft. The mechanism significantly improves performance.

Removing maturity-aware gating significantly hurts task success, proving the mechanism actually works.

Evidence from open-ended environments

The paper tests on two domains designed to be genuinely open-ended. MineDojo provides a rich Minecraft simulation where the agent can pursue countless objectives without a fixed curriculum. Crafter is a smaller 2D survival environment with similar openness. Neither has predetermined tasks or learning order.

Tech tree mastery on Minecraft. PSN unlocks more items and techniques over time, demonstrating continuous knowledge accumulation.

Cumulative reward on Crafter. Shorter curves indicate earlier agent death due to Crafter's survival mechanics (hostile mobs, hunger, hazards). PSN agents survive longer by effectively reusing learned skills.

The results demonstrate three capabilities: skill reuse (the agent successfully applies skills to new tasks), rapid adaptation (new tasks get solved faster as the library grows), and generalization (skills learned for one objective help solve others).

Growth of skill library over training. PSN reuses and optimizes existing skills, maintaining a compact skill repertoire compared to baselines that keep synthesizing new specialized skills.

Notice the library growth curve. PSN's slope is shallow because refactoring prunes redundancy. Baselines balloon with duplicates. The skill library becomes not just bigger but more structured.

What PSN reveals about learning itself

There's a deeper insight buried in the mechanics. In deep learning, early training involves rapid parameter changes as networks find rough solutions. Late training involves smaller, surgical updates that refine them. Skills in PSN follow the same pattern: newly-created skills change frequently (high plasticity), while mature skills change rarely (high stability).

This parallel isn't accident. It suggests something fundamental about any system that must balance stability with exploration. You need to lock down what works while remaining open to improvement. This pattern appears in human learning, biological systems, and now in symbolic skill networks. The learning dynamics mirror deeper principles about how knowledge should evolve.

The larger picture

PSN doesn't require choosing between memorization and forgetting. By treating skills as explicit programs that can be refactored, debugged, and composed, an agent builds a growing library that becomes more structured, more reusable, and more robust over time. It connects to broader work on compositional learning, like approaches to single-agent skill replacement and polymorphic abstraction, which explore how skills can be discovered and reused across different contexts.

The deepest contribution is conceptual. Teaching an AI to learn becomes teaching it to code. Skills are specifications to be refined. Failures are bugs to debug. New knowledge reuses old components. Growth means refactoring toward elegance, not just accumulation. In that shift lies something important about how intelligence actually works.