AI Coding Tip 009 - Compact Your Context and Stop Memory Rot

Stop the memory rot!

TL;DR: You can keep your AI sharp by forcing it to summarize and prune what it remembers (a.k.a. compacting).

Common Mistakes When Coding with AI

• You keep a single, long conversation open for hours.
• You feed the AI with every error log and every iteration of your code.
• Eventually, the AI starts to ignore your early instructions or hallucinate nonexistent functions.

Problems Addressed in this Article😔

• Context Decay: The AI loses track of your original goals in the middle of a long chat.
• Hallucinations: The model fills memory gaps with hallucinations or outdated logic.
• Token Waste: You pay for the AI to re-read useless error logs from three hours ago.
• Reduced Reasoning: A bloated context makes the AI less smart and more prone to simple mistakes.

How to Solve this Problem with AI Coding Assistants

1. Restart often: You can start a new chat once you finish a sub-task.
2. Request a State Summary: Before you close a conversation, ask the AI to summarize the current decisions and plan.
3. Add Human Checkpoints: After the summary, confirm you are still on track.
4. Use Markdown Docs: Keep a small context.md file with your current stack and rules.
5. Prune the Logs: You should only paste the relevant 5 lines of a stack trace instead of the whole irrelevant 200-line output.
6. Divide and conquer: Break large tasks into smaller ones, invoking their own skills with local tokens and a fresh context.
7. Divide the responsibility: A General doesn't need to know what every soldier is doing on the battlefield.
8. Create and persist as Skill: After you have taught the AI, you should refactor the knowledge and business rules.
9. Keep an Eye on the Context Size: Most tools have visual indicators of the window consumption.
10. Use Local Persistence: Some tools allow sharing memory among agents and their sub-agents.

Benefits of this Approach

• You get more accurate code suggestions.
• You avoid divergences
• You follow the AI's train of thought.
• You spend less time correcting the AI's hallucinations.
• The AI follows your project constraints more strictly and keeps focused on your tasks

Additional Context

Large Language Models have limited attention.

Long context windows are a trap.

Many modern models offer a very large context window.

In practice, they ignore a lot of them to your frustration.

Even with large context windows, they prioritize the beginning and end of the prompt.

Reference Prompts

Bad Prompt

Here is the 500-line log of my failed build. 

Also, remember that we changed the database schema 

Three hours ago in this chat.

Add the unit tests as I described above.

Now, refactor the whole component.

Good Prompt

I am starting a new session. Here is the current state: 

We use *PostgreSQL* with the 'Users' table schema [ID, Email]. 

The AuthService`interface is [login(), logout()]. 

Refactor the LoginComponent` to use these.

More Details About this AI Coding Tip

• Type: Semi-Automatic
• Limitations:
  • You can use this tip manually in any chat interface.
  • If you use advanced agents like Claude Code or Cursor, they might handle some of this automatically, but manual pruning is still more reliable.
• Skill level: Intermediate

Related Tips 🔗

https://maximilianocontieri.com/ai-coding-tip-004-use-modular-skills?embedable=true

https://hackernoon.com/ai-coding-tip-005-how-to-keep-context-fresh?embedable=true

AI Coding Tip 010 - Create Skill from Conversation

Conclusion

You are the curator of the AI's memory.

If you let the context rot, the code will rot, too.

Keep it clean and compact. 🧹

Additional Information ℹ️

https://arxiv.org/abs/2307.03172?embedable=true

https://llmlingua.com/?embedable=true

https://www.cursor.com/blog/context?embedable=true

https://www.ibm.com/topics/ai-hallucinations?embedable=true

This AI Coding Tip is Also Known

• Context Pruning
• Token Management
• Prompt Compression

Tools Used

• Claude Code
• Cursor
• Windsurf

This article is part of the AI Coding Tip series.

https://maximilianocontieri.com/ai-coding-tips?embedable=true