AI coding assistants like ChatGPT are everywhere now. They can scaffold components, generate test cases, and even debug code. But here’s the catch: they’re not senior engineers. They don’t have context of your project history, and they don’t automatically spot when the tests themselves are wrong. tests themselves are wrong In other words: treat ChatGPT like a junior dev on your team — helpful, but always needing review. treat ChatGPT like a junior dev on your team My Experience: Fixing Legacy Code Against Broken Tests I was recently working on a legacy React form validation feature. The requirements were simple: Validate name, email, employee ID, and joining date.
Show error messages until inputs are valid.
Enable submit only when everything passes. Validate name, email, employee ID, and joining date. Show error messages until inputs are valid. Enable submit only when everything passes. The tricky part? I didn’t just have to implement the form — I had to make it pass an existing test suite that had been written years ago. existing test suite I turned to ChatGPT for help, thinking it could quickly draft a working component. It generated a solution — but when I ran the tests, they kept failing. kept failing At first, I thought maybe I had misunderstood the requirements, so I asked ChatGPT to debug. We went back and forth multiple times. I provided more context, clarified each input validation rule, and even explained what the error messages should be. ChatGPT suggested fixes each time, but none of them worked. multiple times none of them worked It wasn’t until I dug into the test suite myself that I realized the real problem: the tests were wrong. wrong The Test That Broke Everything One test hard-coded "2025-04-12" as a “future date”: "2025-04-12" changeInputFields("UserA", "user@email.com", 123456, "2025-04-12");
expect(inputJoiningDate.children[1])
  .toHaveTextContent("Joining Date cannot be in the future"); changeInputFields("UserA", "user@email.com", 123456, "2025-04-12");
expect(inputJoiningDate.children[1])
  .toHaveTextContent("Joining Date cannot be in the future"); The problem? We’re already past April 2025. That date is no longer in the future, so the expected error message would never appear. The component was fine — the tests were broken. already past April 2025 tests were broken I had to dig through the logic, analyze the assumptions, and rewrite the test with relative dates, like so: relative dates // Corrected test using relative dates
const futureDate = new Date();
futureDate.setDate(futureDate.getDate() + 30); // always 30 days ahead
const futureDateStr = futureDate.toISOString().slice(0, 10);

changeInputFields("UserA", "user@email.com", 123456, futureDateStr);
expect(
  screen.getByText("Joining Date cannot be in the future")
).toBeInTheDocument(); // Corrected test using relative dates
const futureDate = new Date();
futureDate.setDate(futureDate.getDate() + 30); // always 30 days ahead
const futureDateStr = futureDate.toISOString().slice(0, 10);

changeInputFields("UserA", "user@email.com", 123456, futureDateStr);
expect(
  screen.getByText("Joining Date cannot be in the future")
).toBeInTheDocument(); This small change makes your test time-proof, so it will work regardless of the current year. Lessons Learned AI will follow broken requirements blindly - ChatGPT can’t tell that a test is logically invalid. It will try to satisfy the failing test, even if the test itself makes no sense.


Treat output like a junior PR - ChatGPT’s suggestions were helpful as scaffolding, but it struggled to see the root cause. I had to step in, dig through the legacy code, and analyze the tests myself.


Tests can rot too - Hard-coded dates, magic numbers, or outdated assumptions make test suites brittle. If the tests are wrong, no amount of component fixes will help.


Relative values keep tests reliable - Replace absolute dates or values with calculations relative to today. This ensures your tests work across time. AI will follow broken requirements blindly - ChatGPT can’t tell that a test is logically invalid. It will try to satisfy the failing test, even if the test itself makes no sense. AI will follow broken requirements blindly - ChatGPT can’t tell that a test is logically invalid. It will try to satisfy the failing test, even if the test itself makes no sense. AI will follow broken requirements blindly - Treat output like a junior PR - ChatGPT’s suggestions were helpful as scaffolding, but it struggled to see the root cause. I had to step in, dig through the legacy code, and analyze the tests myself. Treat output like a junior PR - ChatGPT’s suggestions were helpful as scaffolding, but it struggled to see the root cause. I had to step in, dig through the legacy code, and analyze the tests myself. Treat output like a junior PR - Tests can rot too - Hard-coded dates, magic numbers, or outdated assumptions make test suites brittle. If the tests are wrong, no amount of component fixes will help. Tests can rot too - Hard-coded dates, magic numbers, or outdated assumptions make test suites brittle. If the tests are wrong, no amount of component fixes will help. Tests can rot too - Relative values keep tests reliable - Replace absolute dates or values with calculations relative to today. This ensures your tests work across time. Relative values keep tests reliable - Replace absolute dates or values with calculations relative to today. This ensures your tests work across time. Relative values keep tests reliable - How to Work Effectively With AI Tools Give context, but don’t rely on it to reason like a senior dev.


Ask “why”, and inspect its explanations carefully.


Validate everything yourself — especially when working with legacy code.


Iteratively refine — use AI as scaffolding, but you own the fix. Give context, but don’t rely on it to reason like a senior dev. Give context, but don’t rely on it to reason like a senior dev. Give context Ask “why”, and inspect its explanations carefully. Ask “why”, and inspect its explanations carefully. Ask “why” Validate everything yourself — especially when working with legacy code. Validate everything yourself — especially when working with legacy code. Validate everything yourself Iteratively refine — use AI as scaffolding, but you own the fix. Iteratively refine — use AI as scaffolding, but you own the fix. Iteratively refine Closing Thoughts My experience taught me a simple truth: AI can accelerate coding, but it cannot replace human judgment, especially when dealing with messy, legacy code and outdated tests. cannot replace human judgment Treat ChatGPT like a junior teammate: Helpful, eager to please, fast.
Sometimes confidently wrong.
Needs review, oversight, and occasionally, a reality check. Helpful, eager to please, fast. Sometimes confidently wrong. Needs review, oversight, and occasionally, a reality check. If you keep that mindset, you’ll get the productivity boost without blindly following bad guidance — and you’ll know when to dig in yourself. 💡 Takeaway: When working with code, the human developer is still the ultimate problem-solver. AI is there to assist, not to replace your reasoning. Takeaway:

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Using ChatGPT Like a Junior Dev: Productive, But Needs Checking

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps