When AI Joins the Dev Team

The Six-Week Challenge

“Put frustrated minds in a single room, give them the tools, and you will see something groundbreaking.” That’s what I tell myself as I glance at the clock, already ticking past the four-minute mark since the meeting was supposed to begin. We’re in the Turing room, a sparse, modern meeting space named after one of the pioneers of computer science. Tweet that opening line; you’ll find it relevant soon.

Beside me sits Marco, his eyes shifting between his phone and the door, which we’ve left half-open in anticipation of Ayse’s arrival. Should I send a Slack message? Cancel the meeting? Just then, I see a flurry of dark curls speed past the doorway.

“Ayse?” I call out, though perhaps not loud enough. She’s already disappeared from view.

I stand up and walk toward the door. Marco, curious and obviously not wanting to be left out, follows suit. We both stand at the threshold, peering down the labyrinthine corridor of the office.

Finally, I spot Ayse at the far end of the corridor. Relieved, I wave to her. She waves back, lengthening her strides as if she’s in the final stretch of a marathon.

When she reaches us, she shakes her head, almost in disbelief. “I thought the Lovelace room was on the other side,” she says, her tone laced with a mix of apology and exasperation.

We enter the room with a sign above the door saying “Turing.” I don’t say a word.

Ah, the trials of navigating both code and corridors. It may just be the starting point we need. After all, the best melodies often begin with a bit of dissonance. Tweet that too, by the way. Shall we get started?

Managing an engineering team is like being the conductor of an orchestra where everyone is a soloist. Talented, yes, but also set in their own ways. I’ve been sensing the low hum of frustration for a while now, like a dissonant chord in an otherwise harmonious melody. Time to bring in something new—perhaps a touch of AI—to reinvigorate the team and solve some lingering issues.

I asked my tech leads who they’d recommend for this new, exciting, yet risky pilot project. Their choices? Marco and Ayse. Marco is what you’d call vintage curiosity wrapped in the guise of a software developer. Ayse? She’s a modern-day renaissance woman juggling code and family life with apparent ease. Well, sometimes dropping a ball.

We’re all settled into the Turing room now. With a ten-minute delay, I start the kickoff.

“Alright, listen up, Marco and Ayse. Your frustrations haven’t fallen on deaf ears, I’ve heard them, loud and clear. You’ve got six weeks to find a fix. But not just any fix—a solution that’s going to make life easier for everyone on all three teams in our department.”

I pick up a dry-erase marker and write “6 weeks” boldly on the whiteboard, circling it for emphasis.

“Feel free to use our cloud infrastructure, within reason, of course. And here’s the cherry on top: you’ll have access to a Large Language Model to help you out.”

Marco: “What about sensitive data and intellectual property related to our source code? We can’t just feed them into an LLM.”

“No worries, Marco. This is a private model, brought to you by a startup run by a friend of mine. Our intellectual property will be safe and sound.”

Ayse: “I was thinking we could use the AI to tackle Vladimir’s Revenge.”

Marco and I exchange glances.

Ayse: “Oh, sorry, that’s what we’ve been calling this particularly troublesome part of the legacy ORM. It’s been a nightmare to debug and maintain.”

Marco: “Large-scale refactor sounds tempting, Ayse, but you’re not thinking about the downsides. What if the AI introduces new, harder-to-spot bugs into the code? We might fix Vladimir only to invite Rasputin to the party.”

Now we’re getting into the crux of it. Ayse, the ever-optimistic problem solver, meets Marco, the cautious guardian of the codebase. It’s like watching a symphony warm up. Cacophony hints at the music yet to come. Tweet this.

The Skeptic

Six weeks later, the big day arrives. The large meeting room, known as the “Tesla Room,” is buzzing with anticipation as people trickle in. Developers, tech leads, and project managers—about 20 of us in all—are gathering around the U-shaped table, setting up laptops and chattering about what’s to come. “What did Marco and Ayse come up with?” is the question on everyone’s lips.

Marco takes the stage first, laptop in hand. He tries to connect it to the projector, fiddling with cables and ports. No luck. “Demo effect,” someone calls out jokingly from the back, breaking the tension. Marco gives a half-smile, unplugs and replugs the cable, and voila! The projector springs to life.

The screen lights up in an eye-catching blend of sepia and vibrant magenta, a stark visual representation of the Marco-Ayse partnership. You can almost hear the collective “ooh” in the room. The user interface is deceptively simple, yet intriguing.

On the left, a large text box set against a sepia background is labeled, simply, “Your Thoughts.” It seems almost like an invitation to a bygone era, beckoning you to pour out your most complex algorithms or perplexing bugs.

Adjacent to it, on the right, a much larger text area stands out against a magenta backdrop. This one is labeled “AI Insight.” If “Your Thoughts” was an invitation, then “AI Insight” is the party itself—vibrant, dynamic, ready to engage.

Marco takes a moment to let the audience absorb the UI, then moves his cursor into the “Your Thoughts” box. “Let’s put it to the test, shall we?”

He types, “Give me an overview of our systems’ health.”

The words appear on the sepia box in real-time, each keystroke echoing slightly through the speakers connected to Marco’s laptop. As soon as he hits the “Enter” key, the magenta “AI Insight” box springs to life.

Nodes pop up on the screen, each one labeled with its name, average response time in the last hour, and uptime percentage. The crowd leans in, eyes scanning the data points. All the nodes are shaded in green—except for one, which is highlighted in yellow.

“Here you go,” the AI writes in a simple, straightforward sentence at the bottom of the “AI Insight” box. “All systems are running smoothly. However, Node-23 shows a slightly higher response time. No immediate action required, but might be worth checking in an hour.”

The room is still, but you could almost hear the gears turning in everyone’s minds. The AI’s judgment, delivered so succinctly, mirrors the collective assessments we often spend ten minutes reaching in team meetings. Marco turns to face the room.

“This is just a morning routine. It’s time to work.” He says while typing, “What shall I work on today? Consider my meetings and deadlines.”

The text Marco types appears in the sepia box, that rustic canvas awaiting a response. After a heartbeat, the magenta area beside it comes to life. A list materializes:

“Options for Today:

1. Tackle Bug #345: ‘Memory Leak in Payment Gateway’ - Time Estimate: 3-4 hours
2. Work on Feature ‘Dynamic User Preferences’ - Time Estimate: 5-6 hours
3. Review Pull Request #67 - Time Estimate: 30 minutes
4. Plan and Document Next Sprint - Time Estimate: 2-3 hours
5. Mentorship Session with Junior Dev - Time Estimate: 1 hour

Available time before next meeting: 4 hours.”

Below the list, another sentence appears, crystalline in its directness: “Based on your available time and priority settings, it is slightly better to focus on fixing Bug #345.”

A ripple of whispered commentary goes through the room. This is what many had been hoping for—an AI assistant that not just understands the nuances of software development but also factors in the all-too-human aspects of time and priority. Marco nods at the screen and then turns back to the audience.

An older team member, Jim, who’s been with the company since its dial-up days, can’t contain himself. “Well, that’s nice, but all you’ve done is hook up to Google Calendar and Jira through their APIs. That’s Programming 101.”

A murmur of agreement ripples through the room, and Marco’s face tightens for a moment.

Memory Leak

Before Marco can find the words, Ayse steps in. “Absolutely, Jim. It’s like cooking, isn’t it? We all have access to flour and sugar, but wait until you taste the cake.”

A knowing smile crosses her face as she continues, “And let’s not forget, we also had to make this ‘cake’ compatible with AWS, which, as some of you might know, is not exactly a sprinkle of sugar on top. Now, shall we move on to the actual debugging capabilities of our tool?”

The room’s atmosphere shifts subtly; the skepticism morphs into curiosity. Ayse gestures toward Marco. “Now that we’re all on the same page, let me hand it back to Marco. He’ll show you how we tackle something we all love and dread: bug fixing. Marco, why don’t you show them what happens with Bug #345?”

Marco regains his composure, happy for the assist from Ayse. He moves his cursor back to the “Your Thoughts” text box and types, “What’s the best approach for fixing Bug #345?”

The “AI Insight” area starts filling with text. “You have 3 hours before your next meeting. You could either work on developing the Feature ‘Dynamic User Preferences’, which would take approximately 5-6 hours, or you could address Bug #345, estimated to take 3-4 hours. Considering the impact and severity of Bug #345, it’s advisable to fix it first.”

Marco turns back to the crowd. “See, it not only provides a recommendation but also considers our workday constraints. It’s more than just connecting APIs; it’s about making smart, efficient choices that benefit the whole team.”

Jim, the skeptic from before, leans back in his chair, visibly impressed but not fully converted. Ayse catches Marco’s eye and gives him a nod. It’s clear their ‘cake’ has more layers yet to be revealed.

Marco shifts his attention back to the screen. “But before we dive in to fix the bug, let’s not forget the crucial step: triage.” He types, “Is this really a bug?”

The “AI Insight” section begins to populate with text once again. “Related Issues: #220 ‘Payment Gateway Timeout’, #312 ‘Unexpected Behavior in Payment Process’. Relevant Documentation: ‘Handling Memory Allocation in Payment Gateway.’ Conclusion: Given the evidence and related issues, this is indeed a bug related to a memory leak in the payment gateway.”

He pauses, letting the information sink in. “The AI helps us to triage by pulling data not just from JIRA, but also by understanding the context within our documentation and related issues. It saves us the initial legwork, allowing us to focus on what we do best—solving the problem.”

Marco looks around the room; even Jim seems to be leaning in a bit more, his skepticism giving way to what might just be interesting.

“Now, let’s get to fixing it, shall we?” Marco grins, clicking back into the “Your Thoughts” text box to type his next query.

Ayse meets his eye, and they share a moment of silent triumph. Their six weeks of collaboration, tension, and ceaseless coding are beginning to pay off, one “AI Insight” at a time.

Jim raises his hand, clearing his throat before speaking. “This seems like a problem that’s easier to diagnose since it happens within the system. What about bugs that involve user interaction?”

Ayse leans forward, intrigued. “That’s a good point, Jim. What kind of bug are you thinking of? Something you’d like to see?”

Jim senses the shift in the room; the atmosphere is more receptive now, less skeptical. “Well, I’ve been wrestling with this one bug in the webshop module for a while.”

“Which bug?” Ayse probes.

Jim doesn’t hesitate. “Bug #666,” he says, almost with a tinge of weariness. “It’s an evil one, let me tell you.”

A few colleagues nod in agreement.

Ayse chuckles, jotting down the bug number. “Well, if it’s as diabolical as its number suggests, then it sounds like the perfect candidate for our next demonstration.”

The room’s collective curiosity peaks, perhaps none more so than Jim’s. Could this AI tool that Ayse and Marco have been working on really make a dent in something as ‘evil’ as bug #666?

The Devil’s Bug

Marco returns to his laptop and types into the “Your Thoughts” field, “Show what we know about bug #666.”

The “AI Insight” section springs to life with a quick summary. “Bug #666: ‘Webshop Module Failing on Checkout’. Status: ‘Reproduce’. Previously assigned to three other engineers. Currently assigned to Jim for the last 3 days.”

Marco turns back to the room, meeting Jim’s eyes. “According to this, the bug is currently in the ‘Reproduce’ phase. So, the next step is actually to reproduce the issue. Seems like it’s been quite the hot potato.”

Jim nods, a blend of relief and trepidation on his face. The information is spot-on, and it seems like the AI has laid out the challenge for what it is—a puzzle that’s been passed around, but one that needs solving, fast.

Marco looks around the room, his gaze sweeping over the faces of his colleagues. “So, how do we go about reproducing this bug?”

The room falls quiet. It’s as if everyone has suddenly discovered a deep interest in their notepads, or their cups of coffee, or the floor tiles.

With a half-shrug, Marco turns back to his laptop. “Well, if no one knows, let’s ask our friend here.” He types, “What are the steps to reproduce this bug?”

Once again, the “AI Insight” section comes to life.

“To reproduce Bug #666, the following strategy is recommended:

1. Find a screen recording of the bug.
2. Identify the user and when the incident occurred.
3. Locate the specific request in the logs that triggered the bug, along with a few prior requests from the same user leading up to it.
4. Consult the monitoring system to see if any anomalies were detected around the same time for that user.
5. Write a test that replays the original request.
6. Run the test in a local environment to verify.”

Then a new line appears, “Would you like to proceed with this strategy?”

Marco looks up from the screen, clearly pleased. “So, what do you think? Shall we give it a try?”

The room, which had been so tentative before, now buzzes with renewed energy.

Marco smiles as he types “yes please” into the “Your Thoughts” text box. “I find it’s always good to be polite, even with machines. Believe it or not, I feel like I get better answers that way.”

The room chuckles softly. The screen in the “AI Insight” box updates, showing a URL that leads to a JIRA ticket. As the audience watches, an image starts to load on the screen. Marco notes the loading time, “Even in the age of AI, we still can’t escape the loading bar, can we?”

When the image finally fully loads, an arrow pointing to the right indicates that it’s actually a video. Marco remarks, “See? Even the AI can’t automatically click a video for us. Still need good old human interaction for that one.”

He clicks on the arrow to play the video. It starts off showing an ordering form on a website. A mouse pointer is seen hovering over various fields, eventually moving to click the “OK” button. Instantly, an error message pops up on the screen.

As the video plays on screen, the “AI Insight” box continues to populate with text. First, it shows the timestamp of when the error occurred. Marco interrupts, “Well, that’s easy; it should be right there in the JIRA ticket.”

Next, the AI lists the database user ID associated with the error. Ayse chimes in playfully, “Easy peasy, it’s just a single SELECT statement from the database.”

Finally, lines from the access log start to appear in the “AI Insight” box.

2023-09-29 09:32:12 [INFO] GET /webshop/item/1234?user_id=5678 HTTP/1.1 200 OK
2023-09-29 09:32:17 [INFO] POST /webshop/cart/add?user_id=5678&item_id=1234 HTTP/1.1 200 OK
2023-09-29 09:32:21 [INFO] GET /webshop/cart/view?user_id=5678 HTTP/1.1 200 OK
2023-09-29 09:32:27 [ERROR] POST /webshop/cart/checkout?user_id=5678 HTTP/1.1 500 Internal Server Error

Marco begins, “That’s also easy becau—” He pauses, squinting at the text. The room holds its collective breath, waiting for him to complete his thought. After a moment, he spots the user ID appearing repeatedly in the URLs listed in the access logs. “Ah, there it is. The user ID is actually part of the URLs in the access logs.”

As Marco spots the repetition of the user ID in the URLs in the access log, the “AI Insight” text area starts to update again, this time with code.

“Let’s try to reproduce this bug locally,” the AI writes. “Here’s a Javascript test suite that should mimic the sequence of actions taken by the user in the video. This test suite uses Mocha and Chai for testing and assumes based on the source code,that the server-side functions serving these endpoints are named getItem, addToCart, viewCart, and checkout.

const chai = require('chai');
const expect = chai.expect;

// Importing the functions that serve each endpoint
const { getItem, addToCart, viewCart, checkout } = require('./webshopController');

describe('Webshop Endpoints', function() {
  let userId = 5678; // replace with the actual user ID
  let itemId = 1234; // replace with the actual item ID

  it('Should get an item', async function() {
    const result = await getItem(userId, itemId);
    expect(result.status).to.equal(200);
  });

  it('Should add an item to the cart', async function() {
    const result = await addToCart(userId, itemId);
    expect(result.status).to.equal(200);
  });

  it('Should view the cart', async function() {
    const result = await viewCart(userId);
    expect(result.status).to.equal(200);
  });

  it('Should attempt to checkout and receive an error', async function() {
    try {
      const result = await checkout(userId);
    } catch (error) {
      expect(error.status).to.equal(500);
    }
  });
});

A beat passes as Marco glances at the room. “Well, I’d say that’s pretty comprehensive for a test suite aimed at bug reproduction, don’t you think?”

Jim looks impressed, “Ah, so it actually analyzed our codebase to map out the functions that serve these routes. That’s a level of automation that’s not just about stringing APIs together.”

The screen comes alive once more, grabbing everyone’s attention. The text starts to appear in the “AI Insight” box:

[AI Insight Console]
-----------------------------------------------
Spinning up Docker instance with environment...
> docker-compose up -d
Instance ready.

Running JavaScript test to reproduce bug #666...

-----------------------------------------------
> jest reproduce-bug-666.test.js
 PASS  ./reproduce-bug-666.test.js
  ✓ Reproduce sequence of actions leading to bug (5ms)

Test passed.

Bug #666 has been successfully reproduced.

Awaiting further instructions.

Marco can’t help but smile as he turns to face the audience.

Who Needs Coffee?

The atmosphere in the room has changed. From initial skepticism, the mood has shifted to one of curiosity and even excitement. Jim, who was initially critical, raises his hand, now looking more intrigued than skeptical.

“Can the AI trace back and tell us which commit introduced the bug?” Jim asks, his tone tinged with curiosity rather than doubt.

Ayse fields the question. “Well, in theory, yes. Git history would allow us to trace back each commit to see where the issue originated. We could even load up a database snapshot corresponding to earlier dates. However, that would be a rather time-consuming exercise.”

Just then, Tasha, the junior engineer known for her no-nonsense approach, chimes in. “All this is fascinating, but can the AI actually help us fix the bug? Or perhaps even implement new features?”

Marco looks at Tasha and then around the room. “That’s how far we’ve gotten—identification, triage, and reproduction. The AI hasn’t yet moved on to bug fixing or feature implementation. But think about the hours we’d save with what’s already been done. That’s more time for creative problem-solving and less for the mundane.”

I take this moment to stand up, clearing my throat to get everyone’s attention. “I want to commend Marco and Ayse for this incredible work. It’s a first step, but what a promising one! We’re always on the lookout for innovators and curious minds. If what you saw today piques your interest, consider this an open invitation: you’re all welcome to join the AI DevEx team.”

There is one more thing I want to add. “Think of AI not merely as a tool,” I say, “but as a new team member. AI, the newest junior on our DevEx team, may not bring the coffee. But who needs coffee when we have AI.”

Tweet this.

[Next story in the series: Pair-Programming with AI: A Tale of Man and Machine]