Making Code Make Sense: The Challenge Of Comprehension In AI-Driven Programming

Written by pairprogramming | Published 2025/08/13
Tech Story Tags: large-language-models | ai-assisted-programming | github-copilot | code-generation | programmer-assistance | end-user-programming | usability-studies | inferential-assistance

TLDRExploring a major hurdle for AI-driven programming: code comprehension. Discover why generated code is hard to understand and how this affects accuracy, customization, and debugging for end-user programmers.via the TL;DR App

Table of Links

Abstract and 1 Introduction

2. Prior conceptualisations of intelligent assistance for programmers

3. A brief overview of large language models for code generation

4. Commercial programming tools that use large language models

5. Reliability, safety, and security implications of code-generating AI models

6. Usability and design studies of AI-assisted programming

7. Experience reports and 7.1. Writing effective prompts is hard

7.2. The activity of programming shifts towards checking and unfamiliar debugging

7.3. These tools are useful for boilerplate and code reuse

8. The inadequacy of existing metaphors for AI-assisted programming

8.1. AI assistance as search

8.2. AI assistance as compilation

8.3. AI assistance as pair programming

8.4. A distinct way of programming

9. Issues with application to end-user programming

9.1. Issue 1: Intent specification, problem decomposition and computational thinking

9.2. Issue 2: Code correctness, quality and (over)confidence

9.3. Issue 3: Code comprehension and maintenance

9.4. Issue 4: Consequences of automation in end-user programming

9.5. Issue 5: No code, and the dilemma of the direct answer

10. Conclusion

A. Experience report sources

References

9.3. Issue 3: Code comprehension and maintenance

A third challenge with AI-driven programming is the issue of code comprehension. During GridBook’s user evaluation, participants mentioned that the generated formulas are hard to understand, even when users were familiar with the target language. This has potentially severe consequences: from evaluating the accuracy of the program by verifying logic, to the ability to customize code, to future debugging and reuse. As we discussed earlier, this problem also exists for trained developers.

One approach to address this issue is for the AI system to include some notion of code readability or comprehensibility as a factor in code synthesis, such as during the learning phase, or when ranking suggestions, or even take it as input to the model (similar to the ‘temperature’ parameter in Codex). This approach is useful more broadly to synthesize high quality code, such as optimizing for performance or robustness. A second solution to tackle the comprehension problem is to explain the generated code to their users in a manner that is less ‘programmerese’ and more centered around the user’s current task and context. Initial evidence suggests that participants were open to these ideas; thus, these areas are ripe for future exploration.

Authors:

(1) Advait Sarkar, Microsoft Research, University of Cambridge ([email protected]);

(2) Andrew D. Gordon, Microsoft Research, University of Edinburgh ([email protected]);

(3) Carina Negreanu, Microsoft Research ([email protected]);

(4) Christian Poelitz, Microsoft Research ([email protected]);

(5) Sruti Srinivasa Ragavan, Microsoft Research ([email protected]);

(6) Ben Zorn, Microsoft Research ([email protected]).


This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.


Written by pairprogramming | Pair Programming AI Companion. You code with me, I code with you. Write better code together!
Published by HackerNoon on 2025/08/13