Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Prior conceptualisations of intelligent assistance for programmers 2. Prior conceptualisations of intelligent assistance for programmers 3. A brief overview of large language models for code generation 3. A brief overview of large language models for code generation 4. Commercial programming tools that use large language models 4. Commercial programming tools that use large language models 5. Reliability, safety, and security implications of code-generating AI models 5. Reliability, safety, and security implications of code-generating AI models 6. Usability and design studies of AI-assisted programming 6. Usability and design studies of AI-assisted programming 7. Experience reports and 7.1. Writing effective prompts is hard 7. Experience reports and 7.1. Writing effective prompts is hard 7.2. The activity of programming shifts towards checking and unfamiliar debugging 7.2. The activity of programming shifts towards checking and unfamiliar debugging 7.3. These tools are useful for boilerplate and code reuse 7.3. These tools are useful for boilerplate and code reuse 8. The inadequacy of existing metaphors for AI-assisted programming 8.1. AI assistance as search 8.1. AI assistance as search 8.2. AI assistance as compilation 8.2. AI assistance as compilation 8.3. AI assistance as pair programming 8.3. AI assistance as pair programming 8.4. A distinct way of programming 8.4. A distinct way of programming 9. Issues with application to end-user programming 9. Issues with application to end-user programming 9.1. Issue 1: Intent specification, problem decomposition and computational thinking 9.1. Issue 1: Intent specification, problem decomposition and computational thinking 9.2. Issue 2: Code correctness, quality and (over)confidence 9.2. Issue 2: Code correctness, quality and (over)confidence 9.3. Issue 3: Code comprehension and maintenance 9.3. Issue 3: Code comprehension and maintenance 9.4. Issue 4: Consequences of automation in end-user programming 9.4. Issue 4: Consequences of automation in end-user programming 9.5. Issue 5: No code, and the dilemma of the direct answer 9.5. Issue 5: No code, and the dilemma of the direct answer 10. Conclusion 10. Conclusion A. Experience report sources A. Experience report sources References References A. Experience report sources This appendix contains a list of sources we draw upon for the quotes and analysis in Section 7. While all sources were included in our analysis, we did not draw direct quotes from every source in this list. A.1. Blog posts and corresponding Hacker News discussions A.1. Blog posts and corresponding Hacker News discussions Andrew Mayne, March 17 2022, “Building games and apps entirely through natural language using OpenAI’s code-davinci model”. URL: <https://andrewmayneblog.wordpress.com/2022/03/17/building-games-and-apps-entirely-through-natural -language-using-openais-davinci-code-model/>. Hacker News discussion: https://news.ycombinator.com/item?id=30717773


Andrew Mouboussin, March 24 2022, “Building a No-Code Machine Learning Model by Chatting with GitHub Copilot”. URL: https://www.surgehq.ai/blog/building-a-no-code-toxicity-classifier-by-talking-to-copilot. Hacker News discussion: https://news.ycombinator.com/item?id=30797381


Matt Rickard, August 17 2021, “One Month of Using GitHub Copilot”. URL: https://matt-rickard.com/github-copilot-a-month-in/.
Nutanc, November 15 2021, “Using Github copilot to get the tweets for a keyword and find the sentiment of each tweet in 2 mins”. URL: https://nutanc.medium.com/using-github-copilot-to-get-the-tweets-for-a-keyword-and-find-the-sentiment-of-each-tweet-in-2-mins-9a531abedc84.


Tanishq Abraham, July 14 2021, “Coding with GitHub Copilot”. URL: https://tmabraham.github.io/blog/github_copilot.


Aleksej Komnenovic, January 17 2022, “Don’t fully trust AI in dev work! /yet”. URL: https://akom.me/dont-fully-trust-ai-in-dev-work-yet. Andrew Mayne, March 17 2022, “Building games and apps entirely through natural language using OpenAI’s code-davinci model”. URL: <https://andrewmayneblog.wordpress.com/2022/03/17/building-games-and-apps-entirely-through-natural -language-using-openais-davinci-code-model/>. Hacker News discussion: https://news.ycombinator.com/item?id=30717773 Andrew Mayne, March 17 2022, “Building games and apps entirely through natural language using OpenAI’s code-davinci model”. URL: <https://andrewmayneblog.wordpress.com/2022/03/17/building-games-and-apps-entirely-through-natural -language-using-openais-davinci-code-model/>. Hacker News discussion: https://news.ycombinator.com/item?id=30717773 https://news.ycombinator.com/item?id=30717773 Andrew Mouboussin, March 24 2022, “Building a No-Code Machine Learning Model by Chatting with GitHub Copilot”. URL: https://www.surgehq.ai/blog/building-a-no-code-toxicity-classifier-by-talking-to-copilot. Hacker News discussion: https://news.ycombinator.com/item?id=30797381 Andrew Mouboussin, March 24 2022, “Building a No-Code Machine Learning Model by Chatting with GitHub Copilot”. URL: https://www.surgehq.ai/blog/building-a-no-code-toxicity-classifier-by-talking-to-copilot. Hacker News discussion: https://news.ycombinator.com/item?id=30797381 https://www.surgehq.ai/blog/building-a-no-code-toxicity-classifier-by-talking-to-copilot https://news.ycombinator.com/item?id=30797381 Matt Rickard, August 17 2021, “One Month of Using GitHub Copilot”. URL: https://matt-rickard.com/github-copilot-a-month-in/.
Nutanc, November 15 2021, “Using Github copilot to get the tweets for a keyword and find the sentiment of each tweet in 2 mins”. URL: https://nutanc.medium.com/using-github-copilot-to-get-the-tweets-for-a-keyword-and-find-the-sentiment-of-each-tweet-in-2-mins-9a531abedc84. Matt Rickard, August 17 2021, “One Month of Using GitHub Copilot”. URL: https://matt-rickard.com/github-copilot-a-month-in/. https://matt Nutanc, November 15 2021, “Using Github copilot to get the tweets for a keyword and find the sentiment of each tweet in 2 mins”. URL: https://nutanc.medium.com/using-github-copilot-to-get-the-tweets-for-a-keyword-and-find-the-sentiment-of-each-tweet-in-2-mins-9a531abedc84. https://nutanc.medium.com/using-github-copilot-to-get-the-tweets-for-a-keyword-and-find-the-sentiment-of-each-tweet-in-2-mins-9a531abedc84. Tanishq Abraham, July 14 2021, “Coding with GitHub Copilot”. URL: https://tmabraham.github.io/blog/github_copilot. Tanishq Abraham, July 14 2021, “Coding with GitHub Copilot”. URL: https://tmabraham.github.io/blog/github_copilot. https://tmabraham.github.io/blog/github_copilot Aleksej Komnenovic, January 17 2022, “Don’t fully trust AI in dev work! /yet”. URL: https://akom.me/dont-fully-trust-ai-in-dev-work-yet. Aleksej Komnenovic, January 17 2022, “Don’t fully trust AI in dev work! /yet”. URL: https://akom.me/dont-fully-trust-ai-in-dev-work-yet. https://akom.me/dont-fully-trust-ai-in-dev-work-yet A.2. Miscellaneous Hacker News discussions A.2. Miscellaneous Hacker News discussions https://news.ycombinator.com/item?id=30747211


https://news.ycombinator.com/item?id=31390371


https://news.ycombinator.com/item?id=31020229&p=2


https://news.ycombinator.com/item?id=29760171


https://news.ycombinator.com/item?id=31325154


https://news.ycombinator.com/item?id=31734110


https://news.ycombinator.com/item?id=31652939


https://news.ycombinator.com/item?id=30682841


https://news.ycombinator.com/item?id=31515938


https://news.ycombinator.com/item?id=31825742 https://news.ycombinator.com/item?id=30747211 https://news.ycombinator.com/item?id=30747211 https://news.ycombinator.com/item?id=30747211 https://news.ycombinator.com/item?id=31390371 https://news.ycombinator.com/item?id=31390371 https://news.ycombinator.com/item?id=31390371 https://news.ycombinator.com/item?id=31020229&p=2 https://news.ycombinator.com/item?id=31020229&p=2 https://news.ycombinator.com/item?id=31020229&p=2 https://news.ycombinator.com/item?id=29760171 https://news.ycombinator.com/item?id=29760171 https://news.ycombinator.com/item?id=29760171 https://news.ycombinator.com/item?id=31325154 https://news.ycombinator.com/item?id=31325154 https://news.ycombinator.com/item?id=31325154 https://news.ycombinator.com/item?id=31734110 https://news.ycombinator.com/item?id=31734110 https://news.ycombinator.com/item?id=31734110 https://news.ycombinator.com/item?id=31652939 https://news.ycombinator.com/item?id=31652939 https://news.ycombinator.com/item?id=31652939 https://news.ycombinator.com/item?id=30682841 https://news.ycombinator.com/item?id=30682841 https://news.ycombinator.com/item?id=30682841 https://news.ycombinator.com/item?id=31515938 https://news.ycombinator.com/item?id=31515938 https://news.ycombinator.com/item?id=31515938 https://news.ycombinator.com/item?id=31825742 https://news.ycombinator.com/item?id=31825742 https://news.ycombinator.com/item?id=31825742 Authors:
(1) Advait Sarkar, Microsoft Research, University of Cambridge (advait@microsoft.com);
(2) Andrew D. Gordon, Microsoft Research, University of Edinburgh (adg@microsoft.com);
(3) Carina Negreanu, Microsoft Research (cnegreanu@microsoft.com);
(4) Christian Poelitz, Microsoft Research (cpoelitz@microsoft.com);
(5) Sruti Srinivasa Ragavan, Microsoft Research (a-srutis@microsoft.com);
(6) Ben Zorn, Microsoft Research (ben.zorn@microsoft.com). Authors: Authors: (1) Advait Sarkar, Microsoft Research, University of Cambridge (advait@microsoft.com); (2) Andrew D. Gordon, Microsoft Research, University of Edinburgh (adg@microsoft.com); (3) Carina Negreanu, Microsoft Research (cnegreanu@microsoft.com); (4) Christian Poelitz, Microsoft Research (cpoelitz@microsoft.com); (5) Sruti Srinivasa Ragavan, Microsoft Research (a-srutis@microsoft.com); (6) Ben Zorn, Microsoft Research (ben.zorn@microsoft.com). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Microsoft

OpenAI

Building On Giants: A Look At The Research That Shaped This Paper

The Great Transformation: How LLMs Remake Every Programming Activity

Pair Programming AI Agent 

Too Long; Didn't Read

Get Free Unlimited IP Geo Data With IPinfo Lite

A Look At The Data: Blogs, Forums, And The Rise Of LLM Tools

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

11 Proven Solutions to Common GitHub Copilot Problems

Human-Human vs Human-AI Pair Programming

The Pros and Cons of Pair Programming

What Makes Pair Programming a Success (or a Failure)?

Breaking Down the Human-AI Pair Programming Experience

LLM, Your pAIr Programmer?

11 Proven Solutions to Common GitHub Copilot Problems

Human-Human vs Human-AI Pair Programming

The Pros and Cons of Pair Programming

What Makes Pair Programming a Success (or a Failure)?

Breaking Down the Human-AI Pair Programming Experience

LLM, Your pAIr Programmer?

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps