Devin AI: End of Software Engineers or Just A Tool?

By Mostofa Adib Shakib

In the rapidly evolving landscape of technology, I find myself pondering a significant question as a software engineer: Will Devin AI take my job or merely serve as another tool in our expansive arsenal? For those unfamiliar with the name, Devin is not your average newcomer to the tech scene. It's the moniker given to the world's first AI Software Engineer, developed by the visionary minds at Cognition Labs. Since its inception in November 2023, Cognition Labs has secured**$21 million** in funding, notably fromPeter Thiel's Founders Fund and former Twitter executive Elad Gil. Thiel, celebrated for co-founding PayPal and being the first outside investor in Facebook, marks the caliber of faith vested in this endeavor. On March 12, 2024, Cognition Labs unveiled Devin through a series of videos, showcasing the capabilities of this fully autonomous AI, which has sent ripples through the tech community, sparking debates on job security across all levels of software engineering.

Behind this groundbreaking innovation is Scott Wu, leading a small yet extraordinarily talented team at Cognition Labs. Comprising 10 individuals, each a gold medalist from the International Olympiads in Informatics (IOI), their team's pedigree is unmatched. Wu, a Harvard-educated math prodigy, boasts an impressive resume, including three IOI gold medals with perfect scores in 2014, and holds the title of Legendary Grandmaster on Codeforces. His journey from a math enthusiast to the co-founder and CTO of Lunchclub, an AI-powered networking platform, alongside his brother Neal Wu, a fellow prodigy and co-founder of Cognition Labs, lays the foundation for Devin's creation. The Wu brothers' unique combination of mathematical brilliance and entrepreneurial spirit is the driving force behind the development of Devin, positioning Cognition Labs at the forefront of AI innovation in software engineering.

So, what exactly can Devin do? According to Cognition Labs' demonstrations, Devin is 9% better at solving real Software Engineering tasks compared to all the existing models and capable of self-training, identifying bugs in code, and resolving issues on platforms like GitHub and Upwork. Today I will gauge the impact of this AI on the industry, and analyze the claims made by Cognition Lab on Devin’s capabilities.

While Devin is impressive, I see it as something other than replacing me or being able to perform the tasks I did during my time as a Software Engineer at Snapchat. The problems solved by Devin in one of the 5 videos released by Cognition Labs had clear instructions and guidelines. However, in the real world, problems are often ambiguous, and sometimes we need to first figure out what exactly we are solving, and to do that cross-functional communication between several teams is needed. At Snapchat, one time I worked on a bug that blacklisted Snapchat in Samsung’s multi-hierarchy window for a few months. To solve this issue and unblock Snapchat, I had to take an iterative approach and talk back and forth with different cross-functional teams until a solution had been found something AI could not do. I also spent the majority of my time identifying deep performance bottlenecks, scalability issues, and memory corruption as part of Snapchat’s platform & performance team, challenges that AI cannot yet tackle. Moreover, software engineers play a crucial role in challenging technical designs and validating the code generated by AI.

At first glance, the claims made by Cognition Labs regarding Devin look very ambitious and scary, but if you dive deep, it’s actually a bit misleading and not as scary as people think. For example: In one of the videos released by Cognition Labs, they claimed that Devin can teach himself by learning from blog articles and other resources. However, if you visit the blog article that was used in the video, you will notice that it links to a GitHub page where all the code and the exact steps to take are already provided. So essentially, Devin is just following the instructions instead of writing custom code and figuring things out on the fly. Another key point to note is that, unlike ChatGPT, Devin currently takes 2-3 hours to perform basic tasks that a Junior Software Engineer will be able to accomplish much faster, and another human Software Engineer has to guide Devin through the entire process to make sure Devin is going down the right path.

As an AI2030 Global Fellow and drawing from my extensive experience in AI research, Devin is impressive and will change the way Software Development is currently done, much like how calculators made mathematicians far more productive. AI will serve as a productivity tool for software engineers, automating basic tasks and allowing them to focus on more creative endeavors. It won't replace them, just as calculators didn't replace mathematicians. This comparison between calculators and mathematicians, software engineers, and Devin, is particularly apt. When calculators were first introduced, there was a fear that they would render mathematicians obsolete. However, calculators became tools that enhanced the productivity of mathematicians, allowing them to perform calculations more quickly and accurately, and freeing up their time to focus on more complex and creative mathematical problems. Similarly, AI tools like Devin are not meant to replace software engineers but to augment their capabilities. By automating routine tasks such as bug identification and code optimization, Devin can enable software engineers to concentrate on higher-level design, architecture, and problem-solving aspects of software development.

Moreover, the human element in software engineering cannot be understated. Software engineers bring creativity, critical thinking, and the ability to understand and respond to user needs—qualities that AI, in its current state, cannot replicate. While AI can follow instructions and learn from patterns, it lacks the intuition and emotional intelligence that humans possess. Just as mathematicians use calculators as a tool to aid their work, software engineers can use AI like Devin as a tool to enhance their productivity and creativity.

In conclusion, the relationship between software engineers and AI tools like Devin is symbiotic. AI can automate certain aspects of software development, but it is the human software engineer who provides the creative vision, strategic direction, and critical oversight that are essential to successful software projects. As AI continues to evolve, it will undoubtedly transform the landscape of software engineering, but it will do so in a way that complements and enhances the work of human engineers rather than replacing them.