In partnership with

Want to get the most out of ChatGPT?

ChatGPT is a superpower if you know how to use it correctly.

Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.

Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.

Download the free guide

A recent randomized controlled trial by Anthropic has reignited debate about the long-term impact of AI coding assistants on software developers’ learning and skill development. While AI tools promise increased productivity and faster code generation, the study suggests that these gains may come at a cost: reduced comprehension and weaker debugging skills when developers rely too heavily on automation.

Inside the Study

The research involved 52 mostly junior engineers, each with at least one year of weekly experience in Python. None of the participants were familiar with Trio, an asynchronous programming library they were asked to learn during the experiment.

Participants were randomly divided into two groups:

• A manual coding group, which completed tasks without AI assistance

• An AI-assisted group, which was allowed to use AI coding tools

Both groups were assigned two coding tasks using Trio. After completing the tasks, they took a quiz designed to assess their understanding of the material. The quiz covered debugging, code reading, and conceptual understanding.

The AI-assisted group completed the tasks approximately two minutes faster on average. However, researchers noted that this difference was not statistically significant. In other words, the productivity boost was small and could not be confidently attributed to AI use alone.

The more striking difference appeared in quiz results. The manual coding group scored an average of 67%, while the AI-assisted group averaged just 50%. The largest performance gap was found in debugging questions — an area that requires deep understanding and problem-solving skills.

Overall, developers using AI assistance scored 17% lower on comprehension tests than those coding manually.

How AI Was Used Made the Difference

One of the most important findings was that outcomes depended less on whether AI was used and more on how it was used.

Developers who scored below 40% typically showed patterns of heavy delegation. These included:

• Allowing AI to generate complete code solutions

• Gradually handing over more responsibility to AI

• Relying on AI to solve debugging issues rather than understanding the root cause

In contrast, developers who scored 65% or higher demonstrated active cognitive engagement. They used AI more selectively and strategically, such as:

• Asking follow-up questions after generating code

• Requesting explanations alongside code suggestions

• Using AI primarily for conceptual clarification while writing code independently

This distinction highlights a key tension in modern software development: AI can either support learning or replace it. When developers remain mentally engaged and treat AI as a tutor, comprehension remains strong. When they offload too much thinking to AI, skill development appears to suffer.

Community Reactions

The findings sparked discussion across developer communities. On Hacker News, one commenter summarized the concern succinctly:

“You’re trading learning and eroding competency for a productivity boost which isn’t always there.”

Another commenter raised a broader, generational worry: if junior developers grow accustomed to relying on AI for solutions, will they ever build the foundational skills needed to work independently?

These concerns reflect a deeper question facing the industry: Is AI reshaping the way developers learn — and if so, at what cost?

Supporting Evidence from Academic Research

The Anthropic study does not stand alone. A 2024 peer-reviewed experiment conducted at the University of Maribor (Applied Sciences) examined 32 undergraduate students learning React over a 10-week period. The results closely mirrored Anthropic’s findings.

Researchers found significant negative correlations between the use of large language models (LLMs) for code generation and debugging and students’ final grades. However, when LLMs were used for explanations or conceptual clarification, there was no significant negative impact.

The authors concluded that explanatory use of LLMs “might not hinder, and could potentially aid, student performance.” This reinforces the idea that the tool itself is not inherently harmful — misuse or overreliance is the problem.

Productivity Versus Skill Acquisition

Interestingly, previous observational research from Anthropic showed that AI can dramatically accelerate productivity — reducing task completion time by up to 80% for tasks where developers already possess relevant skills.

This suggests a nuanced reality:

• For familiar tasks, AI enhances efficiency.

• For unfamiliar tools or new concepts, AI may hinder deep learning if overused.

In other words, AI appears highly effective as a productivity multiplier for experienced developers but potentially risky as a substitute for foundational learning.

Learning Modes and Responsible Design

Recognizing these risks, major AI providers such as OpenAI and Anthropic have introduced learning-focused modes. These include Claude Code’s Learning and Explanatory Mode and ChatGPT Study Mode.

These modes are designed to prioritize understanding over delegation. Instead of providing full solutions immediately, they emphasize guided reasoning, step-by-step explanations, and interactive clarification.

Anthropic researchers recommend deploying AI tools with intentional design choices that encourage engineers to remain cognitively engaged. The goal is to preserve critical skills — especially debugging and validation — which are essential for overseeing AI-generated code safely and effectively.

A Balanced Perspective

The study does not argue that AI coding assistants are harmful by default. Rather, it highlights the importance of mindful usage. AI can function as an incredibly effective personal tutor, offering explanations, clarifying unfamiliar concepts, and accelerating routine tasks.

However, when developers allow AI to handle entire workflows without engaging deeply with the underlying logic, their comprehension may decline — particularly in areas requiring diagnostic reasoning and independent problem-solving.

As AI tools continue to evolve and integrate into development environments, the challenge will be finding the right balance between productivity and mastery.

The future of software development may not be about choosing between humans and AI. Instead, it may depend on how effectively humans collaborate with AI — maintaining ownership of learning while leveraging automation to enhance performance.