In coverage that drew 1,306 points on Hacker News, Sean Goedecke advanced a deceptively simple thesis: working with an LLM amplifies the skilled and exposes the unskilled. Expertise, he argued, is rewarded rather than replaced (Goedecke, “LLMs reward expertise”). The post resonated because it flatly contradicts both panic narratives — that AI will erase knowledge workers — and triumphalist ones — that anyone can now produce expert output by typing a sentence.

The argument deserves scrutiny because it makes a falsifiable empirical claim, not a philosophical one. Does the data support the idea that expertise is the variable that determines how much value a person extracts from an LLM? Or does the evidence point somewhere more nuanced? Goedecke himself flagged the risk in his own comment thread: some readers, he noted, are “rightly suspicious of a view that’s reassuring them about how they’re still valuable.” That honesty is the right starting point. We should test the claim against studies, not vibes.

What Goedecke actually argues

Goedecke’s core mechanic is straightforward. Before LLMs, a technical gap — say, not knowing CSS — forced you to either recruit a skilled colleague or hope a matching answer already existed online. Today the same person can delegate that gap to a model and produce “sort-of-okay” output. Everyone becomes a generalist (Goedecke).

From this, a tempting conclusion follows: if everyone talks to the same model, prompting skill is irrelevant and expertise no longer matters. Goedecke rejects that. His central claim is that the most important skill in prompting is expertise in the domain you are prompting about. A novice and an expert may get similar first drafts, but only the expert can steer, prune, and verify the result. He extends this to codebases specifically: if you hold a strong “theory of your codebase,” you can push the LLM far harder than someone with no familiarity, because you have a prior sense of what a good solution looks like.

The mechanism he proposes is information retrieval, not magic. The answer is “in the model” already; the scarce resource is the human ability to pull the right answer out. That reframes expertise as a compression and filtering skill: knowing which of the model’s many plausible lines to keep, and which to throw away before they calcify into a confident mistake.

The Terence Tao conversation, and what it really shows

His flagship exhibit is Terence Tao’s public conversation with ChatGPT about a recently discovered counterexample to the Jacobian Conjecture (Tao’s shared chat). Tao does not merely prompt; he makes leaps, proposes reformulations, and pushes back when outputs “look weird.” Goedecke notes the model shifts into “talking-to-mathematicians” mode for Tao, producing terser, denser replies than a layperson receives.

The lesson is not that Tao has mastered a secret prompt syntax. It is that domain knowledge lets you pull the right idea out of a multi-paragraph response and discard the rest. As we explored in our own analysis of Tao’s method, the genius sees the shape of the problem before the model finishes speaking, and he almost never takes the model’s advice about where to go next (TheAIprism: what a genius sees that we don’t). The model is a sparring partner, not an authority.

This maps onto a broader pattern Goedecke observes in his own engineering work: familiarity with concrete specifics beats generic principles. He can ask sharp questions about the systems he owns at GitHub that he could never ask about abstract mathematics. Expertise, in other words, is local — and LLMs reward the locality. The same person can be expert and novice in the same afternoon, depending on the domain, which is why the claim “expertise is rewarded” is true only relative to a specific task.

What the controlled studies actually say

Goedecke’s claim is anecdotal. The cleanest counter-evidence comes from randomized experiments. In a 2023 study, Shakked Noy and Whitney Zhang gave 453 professionals incentivized writing tasks, randomly assigning ChatGPT access (Noy & Zhang, Science). Output quality rose and completion time fell — but the gains were concentrated among lower-ability workers. The productivity distribution compressed rather than spread.

A large field study of customer-service agents reached the same pattern at scale. Brynjolfsson, Li, and Raymond studied thousands of agents before and after AI deployment and found an average 15% productivity lift, but a 34% lift for novice and low-skilled workers, with minimal effect on the best performers (Brynjolfsson, Li & Raymond, “Generative AI at Work”). Their interpretation: AI transmits the best practices of top performers downward, lifting the floor for everyone below them.

A 2024 age-classification experiment by Caplin et al. compounds the point: AI raised performance across ability levels but reduced dispersion most when users were well calibrated about their own skill (Law & Economics Center review). The recurring result across writing, support, and classification tasks is skill compression, not elite-only reward. If the only evidence were these three papers, Goedecke’s thesis would look wrong — which is exactly why the next study matters.

The mechanism behind compression is best-practice transmission. A junior who has never seen a strong example of the task suddenly has one on tap, every time. The senior, who already embodied those practices, gains little from re-encountering them. That is why the floor moves and the ceiling barely does — at least on the tasks these studies measured, which tend to be bounded and verifiable.

The jagged technological frontier

The most important caveat comes from the BCG–Harvard study of 758 consultants and 18 realistic tasks (Dell’Acqua et al., “Navigating the Jagged Technological Frontier”). Within the model’s capabilities, GPT-4 users completed 12.2% more tasks, 25.1% faster, and 40% produced higher-quality work. But on tasks just outside that “jagged frontier,” AI users were 19% less likely to reach a correct answer than people with no AI at all.

This is the crux. AI does not fail uniformly; it fails on a ragged boundary the user cannot see. The researchers distinguish “centaurs” (clean human/AI task splits) from “cyborgs” (constant interaction) — both work, but both depend on the human knowing where the frontier sits. Lakhani’s blunt warning captures it: “This is not Google.” Treating the model as a search box is exactly how users fell 19% behind on the hard tasks.

The frontier finding does something subtle to Goedecke’s thesis. It suggests the expert’s advantage is not merely producing better drafts — it is knowing which tasks to hand the model at all. Outside the frontier, more delegation is worse. That is a meta-judgment the novice lacks, and it is invisible in aggregate productivity numbers that average over easy and hard tasks alike. The expert’s reward, in other words, shows up as avoidance of catastrophe rather than headline speed gains.

Two effects, not one: compression and amplification

Set the studies side by side and a cleaner picture emerges. There are two distinct effects running in opposite directions:

Compression at the floor. AI lifts weaker workers most. Noy & Zhang, Brynjolfsson et al., and Caplin et al. all find performance dispersion shrinks, especially when users are well calibrated about their own skill. The floor rises fast.

Amplification at the ceiling. Experts extract more at the top. Goedecke’s Tao example and the BCG finding — that outside-frontier failure depends on user judgment — both imply the expert’s edge grows precisely where tasks are hard and the model is silent or wrong.

Goedecke is right that expertise is rewarded. He understates how much AI compresses the gap below. The honest synthesis is asymmetric: the floor rises faster than the ceiling. A junior with a model can now mimic a competent senior on routine work, but no amount of model access converts a novice into Tao on the frontier.

Consider a concrete split. On a bounded writing task — summarize this memo, draft this email — the novice-plus-model and the expert-plus-model land close, because the frontier encloses the task and the model supplies the missing structure. On an open mathematical proof or a subtle production incident, the novice gets fluent nonsense the expert immediately flags. The same tool, two regimes: compression where the frontier is generous, amplification where it is thin.

One caveat tempers the whole comparison: the frontier is not fixed. As models improve, tasks that were once outside it migrate inside, and the compression effect expands with them. If the boundary keeps moving outward, the era in which expertise is decisively rewarded at the ceiling may shrink to a thinner and thinner sliver of remaining-hard problems — unless expertise itself is what defines where the frontier currently lies.

Why calibration is the new bottleneck

If AI both lifts the floor and rewards expertise at the ceiling, what exactly does the skilled person contribute? The BCG data points to one scarce skill: calibration — knowing when to trust the model and when to ignore it. Outside the frontier, over-trust was actively harmful (−19% correctness). The human who suspects “this looks more complex than I hoped” and reroutes is the human who stays accurate.

Goedecke’s own phrasing fits: “the human is the bottleneck, not the model,” because the hard part is communicating exactly what solution you want (Goedecke). I would sharpen that: the bottleneck is judgment about the model’s limits, a meta-skill that sits above raw domain expertise. Domain expertise helps you recognize a wrong answer; calibration tells you whether to ask at all. Both are human, neither is automatable yet.

This also explains the Hacker News skeptic’s objection — that anyone can now feel rewarded. Feeling rewarded and being right are different. The model happily confirms the incompetent, which is precisely why calibration, not confidence, separates the expert from the amateur. Calibration is buildable: it grows from repeated, consequential feedback where wrong answers carry a cost the model cannot absorb for you. That is another reason expertise, earned through consequences, stays relevant.

Implications for knowledge work and hiring

For organizations, the data argues against two instincts. First, do not assume AI erases the need for senior talent; you need experts precisely to set direction and catch errors outside the frontier. Second, do not assume juniors are now interchangeable with seniors — AI narrows the gap but does not close it, and someone must still validate the output. The realistic play is mixed teams where experts handle the frontier and juniors, augmented, handle the floor.

This reframes the jobs debate away from “will AI replace us” toward “who can steer it” — a theme we examine in our broader review of what is actually happening to jobs (TheAIprism: separating AI hype from reality). The scarce role is the editor of the machine, not its operator. Hiring should weight demonstrated calibration — can this person tell good model output from fluent nonsense? — above raw output volume, because volume is now nearly free and discernment is not.

The same logic reshapes internal metrics. If AI narrows the spread between your best and worst contributors on routine work, average throughput becomes a worse signal of talent. Managers should monitor the tail — the hard, frontier cases where only calibration prevents regressions — rather than aggregate speed, or they will reward the person who delegates most and ships the most plausible errors.

Implications for education and evaluating AI output

The synthesis also reshapes how we should teach and assess. If AI compresses the floor, drilling rote execution matters less; teaching calibration matters more. Students need to learn not “how to write the essay” but “how to tell whether the essay the model wrote is correct.” That is a higher-order skill, and it is exactly the one experts already possess. Education that skips the fundamentals in favor of pure prompt reliance risks producing adults who cannot catch the model when it is wrong.

For evaluation, the lesson is uncomfortable: we can no longer grade the artifact without grading the process. A flawless draft may be expert-steered or expert-blind. The differentiator is whether the author can defend every line — a capacity AI cannot fake and expertise alone supplies. Assessment must move toward oral defense, source tracing, and revision history rather than the final product alone.

There is a Carnegie-style lesson here too. Just as cognitive tools historically offloaded routine computation, LLMs offload routine composition — and in both cases the expert’s value migrated to the parts the tool could not do. The frontier, not the floor, is where expertise lives, and curricula that teach only floor-level execution are teaching the part the machine now owns. The goal of training shifts from producing flawless executors to producing reliable judges.

The verdict, and an open question

Goedecke’s thesis survives contact with the data, but in a revised form. Expertise is rewarded — yet so is the absence of it, because AI raises the floor for everyone. The net effect is not replacement but reorganization: execution cheapens, judgment appreciates. The people who thrive are those who treat the model as a brilliant, unreliable junior colleague rather than an oracle, and who invest in the calibration the studies show is decisive.

The practical takeaway for knowledge workers is unglamorous. Spend less energy on prompt incantations and more on deepening the domain sense the model cannot fake; build feedback loops where your mistakes are visible; and reserve the model for the tasks inside its frontier while you guard the boundary yourself. Expertise is not obsolete. It is redistributed toward the places the model cannot reach.

That leaves the question the studies have not settled: if AI keeps lifting the floor while the expert’s edge persists mainly at the frontier, will deep expertise become a smaller share of total value — or the only part that still commands a premium?

References

1. Goedecke, S. (2026). LLMs reward expertise. seangoedecke.com. https://www.seangoedecke.com/llms-reward-expertise/
2. Dell’Acqua, F., Lakhani, K. R., McFowland, E., et al. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School / BCG. Coverage
3. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://www.science.org/doi/10.1126/science.adh2586
4. Brynjolfsson, E., Li, D., & Raymond, L. (2023). Generative AI at Work. NBER Working Paper 31161. https://www.nber.org/papers/w31161
5. Law & Economics Center (2024). AI, Productivity, and Labor Markets: A Review of the Empirical Evidence. George Mason University. https://laweconcenter.org/resources/ai-productivity-and-labor-markets-a-review-of-the-empirical-evidence/

The post ‘LLMs Reward Expertise’: What the Data Actually Shows appeared first on The AI Prism.

The essay comes from Lars Faye, a developer who has spent the last year warning about “the skilled orchestrator paradox”: the skills you need to manage AI coding agents are the same skills the agents quietly replace. The more you lean on the tool, the less you practice the craft — and the craft is exactly what the tool demands you know to use it well.

Here at The AI Prism, we’ve been watching this debate build for months. Then three days ago, ZDNet published survey data that gave the worry a number: 80% of developers say their AI use feels more like dependence than an advantage. The collapse argument is no longer hypothetical — the studies are in, the addiction data landed, and the industry is betting trillions on a pipeline that may be eroding the very skills it needs to survive.

Here’s the uncomfortable part: the evidence says the erosion is real. What’s still open for debate is whether it’s inevitable.

The Developers Who Need AI Least Benefit Most

The first thing Faye’s essay does is invert the standard productivity story. The people getting the most out of coding assistants, he writes, are industry veterans — developers with years of experience that predates the tools. The people getting the least, and the most damage, are the novices being told to lean on AI from day one.

The study behind that claim, “The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers” — highlighted recently by JetBrains — analyzed live coding sessions and found heavy AI assistance actively derailed learning. Participants “often skipped crucial planning stages,” finished with an “illusion of competence” instead of understanding, and the novices who performed best were the ones who mitigated or ignored the AI entirely. The best performers had developed what the researchers call “negative expertise” — the ability to ignore incorrect or unhelpful AI suggestions.

That’s the core of the collapse argument: the tool’s value curve is inverted. Experts steer it, audit it, and catch its lies. Beginners hand it the wheel and end up lost. In Faye’s words, the least restricted novices in the study “had skipped crucial steps in the programming problem-solving process, and were now lost.”

The Expert Novice Is Already Here

Now consider the signals the industry is sending. For three years, the mantra has been “AI won’t replace you — someone using AI will.” In the same breath, everyone from senior engineers to Robert “Uncle Bob” Martin says juniors shouldn’t touch AI tooling for their first three years. The tools demand expertise to wield responsibly — and the industry is pushing the least experienced people to use them first.

Faye calls the result the “expert novice”: a developer who needs expert-level judgment to evaluate AI output, but has been denied the years of friction that build that judgment. The HN thread is full of working confirmation. One commenter watching it happen in real time reports reviewing “vibe-coded” work that is “some seriously low quality garbage” — but it’s functional enough to keep the person who generated it convinced they can code. Another describes enterprise mandates straight from leadership: “if you’re writing code manually, you’re doing it wrong,” with engineers producing code faster than humans can understand or honestly review it.

The junior rung of the market is where this squeeze shows up first — and it’s exactly the rung our earlier analysis of the AI job market flagged as the most exposed. The people who most need to build expertise are being trained, by mandate and by habit, to never practice it.

Confidence Without Comprehension

The most dangerous part of the erosion isn’t what developers forget. It’s that they don’t know they’ve forgotten it. The Widening Gap study’s participants “thought it was like having a personal tutor” — and the data showed the opposite: they used the tools as an answer machine, skipped planning, and believed they were learning while their comprehension stayed flat.

The effect isn’t limited to coding. In UPenn’s 2025 study of 1,000 students learning math with an LLM, the AI-assisted group performed 17% worse than students with a plain textbook — while simultaneously believing they were excelling. The tool had produced confidence without comprehension, and the students couldn’t tell the difference.

Faye’s metaphor for this is a compass that always points north, wherever you suggest north might be. When you’re exploring unfamiliar territory, you don’t know what you don’t know — and the model’s accommodating confidence fills the gap with certainty instead of understanding. An infinite answer machine is a terrible teacher, because it never tells you you’re wrong until you’re already lost.

The Friction Is the Feature

The collapse argument rests on a simple claim about how expertise forms: it requires friction. Coding has endless moments of tracing obscure errors with no log file, feeling the performance difference between two approaches, rewriting something that won’t scale. That applied struggle is what builds “developer intuition” — what Faye calls Fingerspitzengefühl, the fingertip feeling that makes a senior dev look at code and think “yeah, this is probably going to cause problems.”

The same UPenn study that found the 17% harm also tested a fix. Students using a “Tutor” version — where they asked for help, then independently solved the problem — performed 127% better in AI-assisted practice. The model wasn’t doing the work; it was forcing the student to.

When the friction stays, the learning lands. When it disappears, so does the imprint.

Anthropic’s 2026 study on how AI assistance shapes coding skill formation reached the same conclusion in its own words: “Cognitive effort — and even getting painfully stuck — is likely important for fostering mastery.” One HN commenter put the muscle analogy bluntly: if you stop training the muscle of logic and reasoning, it atrophies, just like an unused physical muscle. The irony, as Faye notes, is that the most productive learning that can happen with an AI coding tool is when it isn’t used to generate much code at all.

The Addiction Numbers Are In

Then there’s the behavioral layer, which the expertise debate usually treats as a side effect but is increasingly the main event. A Coddy Tech survey of 305 developers, reported by ZDNet on August 22, found 80% say their AI use has felt more like dependence than an advantage.

The same survey found 43% keep coding with AI after hours when they meant to stop. 32% have put off sleep to keep going. 39% say the tools made it harder to switch off from work.

The headline story is Quentin Rousseau, CTO and co-founder of the incident-response company Rootly, who posted on LinkedIn about watching Claude Code refactor a module at 2:47 a.m. with no deadline and no outage: “I can’t stop.”

“Agentic coding is addictive,” he explained. “When the agent gets things right, you get a dopamine hit. When it fails, you get an adrenaline rush.”

Rousseau says he eventually sought medical help. His diagnosis of why the loop hooks so hard deserves a second read: “Watching an agent’s work is passive enough to feel like rest, active enough to keep you hooked.”

Here’s the part that should make every engineering leader uncomfortable: the same survey found 74% of heavy AI users said it made a raise or promotion more likely — and 51% said they were more likely to burn out. The reward and the cost are being handed out together, and the industry is measuring the first while ignoring the second.

Trust Is Falling Faster Than Adoption Is Rising

Addiction isn’t the only crack. The 2025 Stack Overflow Developer Survey shows adoption still climbing — 80% of developers now use AI tools in their workflows — while trust in their accuracy fell from 40% to 29% in a year, and positive sentiment dropped from 72% to 60%. Nearly half, 45%, said they were frustrated by AI answers that are “almost right, but not quite”: output that looks convincing and creates difficult debugging work.

ZDNet’s Vaughan-Nichols gives that phenomenon a name worth keeping: verification debt. The code arrives instantly, but you still have to establish whether it’s correct, secure, maintainable, and consistent with the architecture. The time saved on writing is spent on verifying — and the verification skill is exactly the expertise that’s eroding. One HN commenter summarized the trade-off cleanly: with agentic coding you are “practically trading accuracy and control for speed and efficiency.”

That’s the trap in one sentence. The industry is optimizing the metric it can see — throughput — while the invisible metric, the ability to judge what the machine produced, falls.

And employers amplify the loop: when organizations treat AI as a way to multiply developer capacity, workers face pressure to ship more features and close more tickets in the same hours. The time saved on writing gets re-spent on larger pull requests, more generated changes to inspect, and more operational risk to manage. The dependency is doing the work the org chart assigned to management.

The Pipeline Problem: Who Reviews the Code No One Understands?

The collapse argument gets its sharpest form from the HN thread itself. One commenter describes the industry as a snake eating its own tail: the developers who refuse to “cook their brains” with AI end up reviewing terrible AI-generated code written by people who did, and their reward for staying sharp is more of that review work. “Completely unsustainable.” Another, a security-minded dev, puts it personally: the more the LLM writes the code, “the less good critique I’ll be able to give in the future” — because review ability is itself a skill built on reading and understanding code.

The experts see the same failure mode. David Cramer, co-founder of the error-tracking company Sentry, told an interviewer what he thinks of the belief that future models will clean up the junk this generation generates: “I don’t think that’s true. I think it’s a science experiment… You want to flex that you can generate all of your code and have hundreds of things going in parallel, I will flex and show you how broken the code is 100% of the time.”

The counterargument, well-represented in the 530 comments, goes like this: we’ve been here before. Assembly expertise “collapsed” when compilers arrived, and nobody demands that modern programmers know microcode — the world got faster, not worse. One commenter jokes that “coding expertise is going to collapse from compiler reliance” sounds exactly like a 1985 Turbo Pascal detractor.

Others point out that scarcity can pay: COBOL-style expertise, they note, eventually commands a premium. A third commenter honestly reports learning more from LLMs than losing to them — wider tooling exposure, deeper familiarity, faster reach.

But the strongest rebuttal on the thread cuts the historical analogy apart: programming languages are deterministic, and LLMs are not. The boundary between an abstract concept and something executable used to sit in the programmer’s head; with an LLM, it becomes shared between the programmer and the matrix-multiplication machine. Leaky abstractions are fine when the leak is deterministic. When the abstraction itself invents the leak, the analogy breaks.

Friction-First Is the Only Strategy Left

So what do you do with this? The individual answer is to treat AI as a sparring partner, not a ghostwriter. The studies are consistent: Socratic use — ask the model to question your approach, then solve the problem yourself — produces the learning, while answer-generator use produces the illusion of it.

Faye’s rule for his own work is friction-first: let the model audit your code, but don’t let it write what you haven’t first thought through. That’s the difference between a developer using AI and a developer being used by it.

The organizational answer is harder, because it fights the incentive structure. Companies that measure developers by output will keep getting output — and losing reviewers. The leaders who survive this transition will be the ones who measure understanding: who require that merged AI code be explainable by the person who merged it, who reward the engineers who catch the model’s lies rather than the ones who ship the most tokens.

Joel Spolsky’s Law of Leaky Abstractions, written in 2002, has aged into prophecy: code generation tools which pretend to abstract something, like all abstractions, leak — “and the only way to deal with the leaks competently is to learn about how the abstractions work.”

There’s a third answer, and it’s the one nobody wants to say out loud: the industry may simply be making a trillion-dollar bet that this knowledge won’t matter. Faye names it directly — the bet that LLMs will “take up the slack and effectively become the new generation of developers.” If that bet is right, none of this matters. If it’s wrong, the industry has spent the best years of its junior cohort training them to never become senior.

The Bottom Line

The collapse of coding expertise isn’t a prediction about the technology. The technology is fine — it writes plausible code at astonishing speed. The collapse is a prediction about the people: a generation of developers is being shaped by tools that reward speed over understanding, confidence over competence, and dependence over mastery.

The studies, the surveys, and the 530-comment argument on the front page all point the same direction. Friction is how expertise forms, and we have built an industry-wide machine for removing friction from the exact people who need it most.

None of this has to be inevitable. The tools can be used pedagogically; the incentives can be redesigned; juniors can be protected for three years the way Uncle Bob suggests. But every week the industry spends promoting generation-over-comprehension is a week it spends shrinking the pool of people who can still read the code being written.

The people writing code are losing the ability to read it — 530 HN comments and counting. If the last generation that can read code is the one writing it today, who audits the machines once the machines write everything?

References

1. Lars Faye — “AI Coding will Prevent Expertise”
2. Hacker News — “Coding expertise is going to collapse from AI reliance” (536 points, 530 comments)
3. ZDNET — “‘I can’t stop’: 80% of developers find AI coding more addictive than helpful” (Aug 22, 2026)
4. arXiv — “The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers”
5. PNAS — “Generative AI without guardrail can harm learning: Evidence from high school math” (UPenn, 2025)
6. Anthropic Research — “How AI assistance impacts the formation of coding skills” (2026)
7. Stack Overflow Developer Survey 2025 — AI trust and adoption data
8. Robert C. Martin on junior developers and AI tooling (via HN thread)
9. Joel Spolsky — “The Law of Leaky Abstractions” (2002)
10. The AI Prism — “What Is Actually Happening to Jobs: Separating AI Hype From Reality”

The post Coding Expertise Is Collapsing — and AI Is Why appeared first on The AI Prism.