Are Software Engineers Creating More Value with AI - or Just More Output?

Software engineers and AI: more output, not more value?

A recent Reddit thread from a distinguished engineer in an AWS vertical struck a nerve. The claim is simple: AI has clearly increased visible activity – more documents, more code commits, more test harnesses – but not the value that users actually feel.

“I see a high volume of artifacts and tooling, but very little increase in genuine value-delivering productivity.”

If your apps feel the same or worse despite all the AI fanfare, you’re not alone. The gap between productivity theatre and tangible outcomes is real. Let’s unpack why it’s happening and what UK teams can do about it.

Why AI increases artefacts but not customer value

Generative AI lowers the friction to produce code, docs and tests. It also makes it cheap to scaffold “harnesses” around business processes – templates, pipelines, dashboards. That’s useful, but it also tempts teams to optimise for what’s easy to count, not what matters.

• Misaligned metrics – lines of code, PR counts, or documentation pages are simple to measure and easy to game.
• Goodhart’s law – when a measure becomes a target, it stops being a good measure of success.
• Feature bloat – faster build cycles can fill backlogs with marginal features that don’t solve core user problems.
• Integration tax – generating code is fast; safely integrating it into complex systems, with data, security and performance constraints, is not.
• Quality drift – AI-generated artefacts often look polished but can mask subtle defects, regressions or misaligned behaviours.

“Everyone appears productive and on the ball… but are generally cognitively bankrupt when it comes to actual deliverables people care about.”

Measure value, not activity: a practical framing

You can’t “AI” your way into value without a clear definition of value. Ship faster, sure – but to what end? Below is a simple way to reframe team incentives and instrumentation.

Metric type	Examples	Why it misleads / Why it matters
Output metrics	Commits, PRs closed, docs created, tests written	Easy to inflate with AI; correlates weakly with customer outcomes
Flow metrics	Lead time, cycle time, change failure rate (e.g., DORA)	Healthy indicators of delivery capability, but not value on their own
Value metrics	Adoption, activation, retention, time-to-value, revenue per feature, cost-to-serve, incident rate	Tie work to outcomes users feel; requires instrumentation and product analytics

Define a “Definition of Value” alongside “Definition of Done”

• Who is the user and what job-to-be-done is improved?
• What behaviour change do we expect in 30/60/90 days?
• Which metric moves if this works, and how will we measure it?
• What’s the rollback plan if value isn’t evident?

Where AI can genuinely deliver value in software engineering

AI coding tools are best at accelerating known-good work and reducing high-friction toil. Value emerges when these accelerations are linked to a validated product goal.

• Modernisation and migrations – upgrading frameworks, refactoring legacy modules, or writing missing tests to unlock safer, faster releases.
• Defect reduction – targeted AI-assisted test generation where you’ve got historical incident data.
• Prototyping and discovery – rapid iteration to validate hypotheses via user testing before full builds.
• Contextual support – internal assistants that surface runbooks, logs and code context to cut mean time to recovery.

Two common approaches you’ll hear about:

• RAG (retrieval-augmented generation) – a method where the model looks up relevant documents first, then generates an answer. Keeps responses grounded.
• Fine-tuning – training a model further on your own data to specialise it. More control, more risk and cost. Often overused where RAG would suffice.

UK-specific considerations: privacy, compliance and cost

Engineering teams in the UK face real-world constraints beyond “can we ship it?”. You need to be clear on data flows, legal bases and operating costs.

• Data protection – if prompts, code or logs include personal data, conduct a DPIA and review retention, transfer and vendor terms. See the ICO’s guidance on generative AI and data protection (link: ICO AI and data protection).
• Security – follow the National Cyber Security Centre’s advice on using public AI services safely (link: NCSC: Secure use of AI services).
• Cost control – AI inference adds variable, usage-based costs that don’t look like traditional CI/CD bills. Track cost per request, per feature and per customer.
• Sustainability – model usage has energy and water footprints; UK hosting choices and workload design matter. See my breakdown of AI, data centre cooling and water use.

A playbook to turn AI productivity into customer value

1) Prioritise by outcome

• Adopt a clear North Star metric per product or service.
• Timebox AI experiments and require a value hypothesis before building.

2) Instrument everything

• Add feature flags, event tracking and cohort analysis before launch.
• Run A/B tests or canaries; decide in advance what success looks like.

3) Guardrails for code generation

• Mandatory review for AI-authored changes; be explicit about what’s acceptable (tests, refactors, scripts) and what needs deeper scrutiny (security, payments, auth).
• Use linters, SAST/DAST and policy-as-code to catch silent quality regressions.

4) Data governance for AI tools

• Separate public model usage from sensitive data paths; prefer enterprise offerings with data retention off by default.
• Redact secrets and PII; control training/telemetry settings and access scopes.

5) Choose the simplest AI that works

• Start with prompt engineering and RAG; only fine-tune when you’ve proved the ROI and governance path.
• Track latency, accuracy and cost together; “fast and cheap” is meaningless if answers are wrong.

What “good” looks like: signals AI is delivering value

• Shorter time-to-value – users reach first meaningful outcome faster after onboarding.
• Fewer critical incidents – measurable reduction in P1/P2 issues per release.
• Meaningful adoption – sustained use of the AI-powered feature after the novelty window.
• Improved unit economics – lower cost-to-serve at the same or better quality.
• Staff focus shifts – engineers spend more time on design, discovery and integration, less on rote code and glue work.

Why this matters now

AI has made it easier than ever to look busy. The real opportunity is to use these tools to explore more ideas, validate faster and discard what doesn’t help customers. That requires discipline: better metrics, sharper product thinking and sober governance.

If your apps and games feel worse, it’s not because AI can’t help – it’s because we’re measuring the wrong things and shipping too much that doesn’t matter. Treat AI as a leverage tool for validated problems, not a factory for artefacts, and you’ll see the difference where it counts.