Real-World Feedback From Creating Coding AI Agents
Signs of Divergence Between AI Implementation Realities and Analyst Reporting
In Short…
The leap from advanced auto complete to full autonomy when it comes to Coding Agents is too great.
Granted, coding projects vary in size and complexity, hence looking at benchmark scores helps.
There is a core tension between advanced auto complete and full autonomy…
This workflow layer (the bridge) addresses the core tension — autonomy needs guardrails to align with project practices, reducing errors in tasks.
And acting as a bridge from one side to another.
I love the idea of workflows being the temporary middle ground and the evolution to bridge reactive code completion and snippet suggestion to full autonomy and embedded workflow guidelines.
We are also seeing this in general AI Agent coding for business use-cases where AI Agents are being superseded by Agentic Workflows.
Going into more detail…
MIT coined the phrase, the shadow AI economy, this eludes to the divergence or chasm between what is being implemented and implementation realities, and the analyst reports.
Considering the image blow, a recent study focussed on what AI developers are working on, based on Stack Overflow questions and closed answers.
Exposing a dimension which is not often discussed in official and analysts reports.
AI supported Software engineering is evolving from reactive code completion to autonomous AI agents that tackle end-to-end tasks.
Issues struggled with by developers cover less glamorous topics like installs, figuring out dependancies, alignment and other issues.
A recent study focussed on what AI developers are working on, based on Stack Overflow questions and closed answers. Exposing a dimension which is not often discussed in official and analysts reports.
So it was interesting to see another study cover this topic.
In this case researchers looked in detail at the Claude Code activity in open-source GitHub repos.
Granted, this might not be completely enterprise related, but it is still a good indication of what developers are working on…
The study on Claude Code configurations analysing 328 files from 100 popular open-source GitHub repos.
The workflow layer forms the middle ground bridging from snippets to full autonomy.
Revealing practical realities of AI coding agent deployment that contrast with the often optimistic, enterprise-focused narratives from analysts.
Reports frequently highlight seamless productivity gains and top-down corporate rollouts, this empirical snapshot underscores a more gritty, developer-driven shadow ecosystem.
I would say that analysts often portray AI agents as autonomous out-of-the-box solutions for end-to-end tasks, but the study exposes the heavy lifting required…
72.6% of configs emphasise architecture rules like with devs manually grouping 2,492 sections into SE practices like testing (35.4%) and workflows.
Devs iteratively craft Claude.md files from simple bash commands to orchestrated sequences.
The dataset draws from real-world, dev-led projects across 23 languages, showing adoption driven by individual hackers rather than enterprise IT.
The study reveals that developers favour simple, text-based rules for guiding AI behaviour.
Development and growth are underscoring fragmented, trial-based growth.
The shift from code completion to end-to-end tasks (planning, testing, Git ops) requires broad configs.
Blending high-level overviews (39%) with low-level tactics — yet only 17.4% address tool configs explicitly.
This configuration ecosystem demands upfront dev investment, contrasting analyst claims of instant ROI, 20–30% productivity in related studies.
Overall the study paints a vivid picture of AI realities as a decentralised, config-intensive environment.
Lastly…
Agentic code assistants are autonomous AI systems that independentlyexecute complex software development workflows.
AI supported Software engineering is evolving from reactive code completion to autonomous AI agents that tackle end-to-end tasks.
The workflow layer forms the middle ground bridging from snippets to full autonomy.
Orchestrating sequences with human guardrails.
I love the idea of workflows being the temporary middle ground and the evolution to bridge reactive code completion and snippet suggestion to full autonomy and embedded workflow guidelines.
Obviously there is the question of the scope and complexity of the project.
But, workflows act as the bridge to orchestrate sequences, for now as we move towards advanced autonomy.
This workflow layer addresses the paper’s core tension — autonomy needs guardrails to align with project practices, reducing errors in tasks.
It’s not a strict linear phase but an emergent pattern in configs from popular GitHub projects.
Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. Language Models, AI Agents, Agentic Apps, Dev Frameworks & Data-Driven Tools shaping tomorrow.
