In a recent article in Medium, the author wrote,

“The AI-native SDLC is a reimagined process that combines the old control objectives with new enforcement. Instead of a linear flow, the process becomes a loop, and AI is embedded at each point. The AI-native SDLC promotes automated handover and triggering of subsequent plays…

“Structurally, the six phases survive: plan, design, build, test, deploy, maintain. What changes is how the phases hand off.

“Instead of meetings, tickets, and sign-offs, each phase ends by committing a file to git, and the next phase starts by reading it:

“intent.md → spec.md → plan.md → code and tests → PR with review findings → incident record → back to a fresh intent.md.”

Wait – six phases? Perhaps without realizing it, the author is talking about waterfall, which I thought we had left behind a long time ago for most business software, keeping it only for extremely high assurance systems such as aircraft flight control systems, spacecraft systems, and other systems that must be perfect or someone dies. (My 2006 book High-Assurance Design explained the distinction in detail.)

Apurely spec-driven approach is a step backwards.

For business software, the Agile movement taught us that phases are counter-productive because they force us to get the requirements perfect, which is either prohibitively costly or impractical for business systems.

And now people are bringing back waterfall-like phases with AI-based coding?

To be fair, the Medium article refers to the diagnosis of production incidents as a source of requirements (the “intent.md” document): and in that situation, I would lean toward agreeing with its premise that one can begin with a definitive statement of intent. But outside of fixing production incidents, intention can only begin as highly tentative. Thus, if we are talking about maintenance, or indeed any domain in which the requirements are very narrowly-definable and clear-cut as some kind of “objective function”, then the article’s proposed AI-assisted waterfall-like flow makes sense; but outside of that it is missing the messy element of human-AI collaboration.

It is usually the case that several repos need to be changed at the same time.

I would not be concerned if it were not for the fact that I have seen lots and lots of articles like this, advocating for a “spec-driven” approach when working with AI coding agents. Programmers relish this concept because it replaces messy human discussion with nice and precise machine-readable specs – in effect reducing the whole flow to a kind of meta program – but that is not a prescription for an effective end product. Effective end products result only from those messy human discussions.

For most business software development, apurely spec-driven AI approach is a step backwards. And it is not how we built Throughline by the way.

But maybe the real issue is about what is considered a “spec”, and when it is created:

intent.md → spec.md → plan.md → code and tests → PR with review findings

although the PR process was and still is problematic for today’s systems,

intent.md → spec.md → plan.md → code and tests → PR with review findings

because it is usually the case that several repos need to be changed at the same time (especially if you change an API)...

The Importance of Conversation

It is rare that you can get an initial spec right. The Agile community shouted that loud and clear and they were right. Up-front specs make sense for protocol-based systems where the protocol has been carefully designed: you know exactly what the software needs to do, because you designed it carefully and simulated it. But that is not the case for 99% of business software; and it is well known that if you ask business people what software they need, they will tell you something that is only half right: you have to then build and give them something for them to see and possibly try and then figure out what they really need. The initial spec will be wrong.

Having AI coding agents does not change this.

A conversation is still needed – not just with users, but now also with the agent that will create the code. You need to explain what you think is needed to the agent, and ask it things like, “Does that make sense to you? Am I missing anything? Do you see any problems? How would you approach this problem?”

A conversation is still needed – not just with users, but now also with the agent that will create the code.

And once you, the business stakeholder you are dealing with, and the agent agree on the nature of the solution in terms of the requirements, you need to ask the agent things like “How would you implement that?” and have a conversation from which you end up convinced that the agent’s approach will be the right one.

At that point – and not before – you are ready to write a spec; but by then the spec has advanced greatly in complexity, so it is best to let the agent write it, and then you should read it; so ask the agent to write up a spec in easily understood language, explaining the problem it solves, the key concepts, and providing pseudocode for the key algorithms. That’s your spec. That is what should get saved as an artifact, in git or somewhere else (especially if multiple repos are affected).

It is best to let the agent write the spec, and then you should read it.

Then the AI agent should implement its spec, turning it into code (or into whatever form the product exists in). You can then examine what it created, and if changes are needed, you should ask the agent to update its design spec and implement the changes.

Putting your initial half-baked pathetic intent or spec into git is a waste of memory. No one will ever care about your initial wrong ideas. Even if you treat those initial ideas as an “intent”, no one cares. All that matters is the design: what was elaborated into code and its design-level rationale. That’s a valuable document. AI should create that because then it really reflects what was done, rather than what the developer thinks was done.

And then there is the plan: Coding agents create plans before they start working. You don’t need to log those – they are internal to the coding agent. You only need a plan if it involves you (the developer), and that is only the case if the changes will affect other systems that other agents are working on. The plan needs to include coordination with them. Otherwise, you don’t need to log a plan – it is just overkill.

Don’t Use Markdown For Designs

Also, these should not be markdown documents. They should be bona fide architecture documents, in self-contained HTML or PDF. The reason is that they are human readable and contain embedded diagrams, tables, and generally anything that might go into a document. Markdown is too limiting and does not print well or email well and non-programmers don’t know how to deal with it. Design documents need to be suitable for reading outside of the programming context.

For example, if a document pertains to how your system will create embeddings, that is something you might share for review with an AI expert – not a programmer. Or if the design pertains to how the system will use SolidWorks to perform clearance analysis on a mechanical component, that is something you might share with a mechanical engineer.

Design documents need to be suitable for reading outside of the programming context.

These architecture documents are not just about programming – they are also about the solution domain. They should not be buried in a repo where only programmers know to look: they should be up front in a design/requirements level directory of the repo, and they should be easily printable and emailable.

It is unfortunate that most git repo systems treat HTML and PDF documents as if they are text files; but one can always go to the file system where the document is locally present, and double-click on it to open it and read it.

For the Throughline system, there are currently 48 architecture documents. These all represent the final agreement between the programmer and the coding agent, fully explained by the agent using tables and SVG diagrams. They have helped immensely for review of the many strategies used to implement features. I personally reviewed many of them in PDF form and marked them up with notes, leading to more discussion and often changes – and a shared understanding of how the features were then implemented. Those documents are in the repo – not initial musings about features. If you want the latter, you should look to the project management system where user stories are defined.

What About TDD?

This brings up the issue of test-first development approaches. I will distinguish between “test-driven development” (TDD), which was originally about unit tests, and test-first approaches which include things like behavior-driven development (BDD). People use the term TDD in various ways, so I am defining how I use it here.

For agent-based programming, TDD is irrelevant: agents are writing the code and they just don’t work that way. Don’t force an agent to work in a way that is unnatural for it.

AI agents do not have our limitations and they don’t need BDD.

For BDD, you might choose to do that, but I would advise against it. It is far better to have a conversation with the agent about what you are trying to accomplish, rather than pinning it down to certain inputs and outputs. The agent does not benefit from such constraints. Once the agent knows what you are trying to accomplish, and you have agreed upon an approach with it, it will define the inputs and outputs that make sense and it will write the integration tests. Forget BDD – that is a relic of when humans struggled with complexity, but AI agents do not have our limitations and they don’t need BDD.

That said, the regression suite that is accumulated in the course of agent-based development is extremely valuable and you should ensure that it stays up to date and that it has sufficient coverage – not code coverage, but feature coverage and edge case coverage. Code coverage no longer matters.

Coding agents are brilliant at finding gaps if you explicitly ask them to.

You can achieve sufficient test coverage by discussing the feature and edge case coverage with the coding agent, asking it to identify coverage gaps and then to fill them. Coding agents are brilliant at finding gaps if you explicitly ask them to: it is up to you to know what to ask. In fact, that is the key in a very general sense to using agents well: knowing what to ask.

If you want to audit whether the coverage is sufficient, looking at a code coverage number is a poor approach: much better is to ask a coding agent to audit the coverage and give its opinion of the nature of the coverage gaps that it finds. You can’t reduce that to a number.

Never Let Agents Supervise Alone

There is a lot of effort today going into enabling agents to run for longer, without supervision. The basic approach is to have other agents watch and critique the work along the way.

That’s a great approach for improving the quality of output, but it does not replace human oversight. No agent replaces human judgment yet. Agents do not “live in our world” – they do not know our values and our fractal-like goal system or our internal hierarchy of needs. An agent only knows what we tell it and what it can discover, which is not the full picture.

Throughline is unique in that its agents have a vastly broader picture of what people are trying to do: that is because Throughline’s agents access what we call an intention network – a network of interconnected objectives, strategies, and plans that represent what people in the organization want to achieve and how. But even with that rich context, agents still need supervision.

Agents do not “live in our world” – they do not know our values and our fractal-like goal system or our internal hierarchy of needs.

Trying to enable an agent to run for 20% longer is the wrong approach: agents are already thousands of times faster than people, so making them 20% faster doesn’t help much – we humans are still the bottleneck when we examine what the agent has done and the choices it has made.

The right approach is to try to align the agents better, by giving them more and better quality context, which is Throughline’s approach; but we still need people in the loop. Agents make bad choices – all the time – and that is why it is better to catch those bad choices soon rather than after an agent has run unsupervised for a long time.

The Right Way to Use Agents (Today)

Given the state of AI today, humans need to stay in the loop, to catch and reverse the many, many bad choices that agents make. Agents don’t make a lot of technical mistakes – they are far less error-prone than people – but they make bad judgment calls. That is why we need to watch them closely.

When that is no longer the case, we won’t need people at all – not staff people, not managers, not CEOs – no one. So let’s rejoice that we still need humans in the loop – at least for now.

Agents don’t make a lot of technical mistakes – but they make bad judgment calls.

Given that, the best process is to have conversations with agents until we reach a consensus with the agent about what to do next. Then, if the work to be done is complex, have the agent write up its approach as a design document. Save that because it is a human-consumable artifact that explains what the agent will do and why. If the issues are risky, have domain experts review the design document.

Then let the agent proceed. And when it has finished, examine what it did. When you find issues, either ask it to explain the reason for the issue, or provide more instructions to address the issue. When the result is aligned with what you feel is optimal, the work is truly done.

Also, for software, this presumes that the work is occurring in a true DevOps pipeline. In a trueDevOps pipeline, integration tests are run frequently and are automated. There is no separate integration testing phase. There is no separate deployment integration testing: it happens all along, on a regular or on-demand basis. That’s why the term “DevOps” combines “Development” and “Operations”, because Development includes doing the things Operations used to do: deployment and monitoring. It’s all one phase now. If you are not doing that, you are not doing DevOps. That might be okay, but don’t use the term incorrectly.

Phases are waterfall. In a DevOps approach, there are no phases. AI does not change that.