Agile Procurement

The Executive has a unique power and responsibility with respect to initiating and supporting Agile technologies within government at the beginning and end of the software lifecycle. The Executive can demand that the procurement process — the genesis of much government software — support Agile. This will normally have to be accomplished within a regulatory framework and with procurement professionals.  In the Federal context, these are the Federal Acquisition Regulations and contract officers.

The Agile Manifesto insists on customer collaboration over contract negotiation. This is directly opposed to the traditional “Request for Proposal” process, which one attempts to precisely define what is needed and then perform competitive bidding to get the best value for the taxpayer. This basic approach grew out of civil engineering, and it works well for bridges and roads. It is anathema to good software development.

The Agile Executive must therefore change they way they think about procurement, and lead the organization into doing so as well. The fundamental concept has been articulated by Mark Schwartz: “Build functional teams, not software.”

You have to change the mental script. Here are some examples:

Instead of… Think…
Contracting for working software Contracting for software creation capability
Contracting to hit a deadline Contracting for progress
Writing a detailed RFP Insisting upon ongoing contractor/user interaction
Contracting for working software Contracting for software creation capability
We  want a functional system years from now We want something testable and improvable right away
We want a list of requirements We want a prototype that we can check with users
We want to evaluate how well firms can write an answer to an RFP We want to evaluate how well a firm can produce a prototype in a limited time

In practice this means that “Firm Fixed Price” as a contract mechanism must simply be abandoned, despite the persistent and erroneous belief that it reduces risk. Consider using a meritocratic competence process as pioneered by 18F’s Agile BPA.

If one is not attaching a financial incentive to the completion of a large piece of software, one must find alternative approaches to ensuring good work from contractors.  We recommend providing financial incentives on a periodic basis based on performance.  However, this may be politically uncomfortable and regulatorily tricky. Mark Schwartz uses a simple technique: use more than one firm, and provide more work to the firm(s) that seem to be performing better, and less work to the firm(s) that are performing less well.

You may have to ask your procurement office to step way outside their comfort zone and traditional way of doing things to accomplish this.  It is your job to make them do that, and it is their job to do it, so be prepared to break some eggs to make the omelette. If you allow hide-bound tradition in procurement to force you into detailed RFPs and financial incentive to firms for adherence to contracts rather than listening to the user and really delivering value, you have lost the game on the first move.

Using Agile in a Legacy Environment

Unfortunately, most government executives are placed in a legacy environment that is not of their own making.  This is true both of the procurement practices and the actual existing software.

Rewriting a legacy system is one of the toughest and most common jobs facing government executives. The absolute key to rewriting a legacy system is to find a way to do it one piece at a time. This has been articulated by Martin Fowler as the “Strangler Pattern”. The basic idea is to build Application Programming Interfaces (APIs) which encase particular components. What is encased by an API can be safely rewritten with minimal risk.

The Strangler Pattern and APIs are intimately connected with automated testing. Your engineers should understand this and be able to explain it to you.  Your job is to insist on a culture of automated testing as a risk-mitigation approach.  This will decrease the chance that you have an unpleasant experience testifying to Congress.
You may find engineers who tell you that a system cannot be rewritten using the Strangler Pattern.  If they tell you this, tell them to think harder about it.  If they continue to insist that it can’t be done, demand an outside opinion and a cogent explanation for why it can’t be done.  If your engineers insist on a position that they can’t articulate clearly enough for you and others to understand, then they are not doing their job and you have to fire them.  Only one in hundred legacy systems cannot be rewritten with the Strangler Pattern, and all of them involve some sort of weird hardware integrated with the software system.