Last week at AGL Live, leaders in modern procurement discussed challenges, lessons learned, and practical advice for measuring the success of work done on agile contracts.
- Polly Hall | Strategy Lead at DHS Procurement Innovation Lab
- Josh Seckel | Chief Solution Architect for Agile and DevOps at Sevatec
- Paula Wagner | Division Chief at U.S. Citizenship and Immigration Services
- Jonathan Mostowski | Acquisition Strategist at U.S. Digital Service
- Karl Fogel | Partner, Open Tech Strategies
Moderator: Ben Morris | Partner, STSI
Watch the video and view discussion summary below:
5:00 Is EVM (Earned Value Management) compatible with agile delivery?
6:00 Jonathan Mostowski’s take on EVM to measure work on agile contracts: “It’s an attempt to make a square peg fit into a round hole.”
8:27 Recommendations from the 809 panel examining this and other contracting practices: https://section809panel.org/media/updates/
10:18 Polly Hall says “there are more effective metrics” than EVM for showing satisfactory work on agile contracts – one of which is customer satisfaction
12:25 Getting to the heart of business value: what can we use to satisfy contracting officers to hold vendors accountable?
14:03 How U.S. Citizenship and Immigration Services (USCIS) is trying functional ways to measure success and performance across many agile teams (i.e., number of requirements delivered, rate of deployments, mean time to recover from outages)
15:50 How to measure ongoing contract success from an open source software perspective (i.e., independent third party deployability of the software throughout the development cycle)
18:29 How agile tools of transparency can replace what we think of as “reporting” or metrics (i.e., Slack, Trello, Jira)
21:04 What contracting officers need to understand about agile
22:51 Joshua Seckel: “Focus on outcomes, not outputs.”
25:30 How open source projects provide more accountability and quality (Hint: developers work faster and better when they’re working alongside and being reviewed by other engineers)
27:37 Why shorter, modular contract structures can improve performance and provide more functional products
30:23 Polly Hall: “Build incentives into a contract and be willing to walk away if performance isn’t there.”
30:51 What happens when contractors are prevented from doing good work by the government’s failure to follow agile practices
35:03 How government can work with vendors throughout the process to build trust and learn together
36:51 Karl Fogel shares a crucial question government customers should ask themselves
39:09 How do you scale trust and build communication when you have many people and contracts all working at the same time?
41:30 Paula Wagner shares techniques from USCIS for avoiding vendor lock-in and diversifying cloud infrastructure
43:38 Each vendor on a project must cooperate and collaborate with the other vendors — write it into the contract!
45:48 How working in the open can benefit a project — and why this is a struggle for risk-averse government agencies
49:48 How government can work on de-scaling work and scope rather than scaling teams (i.e., microservices)
51:50 Tactical advice and resources
54:54 Polly Hall: “At DHS we treat our procurements as agile projects, bringing the principles and rituals of agile into the procurement process — even including ‘retrospectives’ by interviewing offerers and using their feedback to iterate and improve.”