Our CEO, Matt Blumberg, has written extensively about agile on his blog and recently wrote a post about Return Path’s agile transformation — Agile Everywhere. Since we’ve been asked share more about our Agile Everywhere initiative, I’ve been expanding on Matt’s original post by writing a blog series that describes our transformation process.
Some of you might be wondering how we’re validating the success of this initiative. Measuring the impact of agile was not an easy task. Initially, we wanted each team to report publicly on the percentage of sprint goals completed so we could quantify each team’s success, and then determine whether we’d hit our goal (90% of teams hitting 80% of their goals). However, there were a few issues with this method:
-
The public reporting aspect of this seemed to impede team learning as teams became more focused on how others would view their metrics rather than the learning itself.
-
In determining how to measure each team’s activity, we discovered we would have to create a custom measurement for each team. For example, recruiting may use “time to fill” and marketing may use “number of leads generated”. Each team would then be measuring different things and make comparison difficult.
-
In measuring a team’s activity, we wouldn’t be measuring whether the team was doing the right work. The recruiting team may have a very short “time to fill” but they may be hiring for roles that don’t move the business forward
-
If we recorded each team’s activity, we would have to manage vast amounts of data that could detract from the more important work of coaching teams.
As a result, we realized we needed to modify our goal and we advocated to change our goal to something that would really move us in the right direction to getting all teams leveraging agile practices by the end of the year. Our goal became “100% of teams having a trained agile facilitator working with them by July 30.”
We were still missing a way to measure the effectiveness of implementing agile practices. Returning back to the initial reasons for the transformation, we wanted to raise accountability and improve the way teams worked to ultimately improve business performance. We decided to measure the impact of Agile Everywhere via global business performance metrics.
Through a lot of deep thinking and hard work, we landed on what we called the “benefit metric”. This metric is a conglomeration of many different measures of business performance, each of which falls into 3 main themes–operating efficiency, planning effectiveness, and business success. The idea is that the more measurements we input into the benefit metric the more powerful it became, as it reduces the noise from each individual metric.
For the year we measured, the benefit metric followed an upward trend, which clearly proved the value or our transformation. We still have a lot of work to do: as we know, with agile, we are never really “done”. Eventually, we’d like to get to a place where leveraging agile practices and principles is automatic and habitual, without the need for regular intervention from a coach. We want to get to a place where the operating system models we originally shared are outdated because all teams have created their own operating systems from their own experiences and learnings with agile practices.