Qualities, User Stories and sad state of Scrum requirements

I’ve been in Belgium for a week now working with a product company who adopted Scrum in April for one team and this week I helped them roll out Scrum to two more teams. All is going well and the teams are off and running. Scrum is a relatively simple method, once you learn the special lingo and how to run a Sprint planning meeting.

In preparation for the first Sprint planning meeting for one of the new teams, I was reviewing the product backlog with the product owner and scrum master. There were some user stories in the format:

As a <User>, I would like to <Do Something>, so that I can <Achieve some Goal>

This format follows Mike Cohn’s style and I rather like it for functional features. It’s short and sweet and hits the major concepts (like who is the specific user and what is their motivation).

But then we came across the following story:

As a Driver, I would like a faster load time for navigation, so I don’t have to wait for it to load.

On the surface, this story seems reasonable. But as we started to define acceptance criteria it became clear that this story would be really hard to estimate without asking “How much faster?”. And based on the answer it may not be doable in a Sprint by the team (nor estimate-able).

The product owner asked, “How do we write this as a user story so the team can estimate it?”.

My simple answer was, “you can’t”. User stories are fine for functional requirements, but they are completely useless for quality requirements (aka non-functional requirements). Many teams I work with new to Scrum often start writing user stories for new features, but either pretend the quality dimensions don’t exist or think the architects will magically handle it. Both of these assumptions are dangerous, especially for world-class products that must perform at very high levels and meet tight service level agreements (SLA’s).

But who can blame these teams? The thought leaders in the Scrum community reinforce this idea through their Certified Scrum Master courses. I should know, I took my SCM class from Ken Schwaber and Mike Cohn, arguably the most knowledgeable agile practitioners in the Scrum community. I recall 180+ slides of material in my 2 day class and not a single mention of how to define quality dimensions of a system. Lots of material on writing user stories on what a system should do, not a single bullet point on how well a system should perform.

How sad. Scrum’s most experienced thought leaders don’t know how to specify a system’s quality dimensions. And even if they did, they don’t bother to mention it in the SCM course. Sigh. I hope all the companies adopting Scrum save a lot of money due to increased efficiency over Waterfall. They’ll need this money when they have to spend weeks or months completely redesigning their system when the number of concurrent users quadruples and the architects say, “that was never in the user story” and the product owner said, “I thought you designed for that”. But I digress....

So how do you deal with system qualities in Scrum? It turns out Tom Gilb has a good method that I use, His assertion is that qualities can 1) be specified numerically and 2) systems can be engineered to meet specific levels of performance. Qualities can be specified using a minimum of six attributes:

Name: A unique name for the quality
Scale: “What” you’ll measure (aka the units of measure, such as seconds)
Meter: “How” you’ll measure (aka the device you’ll use to obtain measurements)
Target: The level of performance you’re aiming to achieve (how good it can be)
Constraint: The level of performance you’re trying to avoid (how bad it can be)
Benchmark: Your current level of performance.

So I worked with the team to re-write the story into the following quality specification:

Name: Navigation Start Up Time
Scale: Elapsed seconds from when Driver clicks Navigation icon until Navigation Main Menu appears (assumes satellite connection is present and working)
Meter: Stopwatch timed from user interface
Target: < 20 seconds <- performance of competing product
Fail: > 50 seconds
Benchmark [Build 1703; Oct 7; First start-up; Standard Environment]: 50 seconds

It turns out this was one of many Usability qualities, so we spent a little more time identifying the others. The product owner helped us with Target, Fail and Benchmarks for some and others we did not know, so we skipped for now. Once we had a good set, we worked with the development team to determine what level of performance could be achieved in the next Sprint. It turns out the team knew of two quick wins for improving navigation start up time and they estimated 13 story points would take ~10-15 seconds off the start up time. They also realized they could start the navigation start up process in the background earlier when the device is first turned on, thus giving the user the appearance of faster start up time even if it wasn’t really, so results could even be better from the user’s perspective.

All of this dialogue resulted because we simply put numbers to this quality. Prior to this, customers said “it’s slow, make it faster.” and the development team was facing a never-ending user story, not knowing how fast “make it faster” meant.

This whole interaction, which was very positive and quite easy once the team got the hang of it, made me think about product backlogs and prioritization and how quality improvements needed to be prioritized alongside user stories for release and sprint planning. In addition to new user stories, product owners should prioritize specific levels of improvement for key system qualities and create sprint plans that focus on both “what the system will do” as well as “how well it’ll perform”. These two dimensions together make for a more complete plan that not only helps set expectations with product owners and customers, but also helps architects ensure the system’s design is capable of meeting the required performance levels.

Perhaps somewhere down the road this type of thinking will become part of the mainstream agile community and possibly even integrated into the SCM classes...


P.S. In case you’re wondering about the cow picture, I took it on the way back from the Abdij St Sixtus abbey in Westvleteren where the monks brew the world’s best beer. I was zipping down the country road and saw this cow, so I pulled over and snapped this shot. It’s got nothing to do with qualities or the sad state of Scrum requirements, but I like it nonetheless.