Monday, February 22, 2010

From Concept to Product Backlog

xUnit Test Patterns: Refactoring Test CodeAt Agile 2009, Gerard Meszaros gave a talk on the steps needed to get to iteration 0. Iteration 0 is an iteration dedicated to setting up the development environment before the 1st real iteration is started. It usually includes installing development tools on workstations and team tools on servers, and priming the user story backlog so that it is ready for the iteration 1 planning meeting. A secondary objective includes calibrating the development team’s velocity and learning any new technologies that will be used.

Gerard shares some examples of poor upfront planning. These include situations where customer was learning on the job and they did not have the productivity to get things going and as a result they could never catch up on the story tests. Another was when fit tests kept breaking due to new logic being introduced and another was when user interface contained many inconsistencies.

Gerard attributes these problems due to lack of upfront planning leading to sub-optimal results. The big picture was missing. In agile planning these is constant tension between BDUF (Big Design Upfront) vs. LRM (Last Responsible Moment). We need to find the right balance between up front planning and decision deferral, and the right balance between high ceremony just because and just enough, just in time. Some ways to achieve this is by prototyping or staged style funding.

Agilists in general are familiar with the five levels of agile planning: Product vision, product roadmap, release plan, iteration plan, and daily plan. However, there is planning that needs to be done before we even get to this stage. Concepts need to be broken into ideas and then into core features. There needs to be an elevator statement and a description of the behavior of the product as well as a high level architecture, cost estimates, and eventually user stories. Gerard describes this as:

· Product Envisioning: concept, major features, and product design

· Product planning: risks, benefits, test strategy, release plan, effort estimate, cost estimate, skills list, budget.

· Project Execution: User Stories, story tests, business staffing, tech staffing, iteration plan, story tests.

Product Envisioning

The goal of product envisioning is to get a collective understanding of the product. We need to get everyone working towards a common purpose. Get everyone’s input and buy-in of the product and help everyone understand the big picture. This is done by workshops for envisioning the product to be built using brainstorming, listing functionality, users, and value. Outputs include the elevator statement, product box, major and differentiating features list, and low fidelity user interface prototype.

Product envisioning involves generating ideas from a concept and then developing core features. Techniques include using the product box and the elevator statement. Use product box technique to help team focus on what is really important. Have a graphic, 15-20 main features, 3-4 key differentiating features. Use the elevator statement to communicate in 30s what your product is about. Use this syntax: For who are dissatisfied with , our product is a that provides . Unlike we have assembled .

Product Design (not software) involves determining the look and feel (screens and navigation flow), capabilities and behavior. From there, you can drill down to high level architecture, cost estimates, and real plans. This becomes the scope of the product. We commit on these and not on the user stories. Only the features we are going to do next get broken up into user stories (just in time). User stories are equivalent of WBS. Detailed estimates are based on user stories and they are a way to organize the work we are going to do.

For Product design look at the big picture. Use case modeling, information and data modeling, interaction and graphic design, and protocol design. The traditional approaches involve user centered design (user roles or personas, detailed task analysis, usability labs) and use case modeling or use case documents. The agile approach involves lightweight user task modeling like actors and goals list or essential use case model, paper prototyping to define what the product looks like, and wizard of Oz testing to get feedback on that design.

In essential Use Case Model we define who are the users and what are the major transaction that user will perform and are trying to achieve. With Personas we describe character of person and ask frequently how would this persona react to this UI or behavior. Paper prototyping is a low fidelity screen mockups and a tangible representation of what app is about. We use it to get feedback from users to arrive at a better design. We do it as cheaply as possible can be manual or automated (using tools like balsamic).

Wizard of Oz Testing is used to validate the product design, find design defects quickly, make design decisions based on data, not speculation or opinion. This is done by testing early versions of the design with users thru simulated execution of paper prototypes. As users simulate actions on the paper, facilitator lays down new screens. Observers record any problems encountered and gather data on alternative design options. Test early, test often, and with actual users.

Product Planning

Next Gerard describes some steps in product planning:

Light Weight Risk Assessment: Cardstorm potential events and write risks on cards. Discuss likelihood and impact and categorize them as green, white and red. Brainstorm mitigation strategy for red risks. Monitor the white risks for change in probability or impact. Repeat every few months or when something significant changes.

Define Product Architecture: The goal here is to understand the key components and technologies that will comprise the finished product. We want to avoid high cost technical changes and reduce technology and schedule risk. We need to validate technology and make sure they all play well together. Activities include proposing architectures, evaluating technologies, building “steel thread”, executable skeleton, or tracer bullets. The architecture is validated via steel thread where just enough of the architecture is implemented to demonstrate working code and proving the architecture hangs together (most logic is hard-coded). This type of early integration puts all the pieces together and helps validate the architecture. It forces us to address technical risk, and estimate functionality. This also allows us to avoid test automation that looks more like a funnel instead of a pyramid. A funnel has large number of automated functional tests and very few automated unit tests. This results in tests that are very fragile and automation becomes difficult and expensive. On the other hand, the test automation pyramid has large number of very small unit tests, smaller number of functional tests for major components and even smaller number of tests for entire application and workflow.

Define test strategy and figure out what kind of testing is required (Unit, functional, component, and workflow), what needs to be automated and what challenges are there (legacy systems, binary data, time/date-based logic).

Release planning ensures the conceptual integrity of release. This involves figuring out the order of rolling out functionality and grouping of related functionality for a specific set of users. More kinds of functionality for other kinds of users can be added in subsequent releases. Release should include full spectrum of priorities (must haves, important, would be nice). This allows “wiggle room” for managing scope later to fit the budget. All priority 1 features should not go into release 1.

Estimating for release planning should be based on features and not user stories. Typically we won’t have all the user stories defined during release planning. We should estimate and sum the feature complexity and guess the feature point velocity to find fit or do more detailed estimating on subset of features. But always estimate complexity and then derive effort.

Managing scope involves finding ways to reduce total work to fit budget by having a more efficient process (increase team velocity by removing waste), or having fewer features, or simplifying features. Agile is the art of maximizing work not done. Look for thing you don’t have to do and don’t do them. Spend time on things that are more valuable.

To define product backlog, take features, think of the test cases and group them into similar things. Features are definition of scope. User stories are the implementation plan of that scope.

For story tests clarify requirements. They show what done looks like. Ensure that you begin with the end in mind, enumerate all the capabilities that must be supported, identify the success paths and the error scenarios and provide detailed examples of input data and responses. Use FIT.

Project Execution

Business/Customer Staffing:

It is important to get the right business resources for the customer team. These resources have strong business leadership and vision, understand the needs of users or can find them out quickly, are willing to learn new ways of working, and can plan ahead and not just react. They are collaborative, decisive, conceptual thinkers. The wrong resources can slow down the project or lead the team in the wrong direction.

The core business team needs to be staffed in time to participate in the envisioning phase. They need to be trained on how to do the customer job and start defining features, user stories and story tests. Additional “do-ers” can be added later but only after taking the time and bringing them up to speed on the vision.

Technical staffing:

Need senior developers early in process to estimate features, define architecture, and validate the architecture. Full staffing should wait for budget approval but need to take the time to bring them up to speed on the vision.

Onboarding the business and technical staff involves bringing them up to speed on the project background and vision including product concept, value proposition, release plans, features or user stories. In addition, include process background and norms including agile team practices, usability practices and technical practices.

Gerard wraps up by reaching iteration 0. By that time we should have entire team on hand (business and technical resources should be onboard), and a “bootstrap story” ready to go (ready to estimate, story tests, additional stories in case team gets done early).

This presentation is available on InfoQ at