Saturday, August 15, 2009

The Dancing Agile Elephant: IBM Software Group’s Transition to Agile and Lean Development

At QCON San Francisco 2008, Sue McKinney gave a presentation on IBM’s agile adoption strategy. She started out by giving an overview of some of the business and operational challenges at IBM. These include innovating the business to differentiate and capture new value, heighten responsiveness and closer linkage to customers, and improved time to value. On the operations side, issues included better workload management, improved quality, improved project development cycle times, improved predictability on schedule, and making better use of resources to be more productive

Next Sue described the IBM environment. IBM has a global team, with different companies brought together by acquisition. There are 500 development teams in 5 divisions. Teams are as large as 800 or as small as 20. Very few teams are collocated. They are highly security conscious. They use many tools (due to acquisitions), and many platforms.

Sue describes how IBM software development transformation went from waterfall in the 1980s (rigid, late feedback, slow reaction to market changes) to iterative (customized RUP, community source and component reuse, emphasis on consumability), to agile (global reach, SOA, agile practices, outside-in development, tool and not rules, continuous learning and adaptive planning

Sue mentions how she sold agile as a way to deal with uncertainty, and to respond to changing markets. Agile gives them transparency to check where things are going and enables them to take a course correction if needed by meeting with stakeholders and deciding what change to make. She kept processes to deal with complexity (team size, geo distribution, compliance or regulatory issues) and move from lightweight to things that are more thorough and allow for traceability.

Next Sue lists some things to consider before getting started. These include getting management support, picking strong experienced leaders, picking the right project as a proof point (project with high visibility and some risk), providing the right education, tooling and governance, ability to allow change to occur, and keeping it simple.

Sue defined Agile as short time boxed iteration with stakeholder feedback. This creates automatic constraints (iteration length) that force the team to find defects earlier and address them. The team becomes more responsive and there is always a sense of urgency. There is also a notion of transparency with demos at the end of each iteration. The constraints forces teams to optimize and eliminate waste (automate). Share holder feedback causes us to focus on the essentials.

The initial project was a team of 200 people over 4 sites with 1 week iteration. 1st 2 months, put together infrastructure to enable the team (test case harnesses, continuous integration, static code analysis), then 1 week iterations began. The software was published for regular consumption and received regular feedback. For sustainability (220 out of 500 teams are using agile), they created a practices for distributed development:

1. Sound development governance principles: lightweight central governance mechanisms, development steering committee, architectural board, culture of sharing and reusem developer web site, centralized development services

2. Enable for success: tool not rules, community source, shared asset responsibility, common components, clearing house for dependency management, educate, enable, and empower

3. Execute agile/lean productivity: Discipline, adaptive development approaches, continous stakeholder feedback to understand changing needs, time boxed iterations, elimintate waste, increase visibility.

These three result in guiding principles for software development: Architecture blueprint, outside-in development, agile/lean approaches, modeling and componentization, fostering communities and sharing best practices

Next Sue describes how to get the adoption going they provided workshops to teach discipline agile (108 workshops/6000 attendees). They also assigned coaches to find out what’s working, and what is not and to improve technical debt, continuous integration, automation.

Sue concludes by describing some of the challenges they face and some solutions. The challenges include: geographically dispersed teams with different approaches to building software, many projects, and large teams, innovating the business to differentiate and capture new value. Solution approaches include: continuously transform development using new techniques and tools, enable and empower teams, collaborate more effectively at larger scales both internally and externally and share knowledge, assets and best practices.

This presentation is available on InfoQ at