Embrace Uncertainty

At CommuniTech, Jeff Patton gave a talk about agile software development. Jeff starts by giving an introduction to agile. He mentions that it is nothing new. Scrum (1986), Crystal (1997), FDD(1998), XP(2000) all lead to the agile manifesto was in 2001. Agile development describes a class of approaches and not just a single approach. It combines elements of Scrum, DSDM, Crystal, FDD and XP.

Jeff then gives a primer on Scrum. A product owner sets product goals and adds requirements or features to a product backlog, he then takes highest priority items and elaborate them into details and start a development cycle of 2 to 4 weeks. Team has a daily stand up meeting to follow progress. At the end of the iteration, they evaluate the working software and get feedback which might affect the product backlog and the next iteration.

Jeff shows how the picture of the scrum process looks like a snowman. The head is stories that get packaged into an iteration. Every iteration does not have to result in a releasable product. It’s a number of iterations that form a release, and a number of releases form the product. Each cycle feeds into planning. At the iteration level, feedback is used to better understand the problem and to reduce risk. At the release level, the feedback ensures proper planning for releasing value. At the product level, the feedback is to update the roadmap.

Next Jeff presents 3 very entertaining scenarios (using personas and music lyrics) which demonstrate some practices that might lead to problems when using an agile model. He concludes that iterate does not mean increment. In incrementing we are piecing things together and it calls for an early well formed idea. Iterate builds a rough version, validates it, and then slowly builds up quality. Quality here refers to look and feel, characteristics and features. The client needs to prioritize the goals that generate return on investment. The developer needs to know what the client wants but always keep in mind what the ultimate goal is. They also need to push decisions to the last responsible moment and build up feature quality iteration by iteration. Instead of writing stories about what to do, write stories about what it intends to accomplish. Have a goal, take action, evaluate the action, and then evaluate if the goal got accomplished. Push decisions to the last responsible moment. Each story has the following characteristics:

1. Necessity: The minimum needed to get working software.

2. Flexibility: what are some alternative ways of doing it, what additional data we want to capture.

3. Safety: better validation rule to avoid ugly error messages.

4. Comfort, luxury, and performance: more usable, sexier to look at (animation), hot keys.

Jeff recommends starting out with the necessities and then slowly build up the product.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Uncertainty-Jeff-Patton

Introduction to Project Estimation

At Devoxx08, Giovanni Asproni gave a talk on Project Estimation. He emphasized the need to distinguish estimate from targets and commitments. Estimates are approximate estimation of the value, number or quantity of something (2 to 4 days, 10 to 20 million). A target is a desirable business objective (Support x users, do not exceed 1 million dollars). Commitment is a promise to deliver functionality with a certain quality by a certain date (search functionality will be available in next release). These three are independent of each other, but you should set targets and commitments based on your estimates. The estimates are not negotiable. But the targets and commitments are. The estimates are used to determine if the targets and commitments are realistic. Usual estimates include size, effort, time, cost, risk. Given some constraints (time and cost), estimate the other.

Estimates should be accurate and not precise. This means use the correct measurement unit. If something will take years, do not estimate in days or hours. Hours will be more precise, but it will be less accurate than a more appropriate measurement unit like years.

The cone of uncertainty shows that your estimates are off by a margin of error at the beginning and gets more and more accurate as the project moves along. The cone is the same for sequential or iterative projects. Do not commit at the beginning of the cone, but at a later date after several iterations or at the product design specification.

People who do the work should create the estimates. All assumptions should be taken into account. All tasks should be included. Make sure to plan for leave/absence and side tasks such as phone, email, meeting, browsing.

There are several techniques for estimation:

1) Count, compute, judge: count if you can, compute if you can’t, judge as a last resort.

2) Mathematical (COCOMO): Tools available but based on “knobs” (inputs unknown/inaccurate).

3) Historical: Industry, company or project data. Can be accurate, avoids wishful thinking.

4) Analogy: Compare to similar projects, simple to implements but subjective and less accurate.

5) Proxy: Use proxy to effort like story points or t-shirt size. Can be highly accurate but require experience.

6) Decompose and recompose: Split into small chunks and aggregate. Can be accurate but be careful because sum of individual chucks will not equal whole.

7) Expert Judgment: Individual or group (Delphi, planning poker). Can be highly accurate, but need experts.

More than one can be used at the same time as a sanity check. Start with historical data if available. If not use, expert judgment, proxy, or analogy. Keep track of your data and use it to improve as you go along. Estimation is an ongoing activity. Refine the estimates as you go along and remember that estimates will always be wrong!

This presentation is available on Parleys at http://parleys.com/#st=5&sl=1&id=14

Technical Practices Turning the Agile Dial to Eleven

Craig Smith and Paul King gave an experience report at Agile 2008 about technical agile practices. They started out with an initial goal of 100% of code coverage for unit test, all production code paired and test driven, minimal design up front with appreciation for when such design made sense, customer focused outcomes, full continuous integration, daily pair rotation, continuous improvements through retrospectives, high levels of automation (installers, vmware instances), and light weight metrics to monitor progress.

Using quality metrics forced team to have a common way of doing things. They tracked duplications, and enforced 0% duplication (4 lines limit). They also tracked method and class complexity & size and kept extremely simple classes and code with approximately 10 lines per method, 80 lines per class.

For testing boundaries used for 3rd party apis, they used an IDE plugin to create static boundaries, used Groovy to automatically eliminate checked exception through a language feature, and used auto boundaries of an interface

To ease the use of mock objects, they built a framework to avoid boiler plate code. They had the IDE convert code into expectations, and used annotation for instance/mock creation.

They implemented auto checking for nulls for public methods and public constructors, auto checking getters and setters, and checked for project wide rules to do with final, equals and hashcode. Other numerous types of classes had further checks for immutable, serializable, data, and domain.

They disposed of interaction based tests. The IDE could recreate them whenever refactoring was required.

They wrapped up by discussing future direction of acceptance-TDD or functional-TDD, evolving how they use TDD and CI (groovy, easyB), leveraging a broader base of testing approaches (jester, AllPairs, Theories, Grids), further using DSL, and further using Groove, or Scala.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Super-Agile-Craig-Smith-and-Paul-King

Is Being Agile Your Favorite Thing?

Here's a look at a team that loves being Agile...

Measuring Agile in the Enterprise: 5 Success Factors for Large Scale Agile Adoption

At Agile 2008 Michael Mah discussed Measuring Agile in the Enterprise. He starts by mentioning that Business will have to soon make a decision to reduce costs. 1) ship development to India, 2) adopt agile or 3) ship to India and adopt agile.

Michael then shares metrics (size, time, effort, defects, and productivity) collected from agile projects against industry averages. The data was mainly from BMC and Follet and looked at cost, schedule, defects, and staffing showing better results for the mature agile teams. Then Michael did a trendline assessment of staffing, schedule, effort, and defects. The data showed the agile project team size is fairly typical, bug rates are significantly lower (they start out higher but fall way down as team matures), and as a whole achieve faster speed. Productivity index (size/(time*effort)) improves the longer you’ve been doing agile, so the sooner you start the better.

Finally Michael interviews Walter Bodwell and describes the BMC secret sauce for success:

1. Buy in: need VP or higher senior executive sponsor, scrum master training, and core team energized and passionate

2. Staying releasable: have nightly builds/tests, 2 week iteration demos, frequent, rigorous code reviews

3. Dusk to dawn team work: Communication techniques for information flow, Wikis, video conferencing, periodic onsite meetings, collocated release planning, scrum of scrum meetings

4. Backlogs: One master backlog and multiple backlog management, one setup for user stories across teams, requirements architect to interface product management and R&D

5. Holding back the waterfall: Use Test Driven Development, retrospective meetings to not regress into old waterfall habits, outside source to audit the process

 This presentation is available on InfoQ at http://www.infoq.com/presentations/5-Success-Factors-Michael-Mah

Forging a New Alliance

At ThoughtWorks quarterly conference, Scott Shaw and Martin Fowler discussed the communication gap that exists between IT and the business side. Originally, IT department was created to centralize desktop administration and to gather MIS data. IT managed billing systems, accounting system or data processing on a centralized computer system. Things changed with the introduction of the PC. A broader range of business functionality came under IT functionality. Martin predicts that custom application provision will see IT department effectively disappear. Commodity standard stuff will move to the cloud and get hosted in a data center while custom software will move closer to the business side and the way to thrive in the IT world is to move closer to the business people.

Next they discuss some disturbing trends in IT that result in this communication divide between business and IT.

1. Slow to market: They note statistics that show about 34% of IT projects are successful. Here success is defined as on time and on budget. Martin notes that success should really be measured as providing value. They also show an input value stream that shows a typical example in which out of 90 days, 18 days are spent adding value and 70 days are spent in waste (waiting for handoffs). To have a successful rapid feedback loop, you need to have minimal latency.

2. Bloatware: This investment in large software platform that is intended to make it more responsive to the business. They give examples showing the same platforms growing to 200x their original size over the past 15 years, yet they mostly still provide the same business value.

3. War for talent: People fleeing the industry. The job is really boring. We took thinking and the design part away from people writing the code. There is no sense of ownership or context around what they are doing.

Next they cover some encouraging trends:

Collaborative team: Face to face communication, joint ownership of outcome, multi skilled people where developers can code and learn business domain (Make jog more interesting because of variety of task), and continuous improvements (plan, do, check). This results in better responsiveness, richer communication, higher quality, and improved staff retention.

Rise of craftsmanship: Master apprentice relationship. Attention to tools and quality and simplicity. Polyglot programmers. Software is all about design. Design is important because of its role in the long term of things. We should not see software as a cycle of building something and then it is delivered and we do not see it again. There should not be a disconnect between building and supporting something. Looking at long term brings out importance of design. It is important to have a code base that can absorb changes and gives the right degree of long term horizon to think of investments.

Perpetual beta: get software out there for people to try, fund things incrementally. Get away from project mentality and move towards product support mentality.

User experience design: 65% of features are rarely or never used Standish group. Bring user experience design into the iterative cycle of user needs analysis, collaborative design with users, design refinements, and test with users.

Domain Driven Design: enhance communication of internals of software. Use same language between business people and developers.

Domain Specific languages: makes the code readable to the business side making it easier for them to understand the code and improves communication.

They also gave several examples of case studies where these trends have helped out. They conclude that today business and IT are separate. In 2020 business and IT will come much closer to together. How are we going to survive the post IT world? What techniques and approaches are we going to use?

This presentation is available on InfoQ at  http://www.infoq.com/presentations/New-Alliance-Shaw-Fowler

Fostering Software Craftsmanship in a Corporate Setting

Scott Dillman gave a talk at Agile 2008 about fostering software craftsmanship in a corporate setting. Scott defined software craftsmanship as taking responsibility, continuous learning, rejecting specialization, pride in quality work, passing on knowledge, and meeting professional standards (test driven development, continuous integration, coding patterns and practices).

Next Scott described 3 steps to evolve a novice software developer into a craftsman.

1. Evaluate the current state of the organization through interviews, surveys and metrics. Surveys can be reused at a later date to evaluate progress. Collecting metrics from automated builds like % passing builds, builds/day, build duration, fail build reasons, code coverage, code complexity. Track historical trends.

2. Educate via pair programming, centralized resources (wiki, educational content, knowledge base, moderated technical forum, book reports, tools and code repository), educational sessions(teasers, lightning sessions, tech talks, workshops), craftsmanship day, 10% time and on team mentoring. To get upper management buy in, highlight benefits, cite case studies, link to corporate goals. Upper management has to allocate time and recognize and reward participation.

3. Measure success by interviews, surveys, metrics, and performance reviews. Make sure reviews have goals that motivate and educate, are relevant, specific, actionable, and verifiable. Have the right people perform the reviews, do 360 reviews, and encourage and reward participation.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Craftsmanship-Scott-Dillman

Prioritizing Product Backlog

At Agile 2008, Mike Cohn gave a talk about product backlog prioritization. Mike starts out by discussing the right level of prioritizing. A well formed product backlog should take the form of an iceberg. Small on the top (sprints), a little bigger down the middle (releases), and much bigger down the bottom (future releases). Mike believes that about 10% of current iteration should involve planning and prioritizing for the next iteration.

He then describes 3 levels of user stories. A user story which is a description of desired functionality told from the perspective of user or client, epics which are very large user stories and themes which are a collection of related user stories. An example of an epic might be as VP of marketing, I want to select type of campaign (direct mail, tv, radio, etc) so that I can identify and repeat profitable ones. This can then move up the iceberg and become as VP of marketing, I want to see direct mailings when reviewing historical campaigns. Mike recommends prioritizing at the theme or epic level. It is a lot of work to prioritize at the user story level and not very meaningful.

Mike then covers 4 techniques for prioritization:

1. Kano analysis:

Kano identifies 3 types of features to measure customer satisfaction:

• 1. Exciter or delighter: Exciter - feature you did not know you wanted until you saw it.
• 2. Mandatory or baseline: must be present for users to be satisfied
• 3. Linear: the more of it the better

Agile Estimating and PlanningAgile Estimating and PlanningThis can be done by expert opinion or surveying a small group of users (20 to 30) and asking both functional and dysfunctional questions like how do you feel if a feature is included and how do you feel if a feature is absent. The available answers are standard: I like it, I expect it, I’m neutral, I can live with it, and I dislike it. The results can then be matched to the Kano matrix to find out if a feature is an exciter, mandatory or linear. Others might fall in a questionable, reverse, or indifferent category. Mike recommends prioritizing by including all mandatory, some linear and leaving room for a few exciters.



2. Theme screening:

Identify 5 to 9 selection criteria for what is important in the next release (Importance to existing customers, competiveness, generating revenue, etc…). Identify a baseline theme that is likely to be in the next release and one that everybody understands (not the best one). For each theme, score the theme based on each selection criteria and compare it to the baseline theme using +1, 0, or -1. Add up the results for each theme and then rank them. Open the highest ranking themes and look at individual stories and use expert judgment on them.

Agile and Beyond – The Power of Aspirational Teams

At Agile 2008, Tim gave a talk on appreciative inquiry. Tim starts by defining appreciative inquiry as the study and exploration of what gives life to human systems when they function at their best. It suggests strengths, hopes and successes are transformational. We control a lot more of our destiny than we are aware of. Agile is similar. It is not just about process and practices but is a belief in achievement. Software development always brings up bad news. We need to think of what we do good in software.

Next Tim asks who defends the honor of the agile manifesto and principles. It’s easy to just adopt some of the principles and skip others. It is easy to assume that we cannot do all the practices and then not aspire for more. The best teams adopt all practices because they know that all the practices are dependent on each other.

Tim then covers several areas to influence appreciative inquire:

1. Working together: Pulling the load together, having fun, being excited about achieving.

2. Project chartering: It helps team understand the parameters you work in like defining success and determining who is part of the community. It also helps self organize over a common vision and goal and forms a contract that binds together the rules of the teams/practices.

3. Group sharing: encourages new ideas, gets more people involved in work.

4. Story cards and planning boards. Puts focus on collaboration. Use magnetic boards, make it fun, try magnetic avatars. Also, instead of statuses of in development, in test, in QA, use started, in progress, done which encompasses the different roles. Story cards are placeholders for a bigger conversation. Important to follow format of Story cards “as role, I want, so that”. Also get good pens to encourage good card writing (dark and thick ). Physical cards much better than huddling over a spreadsheet. If not hand written, print them out.

5. Visualize progress - Burn down chart vs. burn up chart. In general we like to see thing go up kind like a profit graph. For forecasting, simply counting story cards done vs. story cards not complete can be enough. Graph can still be drawn manually.

6. Gold cards: give 2 per developers to use for working on areas where they can innovate.

Concurrency – Past and Present

At QCON London 2008, Brian Geotz gave a talk on concurrency. Brian starts by mentioning that concurrency is hard. It is unnatural, error prone and untestable, hard for most programmers to use. He then explains the basics concepts of concurrency. A thread is a sequential process with its own program counter and call stack. Thread share VM wide resources such as memory, file handles, and security credentials. The pros of threads is fine grained data sharing between threads. However, that is also its disadvantage. Threads execute concurrently unless you use explicit locking to ensure that threads take turns accessing critical resources (synchronize key word in java).

If mutable data is shared between threads, all access requires synchronization. Failing to follow this rule results in data race which results in reading the wrong value or writing the wrong value. This generally occurs whenever there is a if something then do something or ready-modify-write. In the 1st case, another thread might change the value between the time we do the check and the time we call the method. In the second case, 1 statement is actually 3 statements like x++. Again here another thread might read the same value and then increment it by just 1 when it should have gotten incremented by 2.

Adding synchronized on methods in the class that modify data is a start (account with debit/credit), but then we have to be aware of the locking mechanism when composing operations in other classes (accountManager with transfer).

Programmers have to specify which operations are required to be atomic. But when doing so, this might lead to deacklock. Example, transferMoney(me, you, 100) and transferMoney(you, me, 50). One way to deal with this, is to induce a lock ordering. Example, always acquire lock 1st on the smaller value.

If from

synchronize(from) {

synchronize(to) {

}

}


Next Brian explains how this is getting more and more complicated. The main reason is because there is a fundamental tension between concurrency and OO design. OO encourage encapsulation and hiding of implementation details. OO also encourages composition, but composing thread safe objects requires knowing how they implement locking in order to participate in their locking protocols and know how to avoid deadlocks. However, the language is hiding the implementation details.

Next Brian offers some solutions to this problem. One solution is software transactional memory (STM). Explicit locks are replaced by transactional boundaries that define atomic blocks. The VM then provides locking nesting semantics and choose a locking strategy. This is under research and promising but not yet available.

Threads and locks are just one model for concurrency. Lock based concurrency rules hold locks when accessing shared mutable state and hold locks for the duration of atomic operations. Alternatives are don’t mutate state or don’t share state. Functional languages like Haskell or JOCaml have no mutable state. In java, we should get in the habit of making everything final unless you have a clear need to make it mutable. In Erlang everything is an Actor (like a lightweight thread). It has a designated behavior for when a message is received. No shared state. Scala has a similar actor library.

Brian concludes that in java, we can try to restore predictability by limiting concurrent interactions to well defined points, limiting sharing, limiting immutability. Concurrency is hard, so minimize the amount of code that has to deal with concurrency (isolate concurrency in concurrent components such as blocking queues and isolate code that accesses shared state in frameworks), use immutable objects wherever you can. Sometimes it is cheaper to share a non thread safe object by copying than to make it thread safe.


This presentation is available on InfoQ at http://www.infoq.com/presentations/goetz-concurrency-past-present

10 Ways to Screw Up With Scrum and XP

At Agile 2008, Henrik Kniberg gave a presentation on 10 ways to screw up despite Scrum and XP

1. Believing the Hype: That is, believing that if you go agile, all problems will disappear. You throw out everything you are doing now even if it works and then get the latest and best tools and focus on perfection.

2. Definition of Done: But you said you were done? Not having a definition, or not obeying the definition, or having done be outside of the teams control (no access to production). Good examples of Done are: unit and integration tested, acceptance tested and deployed to demo site. Or Releasable (acceptance tested, release notes written) and no increased technical cost.

3. Velocity: Not having one makes it hard to create a release plan. Having one and misusing it by comparing different team velocities. Or having a yo-yo velocity where it is high one sprint and very low on another sprint (indicates spending a lot of time fixing bugs).

4. Retrospectives: We are very busy so we skip it. And if we do it, changes and recommendations are ignored and not implemented. Having unwanted people in the meeting where the team feels uncomfortable and do not participate. Punishing the team for bad changes (the whole point is to try something and later evaluate it to see if things improved).

5. Team commitment: Team is pressured, Team is not sitting together, team does not track and learn and keeps repeating the same mistakes. Team is always undercommiting (team always looks good) or always over committing or has a velocity of 0 (doing a lot of stuff but not finishing stories to the end).

6. Technical Debt: No time to write unit tests or refactor code. This results in duplicate code, unreadable code, lack of code coverage and all add to technical debt. Technical debt slows team down. Things become harder. This also leads to inappropriate solutions like a big bang re-write or fixing the product instead of the process.

7. Teamwork: Fixed roles (no one helping anyone else), personal backlogs (results with attitude of at least I finished my stuff!), not helping each others, personal incentive models, implementing all stories in parallel, management interference (team not self managing).

8. Product backlog/product owner and customer: Product owner does not have time to maintain product backlog so team does not have one or it is not prioritized. Product owner does not know the system or does not have the authority and thus cannot make decisions. Having multiple product owners that do not agree with each other. Product owner becoming a bottleneck where no one can do something unless they talk to the Product owner 1st. Product owner not always available.

9. Mergophobia: Fear of merging code. Not integrating early and often, No branching policies, not taking responsibility (everybody ignoring team rules regarding code checkin),

10. Spring backlog or taskboard: Does not exist, too far from the team (not easily accessible), too complicated (many states to keep track off ) result in it not being used during daily scrum, not owned by the team, no burndown, not updated daily, warning signs ignored.

11. Worry about 1 – 10. Problems are normal. Don’t panic. Don’t despair. Tackle them gradually, prioritize and try to improve sprint after sprint.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Fail-Scrum-Henrik-Kniberg

Future Directions for Agile

At Agile 2008 David Anderson talks about agile principles, practices and the community. David covers multiple topics that reveal how CMMI, Kanban and Real Options Theory help in making organizations scale, lean, and of high maturity.

David mentions that trying to scale agile across an enterprise is forcing a change on the community. We are trying to expand agile to the entire IT department and even more to other areas like marketing and recruiting. We need a wider definition of agile that is not narrowly focused on software development. We have the manifesto and the principles, but it is 7 years old and in badly need of an update. David stresses that we should not underestimate the value of what the manifesto gave us, but it is not everything of what it means to be agile. David controversially states that the agile manifesto is superstitiously declared.

Next David clears up the ideas behind the manifesto and how can we capture them as a model in 3 underlying paradigms

1. High trust, high social capital, highly collaborative culture

David describes the different cultures between the traditional and agile communities. The traditional community was rooted in government and developed a body of knowledge from defense. The high cost of failure (money, lives, politics) makes people conservatives, risk averse, and have low trust. On the other hand, the agile community was started by thought leaders from start-up companies building web sites. They were innovating daily, empowering people to release new functionality with an assumed zero cost of failure. They realized that in order to innovate at a fast pace, there was no time to write requirement and negotiate contracts. It was better to let people work and adopt a failure tolerant attitude. This was a highly collaborative approach with a high trust liberal culture. This led to the 3rd value of the agile manifesto, customer collaboration (high trust) over contract negotiation (low trust). Self organization or empowering, delegating and tolerating failure through adoption of high trust culture is better than contract and audits in low trust culture.

2. Perfect is the enemy of good enough.

David mentions that CMMI taught us that project success came from organizational maturity. Level 1 is chaos, Level 2 is repeatable and Level 3 is a defined process. At this level we should be able to deliver successfully every time. So the assumption was that we need to pursue perfection and eliminate ambiguity and uncertainty. We need accurate plans, analysis, estimates, and diagramming. By the time we are done, they are already out of date. This led to the agile manifesto value of working software over comprehensive documentation.

“We Suck Less” is Not Enough

At Agile 2008, David Douglas and Robin Dymond described the current state of agile adoption and discussed what the legacy of agile will be. They mentioned Forester research from 2007 that says 17% of North American and European companies and are now using agile. They however noted that there is a lot of confusion on what agile really is and described the Nokia test to find out if a team is agile:

1. Iteration must be time boxed to less than 4 weeks

2. SW features must be tested and working at end of each iteration

3. Iteration must start before spec is complete

4. Do you know who the product owner is

5. There is a product backlog prioritized by business value

6. Product backlog has estimates done by team

7. Team knows there velocity

8. There are no managers disrupting the work

They then warned that the attitude of we suck less now than before sets the bar really low. The business will be satisfied and will stop when things get a little better than before, but they are not even close to how much better they can get. Moving from ok to great is a journey that takes time and effort. There are levels of agility that companies can get to based on their level of commitment.

Next they mentioned that there a perception that agile is just a set of tools and practices like Rapid Application Development or that it is cheap, easy, and runs on its own, or that it is a project solution instead of a system solution. Agile is an entire work system. It can produce 5x+ productivity gains, requires organizational redesign, requires significant change management component. We as agilest are currently feeding this perception by training teams on techniques and consciously encouraging growth over quality.



Next, they describe several large organizations that adopted agile and then moved away from it. There were early warning signs of agile adoption stagnation like early declaration of success, increased time slicing and borrowing developers for “important” project, lack of commitment or investment, flight exodus of change agents, executive support waning, continued emphasis on cost reduction without the corresponding ROI and continual disconnect between IT and business and a lack of understanding on how IT delivers value to the business.

The Development of a New Car at Toyota

Kenji Hiranabe gave a presentation at Agile 2008 about Lean at Toyota. He talks about Nobuaki Katayama, former Chief Engineer at Toyota, and how they build new cars. There are 3 phases to car building:

1. Planning and concept development: concept, style, market research, pre-development, cost and profit target. Spend a lot of time in this phase. Start out with a good concept, then visit dealers and users and get feedback to see if this concept work and then redo. The next 2 phases are management of milestones.

2. Real car development: Design prototyping, evaluation

3. Production and sales:

Managing big projects:

1. Planning and concept management: project backbone, clear appealing points are needed, 1st concept and plan has to be refined until satisfaction, discuss multiple opinions from wide views, rules to avoid immature “go”, calm evaluation with 3rd person

2. Development process management: needs to be a strong dedicated leader with org that supports him, with clear milestones, and supporting people to achieve them, rules to avoid proceeding before achieving them, mechanics to avoid “too late”, trigger of 3rd person, honest communication (face to face), bad news 1st rule, early and reliable backup plan.

3. Resource management: people and money, work hard does not work, visualization estimate based on actual record, make projects slim. Cost: Visualize the target cost, leader owns buffer, assign target cost based on actual record, value sense of completion, avoid sense of being “forced to do”, plan in detail and follow up and adjust seeing the whole

4. Cost management: Reduce cost means save profit so that the company can grow, everyone and every division think about global competition and cost reduction. Culture of do not complain but cooperate. Analyze past and present cost. Use it as base cost data. Reduction of cost is accumulated step by step by everybody. Value engineering is the technical competence. Each team is responsible for reducing cost.

5. Benchmarking: it’s about knowing competitors strategy. Not copy others techniques, but focus of parts that are better than ours. Set performance targets including ahead and behind time difference. Study designs of 2 to 3 years ago and develop product 1 to 2 years away. Use it for continuous competiveness.

6. Leader attitude: Leadership, teamwork, macroview, passion are good. Dictator, one man shop, leave all to subordinates, charismatic are bad for leadership.


Kenji then contrasts “do it until it’s done” vs. “do it until the time”. Mr. Katayama explained that for phase 1 use do it until it is done and for phases 2 and 3, do it until time.

Kenji wraps up by mentioning the Nobuaki Katayama values consistency and integrity over passion and strong leadership, values engineering thinking, and tries to understand new ideas.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Toyota-Kenji-Hiranabe

The Yawning Crevasse of Doom

Martin Fowler and Dan North gave a talk at QCON London 2007 about communication gaps.

They started about by describing the cost of communications gaps between customers, users and developers. This cost is obvious in systems that completely fail, but sometimes the cost is not apparent like when about 70% of features delivered are hardly or never used. In this case there is also a cost of failed opportunity to create something more valuable instead. These are costs that we do not even notice.

They suggest 2 approaches to close the communication gap:

1. Ferry men approach: A group of people act as intermediary. They are ferried between the business folks and the developers and act as translators. The business people never talk to the developers.

2. Bridge approach: Provide structure and mechanism for direct communication between developers and business people.

Dan and Martin prefer to use bridges than to ferry men. Dan gives an example of how different solutions can result out of this direct communication or even entire new requirements can appear. He recalls working on a project were the business folks required some printing functionality. When probed, it was discovered that printing was needed to key in data from the printout into another system. Instead of printing and rekeying, Dan suggested to simply sending the data directly to the other system. This was a simple solution that would not have come out if there wasn’t any direct communication so that the developer could really understand what the problem is.

They give 2 reasons for preferring the bridge or the ferry.

1. Efficiency of communication: Use analyst to enable everyone to share knowledge instead of just being a conveyer of information.

2. Caring and Motivation. Having direct contact, you can actually see who you are making a difference for. That is more motivating then looking at a requirements document.

Next, Dan and Martin discuss what techniques effect communication and help us build the bridge:

Succeeding with Agile: A guide to Transitioning

Mike Cohn gave a talk at Agile 2007 about how to succeed in Agile based on his experiences with different clients transitioning and successfully transforming their organization.

Mike starts out by addressing why organizational change is hard:

1. It’s not top down where a strong leader sets a vision and people follow it, and it is not bottom up where team start and others follow until you run into groups that are powerful enough to squelch transition.

2. Best practices are tempting, but once we identify something as best, we stop changing. We have to keep inspecting and adapting otherwise we are not agile.

3. The transition has to also go with a transition in the development process

4. We cannot predict how the organization is going to respond. We want to effect the organizational change in an incremental manner.

Mike describes how if we break things up into small pieces, work on the smaller more manageable pieces and then try to put them back together we do not necessarily get the whole picture. We need to think of organization as complex adaptive system. We need to look at success as achieving a good fit with the environment instead of as closing the gap with the desired state. We have to acknowledge that agents are closing local goals and closing local gaps. They are thinking about their own job security and their own vacation schedule. It is ok to set vision, but each agent is pursuing their own local goals and they are taking local actions and they all interact.

Mike next contrasts the traditional model of change vs. the complex adaptive model for change.

1. Traditionally, behavior is controllable and predictable. In the complex model we cannot control behavior.

2. Traditionally, we can establish directions. . In the complex model direction is established by the moving groups.

3. Traditionally there is clear cause and effect. In the complex model, there is a ripple effect.

4. Traditionally, relationships are directive. In the complex model, relationships are empowering and can go in both directions.

5. Traditionally, focus is on efficiency. In the complex model we are more interested in responding to change.

6. Traditionally, decisions are based on fact and data. In the complex model, decisions are based on patterns and tensions between groups.

7. Traditionally, leaders are the experts. In the complex model, everybody can help lead. Leaders are there to facilitate and support.

Developing Expertise: Herding Racehorses, Racing Sheep

Dave Thomas gave a talk at QCON London 2007 on how teams can write software better. He starts out by saying that picking a new tool or methodology will not help unless we address the root of the problem. That is, software is not written by tools, frameworks, methodology, or languages. It is written by people. In the 70s, there were 5.5 bugs per 1000 lines of code. In the 90s there were 5.6 bugs per. Even though languages changed (from cobol to c++), methodologies changed (from adhoc to PMBOK), and tools changed (from punch cards, to IDEs), there was no improvements in the number of bugs. So to make the software better, we need to make the people better.

Next, Dave looks at how to make the team better and improve their performance. Dave mentions that teams consist of members with different backgrounds and expertise, yet we like to treat them all the same. They all follow the same procedures even though they work differently and have totally set of needs.

Dave then explains the Dreyfus model for skill acquisition and its stages. The model suggest that most people start out as a novice needing rules and procedures until they gain experience and then work of intuition. Intuition is knowing something without knowing how you know it. Things just pop into your head and you have no idea where they came from. The model defines 5 stages:

1. Novice: You are doing something for the 1st time. You have no idea what you are doing. No interested in learning. You want detail instructions on how to solve the problem and get the job done. You are vulnerable to confusion and don’t know how to respond to mistakes. Novices require short term goals. As you do something over and over again you assimilate it (kind of like driving). It become below the level of consciousness and you get to become an advanced beginner.

2. Advanced beginner: you start trying things on your own, but still need to be told what to do. You have difficulty troubleshooting. You want information fast. You can place some advice in context. You begin formulating some principles but without holistic understanding. You are too busy concentrating on what they are doing. They cannot step back. Things that can help are pairing programming and unit testing.



3. Competent: You have big picture. You can do things for yourself. You seek out expert user advice. You are conscious and look for long terms plans and goals. You can troubleshoot on your own.

4. Proficient: You realize that there is a bigger domain. You are frustrated by oversimplified information. You will self correct previous poor task performance. You will learn from experiences of others. (It is like 3 year old always asking why?)

5. Expert: You are a primary source of knowledge and information. You continually look for better methods. You work primarily from intuition not reason. Forcing to follow set rules degrades performance. You require autonomy. You require the big picture.

Introduction to Agile for Traditional Project Managers

At Agile 2007, Stacia Broderick mapped the PMBOK process groups and knowledge areas to agile practices. She started by defining agile as a value driven approach to building working software by collaborating with customers and embracing change. The characteristics of agile are:

1. Iterative, incremental

2. Risk mitigation via use of timeboxes

3. Continuous improvements

4. Team ownership of quality

5. Value based prioritization of requirements

6. Dedicated self managing teams

7. Collaborating with customer

8. Embracing change

9. Modern engineering practices

Next she went over the Agile manifesto and the principles of:

1. Individuals and interactions over processes and tools

2. Working software over comprehensive documentation

3. Customer collaboration over contract negotiation

4. Responding to change over following a plan

She then mapped the PMBOK process groups of Initiate, Plan Execute, Control, and Close to the agile practices of Envision, Speculate, Explore, Adapt, and Close. She described how the traditional iron triangle of Scope, Time, Cost or Budget has a fixed scope from which a date and cost is derived where as in agile, the triangle is reversed with the date and cost fixed and the scoped varied or derived based on a prioritized product backlog.

Agile Project Management: Lessons Learned at Google

At QCON London 2007, Jeff Sutherland described Google’s Ad Words team’s approach to implementing SCRUM. He started by describing the similarities between the Toyota way and the Google way. At Toyota, the team gets together and 1st produces a working prototype. After examining and learning from the prototype and determining what failed and what worked, another prototype is made. By constant improvement, excellence is built in both the team and process. At Google, management was removed and the developers were empowered to take charge of their own products. They were in direct contact with the users and were able to get continues feedback. They would quickly learn when they failed and be able to move on to the next thing.

Next Jeff describes different social structures:

1. Bureaucracy: Disciplined and coercive – Rigid rules enforcement, extensive rules and regulation, hierarchical controls

2. Autocracy: Whimsical and coercive – Top down control, minimum rules and procedures, hierarchical controls

3. Leadership: Disciplined and empowering – empowered employees, rules and procedures as enabling tools, hierarchy supports organizational learning

4. Organic: whimsical and empowering - empowered employees, minimum rules and procedures, little hierarchy


Then Jeff gives a brief overview of the Ad words product. It has millions of users. It’s in many languages. When released, there needs to be new training. It’s about 500KLOC and growing. There are 5 distributed teams and multiple projects going on with the teams. Because of size and complexity, a group of managers get together regularly. They noticed they did not have enough information to make basic decision.

The ScrumMaster introduced some structure gradually. He asked developers what their biggest problem was. They identified missing dates as a major issue. The ScrumMaster then suggested a release burn down chart to track progress. He knew that because he is doing only 1 of the Scrum practices that there will be many other things not working, but will address the others gradually. He did not implement anything top down. He asked engineers about their problems and then suggested possible ways to solve them. After adopting the burn down chart, the next set of problems was QA not knowing how to test the features, duplication of work, and dependency creation. The ScrumMaster suggested having daily meetings to address these issues. However, the burn down and chart and daily meetings were not enough to meet dates. Velocity was decreasing week by week, but tech leads still insisted that they will deliver on time. The problem was tracking finished tasks vs. finished features. It’s a work in progress problem, so they started keeping # of opened tasks to a minimum.

The 1st retrospective on this partial implementation revealed that the burn down chart helped and that there was more team work and more QA into the process but there were still too many bug, priorities were fuzzy and the team was still blowing dates. The need to clearly prioritize the product backlog was identified along with iterative development. They extended the definition of done to include all development including testing. They added the Sprint burndown. This would show failure a lot sooner.

Gradually all practices of SCRUM were implemented to pass the Nokia test. Jeff defines the Nokia iteration test as having timeboxed iterations of less than 6 weeks with tested and working software at the end of the iteration (unit and functional testing). Also, the iteration must start before the specification is complete.

Then there is the Nokia Scrum test which means you know who the product owner is, the product backlog is prioritized by business value, the estimates are created by the team, they generate the burndown charts and know their velocity, and there are no project managers disrupting their work.

The team kept on operating the same way even after the ScrumMaster went on vacation for 3 weeks.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Agile-Management-Google-Jeff-Sutherland

Heartbeat Retrospectives to Amplify Team Effectiveness

At Qcon London 2007, Boris Gloger described a 6 step process for performing retrospectives:

1. Security – Do not do a retrospective in a stressful environment. Make sure that no one is to blame. Regardless of what we discover, we understand and believe that everyone did the best job they could given they knew at the time, their skills, abilities and resources and the situation at hand.

2. Collect facts - Have a timeline with post-its about experiences of important events. Have the team members tell a short story of every event that is important from their point of view. Concentrate on facts and not emotions.

3. What went well? – Again, post what went well.

4. What could be improved? – Post what could be improved. Environment, skills, resources, hardware. Gather new ideas.

5. Who is in control? – All the ideas are coming from the team. Who is in control of improving what is discussed. Is it the team or the organization?

6. Prioritize – The goal is to improve the next iteration immediately. Have a team backlog that includes things the team must work on besides the product backlog (training, resources, etc.) Have an impediment backlog that includes organizational issues that impedes the team Prioritize and sort them then and add stories to the next sprint.

The retrospective should take between 10 to 90 minutes. It should not be held in the team room, but instead in a dedicated and neutral room that provides a secure environment. Attendees should include the entire team and whomever else the team would like to include.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Heartbeat-Retrospectives-Boris-Gloger

Agile in the Waterfall Enterprise

At Agile 2007, Michelle Sliger discussed some hurdles that agile teams face in a waterfall enterprise and how to clear them:

1. Resistance: There will be resistance from management, the business and the team. Don’t try to sell agile to management. Instead fly under the radar, operate in stealth mode and just go ahead and product working software of high quality. When talking to management about agile, advocate using their language. Avoid using terms like agile, xp, scrum sprints and use terms they are already familiar with like ROI, cutting costs, increasing quality, earlier to market, or share success stories that prove increase in productivity. When dealing with resistance from the business, try to get them involved early on and suggest trying things for 30 days and then have a retrospective. Try to get a client or at least a proxy on site. It is easier to start out with XP technical practices. When facing resistance from the team, try to get the right people on board and empower them.

2. Culture: Review the companies mission statement and see if there are values mismatch between what is stated and what goes on a daily basis. Try to clear up misconceptions of Agile. It’s a new methodology but not a fad.

3. Resource management: What if the organization has a matrix environment. Agile teams are collocated and dedicated to the project. But agile can be distributed, but it won’t be as effective and efficient as collocation. Dedicated is different from devoted. Dedicated means team sticks to the project through completion. Use a release plan or quarterly plan to help resource manager in allocating people. Use release plan and product roadmap to assist project managers in the allocation of the budget.

4. Vendors and Contracting: If a vendor is not agile, evaluate the duration of the relationship. If it is short term, then there is no need to explain agile, but if it is longer, then you should explain it. If customer is not agile, then think about how to present the contract. Start a conversation with them. Would you like us to show you progress on application every 2 weeks? Would you like to change or adjust your requirements after reviewing the progress? Work through time and material contract with option to renew or cancel every 30 days.


5. Facilities and Tooling: Use what works for you. Ask the team and find a solution, create or find an open space, make it a war room, use a whiteboard. Use open source tools.

6. Cost accounting and reporting: How are we going to do the budgeting without a BUFD (Big upfront design document)? You have to do it, but you do it differently. Find out cost per iteration. Find out cost per story points. Find out what metrics need to be tracked. If GANTT chart is still needed then you can give it but duration will always be the same, task assignment will always be ‘TEAM’. Are you capitalizable or an expense?

7. Auditors: talk to auditor and find out what they are looking for. Add those to the product backlog and include them in an iteration to create these docs. Be barely sufficient in your documented proof.

8. Communication is key. Agile is a silver bullet but only if it is implemented properly.. Inspect and adapt and communicate. Have a communication plan in your transition rollout plan. Provide support for sharing knowledge and experience and continued education. Define escalation and issue resolution.

9. Miscellaneous:

• a. passing off code to production (add one iteration to make it production ready – get sign offs, get help desk up to speed, get training materials, get docs).
• b. Reward and recognition: Not about individual performance but about the team making its commitment. How do we do that? Ask the team. As agile team become more and more performing, they will level off. We need to keep them consistently challenged and motivated. Organizational change will happen. Instead of silos, there will be groups around product lines.

Next, Michelle lists 10 tips for an agile/waterfall cooperative:

1. Find an executive champion that will help you by pulling those hurdles out of the way.

2. Socialize, don’t evangelize

3. Use the power of the backlog It should include more than just coding items.

4. Do not wait until everything is figured out and perfect. Just get started. Later you can inspect and adapt

5. Use barley sufficient guideline. Do the simplest thing possible to satisfy a request

6. Invite non agile reps to all agile planning meeting

7. Establish a rhythm of inspection and adaption

8. Start sending the ideas up the chain. Help your managers become more agile

9. Pay attention to your behaviors and your team’s behavior. It is easy to revert back to the old ways especially under stress and pressure.

10. Include everybody in project retrospective. Improve things in the team and across the organization

Finally, Michelle concludes that Agile and waterfall can coexist. You need to build the bridge together. Invite everyone who is interested to help you with the transition to agile.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Agile-in-the-Waterfall-Enterprise-Michele-Sliger

Dealing with Organizational Challenges to Agile Adoption

At Qcon 2007, Joseph Pelrine addressed issues dealing with agile adoption.


He started out by describing how an organization is composed of multiple teams with some being flexible and others rigid. The flexible teams have resonance and will quickly sync up with you. Using resonance can help you bring change. However, too much resonance is a bad thing. It’s like how running down hill too fast can make you fall down. The rigid teams Don’t want to change, do not like what you are doing and do not want to move. They will resist your process. You can use this as a buffer so you do not start tripping up. However, too much resistance is not good.

Next Joseph discusses how management is still stuck in Newtonian concepts and mechanics of work where the whole is the sum of the parts. He describes how we deal with different tasks and issues by providing 4 categories based on cause and effect:

1. Simple: Cause and effect is known. Switching a light switch turns light off and on.

2. Complicated: Cause and effect is hidden but knowable. Pressing a button turns a computer on. Pressing button turns on phone.

3. Complex: Cause and effect is retrospective coherence. Cause and effect emerges and system self organizes, but cause and effect is only clear when looking back and in hindsight.

4. Chaos: No patterns of cause and effect. There is no discernable cause and effect.
Between these is a black hole of stuff we haven’t figure out yet. It will stay there until we know which category it belongs too.


Most tasks that management deal with and most deterministic coding tasks are on the side on simple or complicated (know cause and effect). The hard part are thing on the other side like figuring out what customer need and dealing with people. We’ve been taught how to work with 1st set, but are still floundering around with the other set.

Next, Joseph discusses different ways of thinking

1. Process engineering: There is order. Cause and effect is knowable. We can look it up or it or we can call up an expert. This is great for waterfall. When there is order, you can know everything in advance and because agents obey rules you can predict behavior. This is known as process engineering (Taylor).

Agile Architecture is not Fragile Architecture

James Coplien and Kevlin Henney gave a talk at QCON London 2007 addressing the myth that architecture can’t be agile. It is heavy weight and the magic of Agile makes it unnecessary to bother with up front design.

They define a good architecture as something you can do upfront that is light weight that helps you understand the significance of the decisions that you are going to make. It allows you to look at the system make judgments about what things you can differ and what things you should not differ. To be agile you must be fit. You must be standing on a firm foundation. Agile is about understanding what changes we can ignore. You cannot defer all important decisions. Later you will have a lot in place that you can’t do anything about it. A lot of things have to be pushed upfront. Understand the repercussions of a given decision. See the big picture, make a shared vision across the team and use it as a tentative foundation. Describe the things that really matter. Make the critical design decisions. Architecture is not something that lives in a document. It is not a blueprint. The design document is in the code. Architecture is a shared vision that builds on the knowledge of your people. It is about understanding the thing we need to build and how we will build it. It provides a foundation for team structure, a foundation for the GUI and reflects the system architecture.
According to the Oxford dictionary, the some definitions of agile include quick motion, nimble, active, or ready.

They discuss prioritizing and mention that you do not want everything to become significant. We need to know what to care about to reduce the significance of other decisions. Making hard decisions early makes easy decisions easier. Doing the easy ones 1st constrains you and makes the hard ones even harder.

Then they address big upfront design (BUFD), no upfront design (NUFD), and rough upfront design (RUFD) and suggest that there is tension between the levels of detail and a balance is required. The right amount of design depends on the size of the domain and the recommend 1 or 2 weeks. If using Scrum, then have a single Sprint that delivers the architecture as abstract classes.

They wrap up by illustrating what happens if you start immediately without any design and how if you don’t pay now you will end up paying a lot later. They stress on trying to figure out what is critical and what is not critical and spending time on the critical upfront.

This presentation is available on InfoQ at http://www.infoq.com/presentations/Agile-Architecture-Is-Not-Fragile-Architecture-James-Coplien-Kevlin-Henney

Pragmatic Factors for Agile Success

Venkat Subramanian gave a talk on Pragmatic factors for agile success at NFJS. The talk addressed team member’s attitude towards different aspects of software development. He starts out by comparing the attitude of team members to those of drivers sharing a highway. The ride can be tense and lead to road rage or it can be respectful and pleasant. Venkat discusses these aspects using devils vs. angels’ advice along with how to keep some balance between the 2 opinions.

Work for outcome: The devil suggests the 1st and most important step is addressing a problem is to determine who caused it. Find that person to blame and point at them. The angel suggests being a part of the solution and not the problem. Blame does not fix bugs. Use the problem as an opportunity to learn. To keep a balance, remember that it is everybody’s fault. There will always be problems. Just fix them.

Criticize Ideas, Not People: The devil suggests that you have a lot invested into the design. You have a lot of pride. Stick to your design and don’t listen to anyone else. They just don’t get it. The angel suggests sharing your design with others and asking for help on improving it. Take pride in arriving at a solution rather than proving whose idea is better. To keep a balance always contribute ideas, but know that they will not always be accepted. Be realistic and fair, unemotional but not indifferent or unconcerned.

Keep up with change: The devil suggests that technology changes so fast. It’s overwhelming and you can’t possibly keep up. Just stick to the language you know. The angel suggests that it’s in your best interest to keep up to date. You don’t have to be an expert in everything. Learn iteratively and incrementally by attending local user groups, workshops or conferences, and reading. Spend at least 30 minutes each day learning something new. To keep a balance, ask what’s the value of the new technology? What it does? And what it solves? Don’t convert an application to a new technology just for the sake of learning.



Invest in your Team: The devil suggest that you keep things to yourself. Don’t share what you know. It’s to your advantage to be the smart one on the team. The angels suggest having regular brown bag sessions to raise the awareness of the team. Each member has different expertise and strengths and it’s to your benefit to be in a mature and qualified team. To keep a balance don’t turn the meeting into a design session. Pick a book and stretch beyond technical books.

Feel the rhythm: The devil suggests having a code review once to review the entire code base. The angel suggests not letting tasks add up and tackling them regularly. Have constant code reviews (pair programming) or small task code review. Constantly evolve the code and maintain using TDD. It’s like walking, left foot code, right foot test. Walk while keeping a balance. The key is to set small reachable goals, celebrate your success.

The Agile Enterprise: Real World Experience in Creating Agile Companies

Jeff Sutherland discussed some real world scrum successes at Agile 2007. He starts by defining some of the characteristics of real scum as companies having agile as a strategic imperative, institutionalizing scrum and xp and not only in development, passing the Nokia test, and having senior management and developers totally involved.

He defines the Nokia iteration test as having timeboxed iterations of less than 6 weeks with tested and working software at the end of the iteration (unit and functional testing). Also, the iteration must start before the specification is complete.

Then there is the Nokia Scrum test which mean you know who the product owner is, the product backlog is prioritized by business value, the estimates are created by the team, they generate the burndown charts and know their velocity, and there are no project managers disrupting their work.

Next, Jeff gives several real world examples of companies he consulted with that are doing real scrum and are producing incredible results.


One company only hired an experienced scrum product owner and managed to cut their product cycle from 18 months to 2 months.

Another was too hyper productive that management ask developers to slow down. Sales could not keep up with the products. They needed a way to get the whole company agile.

Another company was distributed with 567 developers in many locations:

Jeff discussed outsourcing and gives an example of a 2 million project that would cost around 1.6M if outsourced (usually outsourcing save about 20%). Instead, the project was implemented with a local scrum that increased productivity by 240% (can probably even get up to 400%). The local cost was 0.83million without the risk associated with outsourcing such as inability to stadd the project on time, inability to keep staff on project once it starts (turn over running between 35% to 50%), and poor communication due to distance, time zones, lack of formal process, and language barrier.

Jeff gave another example of a company that created a team in Russia doubling the development team size and as a result doubled productivity. The company used distributed agile with teams split between US and Russia. At the end of the project, the company cut the Russian team, but was able to retain knowledge because the US team members remained. This was an example of a distributed, outsourced hyper productive team that resulted in linear scale when using Scrum. By doubling the work force, productivity doubled. On the other hand, in the waterfall model, it is well known that increasing team size when project is late will decrease productivity and make the project even later.

Context Driven Agile Leadership- Managing Complexity and Uncertainty

Todd Little gave a talk at Agile 2006 about context driven agile leadership. He starts out by comparing agile planning to hurricane prediction. Predictions are made within a cone of uncertainty. It is a forward looking view towards uncertainty. We know generally where we are going, but we are not going to a point, we are steering towards a general direction. As we move forward we adapt and steer towards the market.

Todd mentions that his work was influenced by Jim Highsmith’s adaptive software development, Alistair Cockburn’s crystal methods, and Boehm and Turner’s Balancing Agility and discipline.

Todd briefly discusses the Crystal Method matrix covering number of people involved (1 – 1000) vs. Criticality (Comfort, Discretionary Money, Essential Money, Life). A crystal method (clear, yellow, orange, red) is selected based on project size, system criticality, team priorities (productivity, tolerance, legal liability). Todd notes the crystal clear does not scale into larger teams and does not go upwards towards essential money.

He then discusses how Boehm and Turner took crystal’s 2 dimensions of criticality and number of people involved and made 5 dimensions by adding personnel experience, dynamism (# of requirement changing per month or uncertainty), and culture (preference to structure and order). They observed that neither agile nor planned provide a silver bullet. They each have home grounds where each clearly dominates. Future development will need both agility and discipline and some balanced methods are emerging. It is better to build your method up then to tailor it down and methods are important, but potential silver bullets are more likely to be found in areas dealing with people, values, communications, and expectation management



Todd takes a closer look at the right side of the Agile Manifesto (tools, documentation, contracts, plans) and mentions that the tools should enhance collaboration environment like wikis, documentation should be a consumable rather than a deliverable, contracts should be written in a manner that are consistent with collaboration and agile delivery, and plans should expect and anticipate change.

Todd explains how he evaluated projects based on
1. Complexity: team size, mission criticality, team location, team capacity (novice vs. expert), domain knowledge, dependencies

2. Uncertainty: Market Uncertainty (known vs. new and untested), Technical Uncertainty (enhancement using existing technology vs. new architecture and new technology), Project Duration, # of people dependent on us