The 5 Whys

The 5 whys is a retrospective technique that can be used when the team needs to focus on a specific symptom that does not have an obvious solution. The goal of the 5 whys is to clearly understand the situation and not necessarily to solve the problem.

The process is as follows:

1. Write the problem statement clearly and concisely on the board.
2.  Divide the team into groups of at least 3 but not more than 5. 
3.  In each group, individually answer why did this happen? Number your answer as #1.
4.  Individually answer why did #1 happen? Number your answer as #2.
5.  Individually answer why did #2 happen? And so on until you have around 5 answers or can’t come up with more meaningful answers (try to have at least 3 answers).
6.  Each individual places their chain of responses in a column below the problem statement. The group then discusses the individual answers and tries to find common patterns.
7.  The group consolidates the chains into a single chain.
8.  If necessary, the team consolidates the chains into a single chain.

Participants should focus on exploring the immediate cause of the symptom rather than attempting to immediately jump to a root cause. The intermediate reasons often spark ideas within the group that would otherwise be overlooked.

While this exercise is typically applied against problems, this same technique can also be used to discover what led a team to achieve exceptional success in one area.

Adapted from: Putting the fun back into your retrospectives @ Agile2011

Follow-up on Getting Fit Using Agile Workstations

Back in late December, I blogged about my New Year's resolution of using a standing workstation. Well 6 months later, I can say that I have stuck to it. I can now work standing for about 6 hours a day, with a break during lunch. Overall, I definitely feel better and more energized, but I do not have concrete metrics to show that my health is improving. There are however more studies that have come out on the subject. Some warn of the dangers of sitting, yet other warn of the dangers of standing. Go figure!

http://healthland.time.com/2011/04/13/the-dangers-of-sitting-at-work%E2%80%94and-standing/

http://www.cbsnews.com/stories/2010/07/16/health/main6683528.shtml

http://www.msnbc.msn.com/id/34956099/ns/health-fitness/t/you-sitting-down-experts-say-itll-kill-you/

And here's an info-graphic from medical billing and coding that sums it all up:




Via: Medical Billing And Coding

Importance of Well Designed Software

At the 2011 Agile Development Practices West conference, Martin Folwer addressed trade-able quality.

Martin explains that quality is something we can trade-off for other things. We often hear managers saying “We need to put less effort on quality so we can build more features for our next release.” We also do this every day. We do it when we buy a car. Sure we want the fancy car, but we go for the cheaper one and spend the extra savings on something else. This is a natural part of how we think about the things that we buy every day.

In software, there are visible and invisible quality features. For example, with a UI, you can feel the difference. But internal quality is not visible to the users. As a buyer, if I can’t see it, why do I want it? In that case, I will always pick the cheaper option.

Next Martin introduces the Design Stamina Hypothesis to illustrate why we should care about internal design of software.

Figure 1 - Stamina Hypothesis

The graph shows functionality over time.

The blue line shows bad design. We start out strong, but fade and slow down over time.
The red line shows good design. We start out strong and over time, we can keep a steady pace and even speed up.

With the clean system, a year from now it will have another 5 new features. The bad design system will probably only have 1 or 2 new features as they struggle with their bad code base.

Next Martin introduces the term technical debt which was originally coined by Ward Cunningham. Complexity that we don’t need causes extra effort to add new features. The unnecessary complexity in your code base is like the principal of a debt and the extra cost that you pay for adding new features is like the interest on the debt. This metaphor is useful when talking with non technical people. When building new software, you can choose a quick and dirty approach, increase your debt and increase your interest payment. Or you can pay down the principal and reduce interest payments.

Martin then explains the reasons we get into technical debt.

Figure 2 - Technical Debt Quadrant

1. Prudent/Deliberate: “We must ship now and deal with consequences” – This is a conscious decision. I have a deadline and I’m prepared to tradeoff some quality to meet that deadline.

2. Inadvertent/Reckless: People have no clue about software design.

3. Deliberate/Reckless: Very close to prudent/deliberate however, it is done without understanding what is going on. The decision to trade-off quality (debt) for speed only makes sense when we are below the design pay-off line (where the 2 red and green lines cross in figure 1). The key is to understand how far away the design pay-off line is. It is probably in the order of weeks and not in the order of months. It is a lot shorter than what people tend to think it is. If I am close to the point where I go over the design pay-off line, then there is no point in sacrificing design for speed because them I will end up losing both.

4. Prudent/Inadvertent – Here the financial metaphor breaks down. This case is when we’ve taken on debt without noticing it. It is because we learn new things as we code and when we look back at what we coded we say “Now we know how we should have done it”.

This presentation is available on youtube at http://www.youtube.com/watch?v=p5Qj75nJPEs (starts at minute 29:05)

Martin also discusses technical debt on his blog http://www.martinfowler.com/bliki/DesignStaminaHypothesis.html and http://www.martinfowler.com/bliki/TechnicalDebtQuadrant.html

Command and Control vs. Agile

In her book, Coaching Agile Teams, Lyssa Adkins nicely maps out the transition in beliefs that traditional project managers have to go through when adopting agile.


Lyssa explains that the command and control project management beliefs on the left need to be replaced with the agile beliefs on the right:

Project management belief	Replaced with
We can plan the work and work the plans.	Planning is essential, plans are useless.
The triple constraints can be traded off for one another to correct for unknowns.	Time and budget are held constant. Only scope flexes.
The plan gets more accurate over time as we flush out the project through phases of activity: requirements, design, testing, and so on.	The plan gets more accurate over time because it is constantly reviewed and tuned up based on the teams actual performance.
Delivering on time, within budget and on scope equals success.	Clients getting the business value they need is the only value of success.
Scope can be locked down with later discoveries being handled as change requests against the schedule end date.	Scope remains flexible and changes of any kind are welcomed even late in the schedule.
Controlling through the project plan is my job.	Controlling through a plan is not possible; releasing the team into the safety of agile is my only measure of control. So I coach the team to use agile well.
Completing tasks and delivering deliverables indicate progress and value delivered.	Only delivered end products indicate progress and value delivered.

The Agile Manifesto – 10 Years Later

At the 2011 Agile Development Practices West, Martin Fowler reflected on where we are in the agile movement 10 years after the creation of the Agile manifesto. Martin looked at what was going on back in 2000. We were in a world of badly managed software. A group of people felt that to clean up this mess we needed big methodology, planned driven methodology, or an engineering approach to software development with lots of documents and lots of processes (waterfall).

Others had success with a different approach. They believed in rapid iteration, lots of collaboration, and an evolutionary approach to requirements, design, and architecture. They felt that they were not getting heard and were getting trampled by big methodology approach.

This group got together in Snowbird 10 years ago. The idea for the meeting originated a year earlier when the group got together at a workshop at Kent Beck’s home. Kent described XP as a set of practices governed by a set of principles and values. However, there were some that liked the values or principles, but did not like the practices.

This led to the Snowbird meeting where they concentrated on the values and discussed the different approaches like XP, SCRUM and Feature Driven Development. Uncle Bob felt the group needed a manifesto as a statement of what they believed in. In the end, the people that wrote the manifesto were just the people that were free to meet up that week.

Now 10 years later, a lot of the people that where there at the creation feel a sense of unhappiness with Agile and the manifesto.

Martin believes there are 2 main reasons for this.

The 1st is due to the inevitable consequence of success. The idea of Agile spreads out but the true meaning gets diffused. He refers to this as semantic diffusion. Anything that is successful spreads faster than the semantics can follow it. Martin stresses that we have to keep reminding people about the core concepts.

The 2nd reason is that the Agile revolution is still in its infancy. We are in the early days of a very long running change. Martin remembers the OO revolution. It started in the late 60s, it was put together in the 70s, had a good definition of what OO is in the 80s with small talk and took another 20 years or so before OO became main stream in languages like java, C++ and C#. Still today, 40 years on, people think they are doing OO, but they really just have procedures and naked data structures and not true OO.

Martin thinks Agile will take even longer than OO because it affects more people. It’s not just developers, but it is testers, BAs, managers, etc…

Martin then argues that people should still care about agile. He remembers how the manifesto was written.

The group came up with 8 valuable useful good things in software development. The idea was to arrange them in pairs so that one useful thing was more valuable than the other. The guiding light was the drive to the engineering methodologies. Flipping the values around was the value system of the engineering methodologists.

A lot of companies care about processes and tools over individuals because they want to be able to move people around.


The design and documentation is important to them because they believe that once it is produced they can give it to a bunch of developers that can code it up.

You have to have contracts in place and decided. The conversations are based on what is in the contract.

Following a plan was important because the definition of success was based on whether they stuck to the plan.

Martin argues that if you don’t care about Agile then you can be equally happy in a flipped or agile environment. The order within each pair of values does not matter.

Martin wraps up by looking to the future. He sees 2 emerging areas for Agile. The 1st is Agile User interface design where the UI evolves through customer, designer and developer collaboration instead of handing the UI to a team and then asking the developers to code what was designed (even if the developers see much simpler and faster solutions).

The second is the concept of continuous delivery and improving the process of how we move code from development to test to production.

This presentation is available on youtube at http://www.youtube.com/watch?v=p5Qj75nJPEs (minute 50)

The Semmelwies Reflex

At the 2011 Agile Development Practices West conference, Linda Rising gave a keynote address entitled “Deception and Estimating, how we fool ourselves”. In the presentation, Linda referred to the The Semmelwies Reflex which is a metaphor for the reflex-like rejection of new knowledge because it contradicts entrenched norms, beliefs or paradigms. It is based on the theory that once scientists have crafted a hypothesis they are reluctant to let go.

Linda quotes Max Plank:

“A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it”.

Linda then explains that we have to wait for the old ideas to die, physically die, and she amusingly wonders “How many waterfall people must pass on before Agile takes over!!!”

Non-Determinism and Testing

At the keynote address of the 2011 Agile Development Practices West, Martin Fowler gave a talk on non- deterministic tests. These are tests that when you run them they pass, but if you run them again (without changing any code), they fail. They are also referred to as flaky tests or intermittent failures. The tests seem to pass or fail in a random fashion.

Martin asserts that non-deterministic tests are useless. The whole point of a regression suite is to be a bug detection mechanism. Its value is that you get immediate feedback when you make a mistake. The time between making a mistake and realizing that you did is very short. And because it is short, you can quickly figure out what you did wrong and fix it. With non-deterministic tests you get unreliable information.

Martin argues that non-deterministic tests are worse that useless. They are very dangerous. They are like an infection that infects the entire suite. One failing test will mark the entire suite as red. Usually we will dig in to see which test failed and fix it, but once we start seeing red regularly, we will assume the failure is due to the non-deterministic test and ignore the failure of the entire suite. This makes the entire suite flaky and the entire test suite becomes useless.

Martin purposes setting up a quarantine area for flaky tests. If we pull the tests from the suite, then the suite will remain reliable. The number of tests in quarantine need to be limited and fixed as soon as possible. Martin recommends setting some limits like no more than 3 tests can be in quarantine or a test in quarantine should be fixed within a week.

Martin next describes some causes for non-deterministic tests.

1. Lack of isolation: Tests depend on the order that are run. These are very hard to debug because when a test fails because of the order it ran in the suite, you usually re-run in isolation and then it passes. The real cause of the problem is in another test which is green and very hard to locate. There are two possible solutions:

• Track dependencies: Track the order in which the tests need to run. This is very hard to manage and maintain.
• Isolation: Martin prefers this solution. You can apply a clean-up strategy. That is, make sure there is a tear down that destroys anything you have created. An even better approach is to use a blank slate. Always start with a clean data set. The disadvantage here is that setting up the blank slate can be time consuming.

2. Asynchrony: Testing asynchrony is difficult. A common approach is to use what Martin calls a bare sleep where we introduce a sleep statement.

//pseudo-code
someFunction
sleep(aWhile)
assert(answer)

If we sleep too long, the test will run very slowly. If we reduce the sleep, then when we move to a different machine or if the machine is overloaded, then the tests will fail, so coming up with the correct number to sleep is very hard. Martin warns us against using this approach and recommends another solution using polling:

//pseudo-code
someFunction
timeLimit=Time.now + timeout
while(!answer.recieved) {
  if (Time.now > waitLimit) {
    throw new TestTimeoutException
  }
  sleep (pollingInterval)
}
assert(answer)

Here we have the sleep inside the polling and a timeout interval. You can keep the sleep interval very low and adjust the timeout. You have two times to play with (global constants).

Martin also discusses another approach using callbacks

//pseudo-code
someFunction
callback = function
{ assert(answer) }
answer.ready(callback)
asserts.add(callback, timeout)

The callback method itself contains the verification. This solution depends on the testing framework that you are using.


3. Interaction with remote services: There are many things that can go wrong when dealing with a remote service that have nothing to do with your code (network down or slow, limited availability, unstable test data).

Martin recommends using a a test double. This way, you control the data the service has access to, you can control changes, and ensure fast connections. However, some don’t believe this is testing the real service because we cannot keep the double consistent with the remote service. To solve this problem, Martin recommends that we add an integration contract test that does not run as part of the main build. This test will probably run nightly against the actual service and ensures that the signature of the double is in synch with the actual service.

Martin concludes by warning us that flaky tests are a common problem that can become very dangerous if not addressed immediately.

This presentation is available on youtube at http://www.youtube.com/watch?v=p5Qj75nJPEs

Martin also discusses non-deterministic tests on his blog http://martinfowler.com/articles/nonDeterminism.html