The Kata in the Hat

At Agile 2011, Emmanuel Gaillot and Jonathan Perret performed a coding Dojo entitled "The Kata in The Hat." Emmanuel and Jonathan are behind the original coding dojos in Paris. Usually, katas are performed by practicing TDD and applying baby steps. The main kinds of Katas include:

1. Prepared Kata  where someone comes prepared with a full solution to the Kata and presents it to the group.
2. Randori Kata where the group rotates pairs so solve a particular problem.

The Kata in The Hat is a twist on these katas. Here, the hosts had a hat in which audience members can write-in rules and requests and drop them in the hat. The presenters start with a main idea, but every five minutes one paper is pulled out of the hat, the instructions are read and the timer is reset. The hosts continue to pair program, but now they incorporate the new requests into their code. And so on... The interesting part is seeing how they can finish each request by doing the simplest thing that can possibly work. They quickly adjust and keep the code moving as they try to solve the main problem no matter what is thrown at them. This type of Kata is definitely not for the beginners, but it’s a great show that is fun, entertaining and educational. If they are ever touring at a conference near you, I highly recommend you attend.

"The big idea we want to get across is that programming can be beautiful, and that through creativity and beauty and poetry programmers have much more power than they think."
 http://thekatainthehat.com/ 

Source Code Word Clouds

Here’s a neat way to look at code quality - use source code word cloud generators.
A word cloud or tag cloud is a visual representation for text data. Tags or words are listed and the importance of each word is shown with different font size or color. When applied to well written and clean code, word clouds give us a quick way of understanding what the code is about.
 
 Below is an example from http://fragmental.tw/2009/04/29/tag-clouds-see-how-noisy-your-code-is

Clearly, looking at the second cloud, we know a lot more about the domain than the 1st cloud.

Try it out using the original http://www.wordle.net or http://sourcecodecloud.codeplex.com

Adapted from "Code" @ Agile2011

Measuring Code Quality

A common measure for code quality is to measure Cyclomatic Complexity. Cyclomatic Complexity is used to measure the number of linearly independent paths through a program’s source code. The more conditionals (if statements) or the more loops (for, while …), then the more branches run through the code and thus the code becomes more complex. But what is considered to be a good number for Cyclomatic Complexity? What should developers target?


A study by energy (http://www.enerjy.com/blog/?p=198) reveals the following:

There is a 50% chance of errors with a Cyclomatic Complexity of 38 and almost a 100% chance of error when Cyclomatic Complexity is 74. On the other hand, with a Cyclomatic Complexity of 11, the chances of errors drops to 28%.

So based on this study and the above graph, shoot for a Cyclomatic Complexity of below 20 per class.

Adapted from "Technical Debt Assessment and Reduction" @ Agile2011

Importance of Well Designed Software

At the 2011 Agile Development Practices West conference, Martin Folwer addressed trade-able quality.

Martin explains that quality is something we can trade-off for other things. We often hear managers saying “We need to put less effort on quality so we can build more features for our next release.” We also do this every day. We do it when we buy a car. Sure we want the fancy car, but we go for the cheaper one and spend the extra savings on something else. This is a natural part of how we think about the things that we buy every day.

In software, there are visible and invisible quality features. For example, with a UI, you can feel the difference. But internal quality is not visible to the users. As a buyer, if I can’t see it, why do I want it? In that case, I will always pick the cheaper option.

Next Martin introduces the Design Stamina Hypothesis to illustrate why we should care about internal design of software.

Figure 1 - Stamina Hypothesis

The graph shows functionality over time.

The blue line shows bad design. We start out strong, but fade and slow down over time.
The red line shows good design. We start out strong and over time, we can keep a steady pace and even speed up.

With the clean system, a year from now it will have another 5 new features. The bad design system will probably only have 1 or 2 new features as they struggle with their bad code base.

Next Martin introduces the term technical debt which was originally coined by Ward Cunningham. Complexity that we don’t need causes extra effort to add new features. The unnecessary complexity in your code base is like the principal of a debt and the extra cost that you pay for adding new features is like the interest on the debt. This metaphor is useful when talking with non technical people. When building new software, you can choose a quick and dirty approach, increase your debt and increase your interest payment. Or you can pay down the principal and reduce interest payments.

Martin then explains the reasons we get into technical debt.

Figure 2 - Technical Debt Quadrant

1. Prudent/Deliberate: “We must ship now and deal with consequences” – This is a conscious decision. I have a deadline and I’m prepared to tradeoff some quality to meet that deadline.

2. Inadvertent/Reckless: People have no clue about software design.

3. Deliberate/Reckless: Very close to prudent/deliberate however, it is done without understanding what is going on. The decision to trade-off quality (debt) for speed only makes sense when we are below the design pay-off line (where the 2 red and green lines cross in figure 1). The key is to understand how far away the design pay-off line is. It is probably in the order of weeks and not in the order of months. It is a lot shorter than what people tend to think it is. If I am close to the point where I go over the design pay-off line, then there is no point in sacrificing design for speed because them I will end up losing both.

4. Prudent/Inadvertent – Here the financial metaphor breaks down. This case is when we’ve taken on debt without noticing it. It is because we learn new things as we code and when we look back at what we coded we say “Now we know how we should have done it”.

This presentation is available on youtube at http://www.youtube.com/watch?v=p5Qj75nJPEs (starts at minute 29:05)

Martin also discusses technical debt on his blog http://www.martinfowler.com/bliki/DesignStaminaHypothesis.html and http://www.martinfowler.com/bliki/TechnicalDebtQuadrant.html

Non-Determinism and Testing

At the keynote address of the 2011 Agile Development Practices West, Martin Fowler gave a talk on non- deterministic tests. These are tests that when you run them they pass, but if you run them again (without changing any code), they fail. They are also referred to as flaky tests or intermittent failures. The tests seem to pass or fail in a random fashion.

Martin asserts that non-deterministic tests are useless. The whole point of a regression suite is to be a bug detection mechanism. Its value is that you get immediate feedback when you make a mistake. The time between making a mistake and realizing that you did is very short. And because it is short, you can quickly figure out what you did wrong and fix it. With non-deterministic tests you get unreliable information.

Martin argues that non-deterministic tests are worse that useless. They are very dangerous. They are like an infection that infects the entire suite. One failing test will mark the entire suite as red. Usually we will dig in to see which test failed and fix it, but once we start seeing red regularly, we will assume the failure is due to the non-deterministic test and ignore the failure of the entire suite. This makes the entire suite flaky and the entire test suite becomes useless.

Martin purposes setting up a quarantine area for flaky tests. If we pull the tests from the suite, then the suite will remain reliable. The number of tests in quarantine need to be limited and fixed as soon as possible. Martin recommends setting some limits like no more than 3 tests can be in quarantine or a test in quarantine should be fixed within a week.

Martin next describes some causes for non-deterministic tests.

1. Lack of isolation: Tests depend on the order that are run. These are very hard to debug because when a test fails because of the order it ran in the suite, you usually re-run in isolation and then it passes. The real cause of the problem is in another test which is green and very hard to locate. There are two possible solutions:

• Track dependencies: Track the order in which the tests need to run. This is very hard to manage and maintain.
• Isolation: Martin prefers this solution. You can apply a clean-up strategy. That is, make sure there is a tear down that destroys anything you have created. An even better approach is to use a blank slate. Always start with a clean data set. The disadvantage here is that setting up the blank slate can be time consuming.

2. Asynchrony: Testing asynchrony is difficult. A common approach is to use what Martin calls a bare sleep where we introduce a sleep statement.

//pseudo-code
someFunction
sleep(aWhile)
assert(answer)

If we sleep too long, the test will run very slowly. If we reduce the sleep, then when we move to a different machine or if the machine is overloaded, then the tests will fail, so coming up with the correct number to sleep is very hard. Martin warns us against using this approach and recommends another solution using polling:

//pseudo-code
someFunction
timeLimit=Time.now + timeout
while(!answer.recieved) {
  if (Time.now > waitLimit) {
    throw new TestTimeoutException
  }
  sleep (pollingInterval)
}
assert(answer)

Here we have the sleep inside the polling and a timeout interval. You can keep the sleep interval very low and adjust the timeout. You have two times to play with (global constants).

Martin also discusses another approach using callbacks

//pseudo-code
someFunction
callback = function
{ assert(answer) }
answer.ready(callback)
asserts.add(callback, timeout)

The callback method itself contains the verification. This solution depends on the testing framework that you are using.


3. Interaction with remote services: There are many things that can go wrong when dealing with a remote service that have nothing to do with your code (network down or slow, limited availability, unstable test data).

Martin recommends using a a test double. This way, you control the data the service has access to, you can control changes, and ensure fast connections. However, some don’t believe this is testing the real service because we cannot keep the double consistent with the remote service. To solve this problem, Martin recommends that we add an integration contract test that does not run as part of the main build. This test will probably run nightly against the actual service and ensures that the signature of the double is in synch with the actual service.

Martin concludes by warning us that flaky tests are a common problem that can become very dangerous if not addressed immediately.

This presentation is available on youtube at http://www.youtube.com/watch?v=p5Qj75nJPEs

Martin also discusses non-deterministic tests on his blog http://martinfowler.com/articles/nonDeterminism.html

Software Craftsmanship - It's an Imperative

Thanks to all that attended my presentation on Software Craftsmanship.

Below are the slides and notes from the presentation.