The Code Christmas Tree

The Code Christmas Tree is a technique used to visualize the quality of your code. It is based on a treemap which is an information visualization technique to display hierarchical or tree-structured data as a set of nested rectangles. Branches are represented by rectangles and sub branches are represented by smaller rectangles and so on. The leaf node’s rectangle has an area and color proportional to a specified dimension of data. In the case of the Code Christmas Tree these can be lines of code, cyclomatic complexity, and test code coverage. The different colors and sizes make it easy to see patterns that would be difficult to spot in other ways. Looking at the tree you can quickly identify which code areas are clean and which ones need more tests or major refactoring.

The chart examples shown here were generated by Michael Kaiser and Guy Royse using a utility that displays Big Visible Metrics (BVM). It takes XML metrics generated by Sonar and parses the data and converts it to CSV data that is then consumable using the Microsoft Treemapper and Excell Add-in to generate these colorful charts. The size of the rectangle represents lines of code. The color represents cyclomatic complexity or test coverage. The large red rectangles are the code targets that need to be refactored or need increased code coverage. BVM is on GitHub and you can find out more about it here.

Adapted from The Code Christmas Tree @Agile2011

Project Vital Signs

At Agile 2010, Stelios Pantazopoulos gave a presentation on project metrics entitled “Project Vital Signs”. Stelios starts by pointing out that IT has lost trust and credibility with the business due to projects having a poor track record of success, poor visibility and ROI that is hard to quantify. Projects are over budget, late, or fail all together. To restore trust and credibility, we need to show how and why a project is on track throughout the project’s lifecycle.
Stelios, suggests applying the metaphor of medical care and their use of “vital signs” to help form a holistic view of the state of the project. In medicine, doctors go over the patient’s medical history, review lab test results, look at vital signs, and then use their experience to diagnose the condition and recommend a treatment. Stelios defines vital signs as simple, quantitative, near real time metrics conveyed via a chart published to a location and easily referenced by all. There are 5 vital signs that need to be monitored. 4 are based on PMBOKs Scope, Quality, Schedule, and Budget. In addition, Stelios feels that monitoring the team’s overall health is also very important.
The 5 charts are:

1. Scope Burn Up: Backlog burn up chart (stories in each state vs. time) that tracks schedule and scope to show expected delivery date. Number of stories or story points are be tracked. Stelios mentions that if you have a big enough back log, tracking number of stories worked well for him.
2. Current State of Delivery: Backlog Scrum board that tracks scope and team and shows real time state of delivery. Tracks the state of each story and who is working on which story.
3. Budget Burn Down: Burn down chart (total $ vs. time) that tracks schedule and budget to show remaining dollars.
4. Delivery Quality: Dot Chart (bug category vs. time) that tracks schedule and bugs to show overall quality. Bugs are categorized into 4 categories:
1. High priority/High severity
2. High priority/Low severity
3. Low priority/High severity
4. Low priority/Low severity 
At the end of each iteration, each bug is represented as a dot in the corresponding category. Ideally, as the schedule nears release date, most bugs should fall into the low priority categories. Stelios also mentions that alternatively one can track automated test results as opposed to bugs.
5. Team Dynamics: Chart (state vs. time) that tracks the state of the team and schedule to show the state of team at different stages of delivery. The state is collected in the retrospective by asking team member to have a secret vote on where each feels they are on the Tuckman model of group development (forming, storming, norming, performing). Similar to the delivery quality chart, each team member’s opinion is represented as a dot in the corresponding category.

The 5 project vital signs are combined to create a project dashboard that is placed on the wall for high visibility. Stelios wraps up by going through different sample scenarios and demonstrates how the dashboard provides a holistic view of the project. He then diagnosis the project and performs what if scenarios for different treatments showing the state of the project before and after treatment.

Stelios provided the slide deck for the presentation at http://projectvitalsigns.com/

Agile Project Metrics

Dave Nicolette presented Agile Project Metrics at Agile Conference 2009. He explained that there are 3 levels of maturity in agile teams:

1. Six week iterations. Stories are divided into tasks. Estimating is done in ideal time. Burn down chart is updated daily.

2. Two week iteration. Stories are divided but estimates are made in story points.

3. One week iteration. Stories are kept small. No daily burn down chart.

Also, teams are composed of generalized specialists where everyone can contribute in different areas, Tech lead/Chief where junior members are combined with a tech lead to help out in different situations, specialists with internal handoffs from one member to the other.

Collect metrics for self improvements and discontinue once goal is achieved. To the customer, working software is the most important measure of progress. Things to measure are:

1. Running test features

2. Hard financial values (benefit of using software after every release)

3. Earned Business Value (have customer value features)

4. Velocity

5. Static Code Analysis (statements per method, LOC covered by tests, cyclomatic complexity

6. Earned Value Management

The talk concludes by giving an example of a sample scorecard divided into 4 quadrants:

1. Value delivered: Earned Business Value, Running tested features, Burn down charts.

2. Delivery effectiveness: Burn down with team focus, story cycle time.

3. Software Quality: Customer satisfaction, non functional requirements, testing metrics, static code analysis, observations.

4. Continuous Improvement: Build frequency, escaped defects, use of TDD, refactoring, overtime, issues from retrospective

This presentation is available on InfoQ at http://www.infoq.com/presentations/agile-project-metrics

Metrics in an Agile World

James Shore and Rob Myers presented a talk on “Metrics in An Agile World” as part of the Agile 2009 conference. The talk starts out by covering informational vs. motivational metrics. Motivational metrics are extrinsic motivators (such as bonus, pay increase) and extrinsic motivators lead to poor performance when compared to intrinsic motivators (interesting work, appreciation, recognition, and good environment). Several examples are given as proof that extrinsic motivators lead to poor performance.

1. Children were given magic markers and told to draw. One group was given a certificate when they finished while another group was not rewarded. The study showed that the group that got the certificate lost interest in drawing, while the other group kept on doing it because it was fun.

2. Two groups were asked to identify matching patterns. One group was paid for it while the other was not. The group that was paid for it made more mistakes then the other group.

3. Two groups were given a puzzle-like problem to solve in 2 steps with a break in between. During the break, the group that was getting paid simply waited around, while the other group kept on discussing and trying to solve the puzzle.

When performance is based on reward, others areas which are not being rewarded will suffer because everybody is trying to maximize the reward and will pay less attention to other items. As an example, if the number of features is being rewarded then bugs will increase as developers work quickly to add new features and worry less on testing. Because we cannot measure everything, there will always be an area that suffers be it less features, more bugs, increased technical debt, padded estimates, etc…

They then talked about creating a framework for metrics covering several categories such as Quality, Value, Progress, Team Performance, Code Design. Within each category, each metric is identified as Qualitative or Quantitative. A few examples of metrics are customer satisfaction, user adoption, billable hours, function points, sloc, spog, cycle time, # of defects, etc. Each is then categorized and analyzed with the goal being to avoid motivational metrics and collect informational ones. One technique discussed involves making the data anonymous and to measure up between groups of peers as opposed to on an individual basis. This will change a metric from motivational to informational.

This presentation is available on InfoQ at http://www.infoq.com/presentations/agile-metrics