James Shore and Rob Myers presented a talk on “Metrics in An Agile World” as part of the Agile 2009 conference. The talk starts out by covering informational vs. motivational metrics. Motivational metrics are extrinsic motivators (such as bonus, pay increase) and extrinsic motivators lead to poor performance when compared to intrinsic motivators (interesting work, appreciation, recognition, and good environment). Several examples are given as proof that extrinsic motivators lead to poor performance.1. Children were given magic markers and told to draw. One group was given a certificate when they finished while another group was not rewarded. The study showed that the group that got the certificate lost interest in drawing, while the other group kept on doing it because it was fun.
2. Two groups were asked to identify matching patterns. One group was paid for it while the other was not. The group that was paid for it made more mistakes then the other group.
3. Two groups were given a puzzle-like problem to solve in 2 steps with a break in between. During the break, the group that was getting paid simply waited around, while the other group kept on discussing and trying to solve the puzzle.
When performance is based on reward, others areas which are not being rewarded will suffer because everybody is trying to maximize the reward and will pay less attention to other items. As an example, if the number of features is being rewarded then bugs will increase as developers work quickly to add new features and worry less on testing. Because we cannot measure everything, there will always be an area that suffers be it less features, more bugs, increased technical debt, padded estimates, etc…
They then talked about creating a framework for metrics covering several categories such as Quality, Value, Progress, Team Performance, Code Design. Within each category, each metric is identified as Qualitative or Quantitative. A few examples of metrics are customer satisfaction, user adoption, billable hours, function points, sloc, spog, cycle time, # of defects, etc. Each is then categorized and analyzed with the goal being to avoid motivational metrics and collect informational ones. One technique discussed involves making the data anonymous and to measure up between groups of peers as opposed to on an individual basis. This will change a metric from motivational to informational.
This presentation is available on InfoQ at http://www.infoq.com/presentations/agile-metrics
