22 CSFs for a successful metrics programmes

Some preliminary thoughts about what will help make a success of a metrics programme.

Most are not about metrics at all, though – which should not be too surprising, given that the main problems with metrics programmes are much the same as for any other management programme:

  1. Work out what you are measuring for.
    • The right answer is never ‘to calculate the value of x‘! It should always be some sort of outcome.
    • More precisely, make sure that every metric is tied to a goal you are trying to achieve.
    • More precisely still, make sure that every measurement is explicitly tied to a key performance indicator that is explicitly tied to a critical success factor that is explicitly tied to a goal you are trying to achieve.
  2. Understand your audience.
    • Metrics are expensive. With a robust model of your audiences and what they need to know, you should be able to please different audiences with variations on the same measurements.
    • Always remember that the same numbers can mean very different things from different perspectives.
  3. Make sure that you have the sponsorship needed to force/enforce action.
    • All metrics should leave to improvement, even if it is only eliminating problems.
    • If the person with the relevant authority doesn’t want it enough to make it happen, it won’t happen.
    • So measurement is a waste of time and effort.
  4. Measurement is not the first step in management.
    • It assumes at least a fairly mature and stable management environment.
    • If you don’t have that, it’s hard to say what, if anything, your measurements tell you.
    • So don’t start to measure too early – it’s not just a waste of time, it can be uselessly distracting.
  5. Have a clear, explicit model of what needs to be measured, and how.
    • Quantitative data means nothing in the absence of a convincing qualitative model of how the various factors that cause the measurements work.
    • Without such a mode,you will not be able to identify the levels the results of your measurements mean you need to pull.
  6. Use the right tools.
    • If the data, tools and techniques you use are not at least fairly well established and directly suited to your problem, then your measurements will mean practically nothing.
  7. The last step should always be to turn the numbers into a meaningful statement that means something substantial to real people.
    • Remember, most people really don’t care much about numbers as such!
    • And you need to make that last step to explain why the results matter.
  8. A measurement programme is not just a technical tool, it’s a whole management programme.
    • And as with every management programme, success comes from spreading awareness, commitment and involvement.
  9. Things must be seen to improve following metrics-based reports.
    • Otherwise what is it for, and why should anyone take any trouble over your numbers?
  10. Conversely, only measure things you can really change.
    • Discovering that you are really bad at something you have no choice about (regulations, things that are too expensive or not politically acceptable to change, and so on) is a waste of effort, creates aspirations to improve in areas you don’t control and is just plain depressing.
  11. Avoid relying on single measures.
    • All systems are complex and multi-dimensional, whose reality and performance, even at a single point, almost always rely on the intersection on more than one factor.
    • So your measurements should be comparably complex.
  12. Wherever possible, the programme should both involve and serve the interests of those who collect the data and the people and teams the data is about.
    • Otherwise collecting it will be hard work..
    • … and the quality of the the data will be poor.
  13. Metrics won’t help you to solve unique or isolated problems.
    • If it’s one-off, measurement will seldom help diagnose or redress a problem.
  14. Don’t use metrics to set targets.
    • Targets should be about outcomes and impact.
    • Aiming at a quantitative target leads to quite different behaviour from aiming to achieve a successful outcome.
  15. Don’t use metrics either to recognition and reward or to single out or punish individual culprits.
    • They will soon start to massage the figures.
    • Both personal performance and personal problems are mostly expressions the wider team and system.
  16. Measures must be unambiguously defined, fully understood and consistently applied.
  17. But at the same time, targets should always ranges.
    • Reality is always fluctuating.
    • Your metrics should allow for this fact.
    • But this also means that the boundaries of your ranges should be clear and well defined.
  18. Avoid vanity metrics.
    • Only invest measurement effort in making a difference, not to preen or score political/cultural points.
  19. Make sure the results end up on the right desk.
    • If results aren’t communicated to the person who can take action, they serve no purpose even though they create the (false) impression of useful activity.
  20. Measurement isn’t trivial. Investment, training and tools must all be provided and sustained.
  21. Make sure your metrics programme evolves.
    • Always be ready to move onto a deeper or broader understanding.
    • Always check that your metrics are still meaningful (and mean the same thing) when the system or its inputs and outputs change.
    • Conversely, if you’ve exhausted what you can usefully find out about an area and have brought it under effective control, it may be time to stop measuring it!
  22. Conversely, if you’ve exhausted what you can usefully find out about an area and have brought it under effective control, it may be time to stop measuring it!
  23. Finally, always remain a metrics sceptic.
    • Numbers as such aren’t knowledge.
    • All numbers are inherently indirect representations of the underlying phenomenon.
    • So always emphasise the quality of the problem-solving or analytical method over mere numbers.

By |2018-06-27T15:34:06+00:00Friday, August 8 2008|Categories: All, Metrics|0 Comments

About the Author:

Chief Architect, Agile201.com.

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Want to do more than just build systems?             
error: Agile201 is wholly copyright and copy-protected.