The promise of agile coaching: measuring team performance

Many companies use agile coaching to help teams to become effective at doing agile and scrum. Even though the idea of agile coaching is often described as beneficial, there is surprising little evidence for this effect. Evelien Krooshof has therefore measured the effect of coaching by the agile coach / scrummaster in her recently published master thesis: “The Promise of Agile Coaching”.

Measuring coaching and performance

The agile community prides itself on their feedback driven approach. The idea of measuring results often and responding to feedback is built into the agile methods such as scrum. The effect of coaching is unfortunately hard to measure. First, there is no easy way to measure performance across teams. Secondly, there is no easy way to measure the amount and quality of coaching. In “The Promise of Agile Coaching”, this is addressed by using detailed questionnaires. Evelien has combined questionnaires from previous researchers into a complete measurement tool, and has validated this tool by interviewing 48 agile teams. This questionnaire with previous answers is a very useful practical tool and we hope that other researchers will use the same questionnaire so that research results can be compared and combined.

Main result: coaching has no measurable effect

Evelien Krooshof has used a model, shown below, to illustrate how the effect of coaching is supposed to happen. It is based on the previous studies of coaching and effectiveness, and shows that agile coaching should not only have a direct and immediate effect , but should also work through team empowerment and shared leadership. The effect of coaching should also depend on management support.

Unfortunately, Evelien’s study ‘has not found significant results for the impact of agile coaching on team performance’ (p.35). Management should therefore be critical of employing too much agile coaching in all situations. Coaching probably has some benefits, such as a faster team start or more insight for management. The amount and type of coaching however should be adjusted regularly to the composition and maturity of the team.

This conclusion from this thesis fits well with the experience of ICT Institute: we assign coaches to teams for fixed periods, such as the startup phase and adjust coaching regularly based on the team development. In the end, the team itself is the most important success factor. Companies that want highly effective teams should invest first in good team members and product owners. Agile coaching should be evaluated and adjusted regularly.

Using the Krooshof Questionnaire as a team tool

The questionnaire that Evelien Krooshof developed is a very useful tool for understanding team effectiveness. Teams can use the tools to gather input on team performance from all team members, and to see how they differ from other teams. Below are the top 10 questions from her list that have the highest correlation to team performance. Teams that have higher scores on these questions, also rate their overall performance higher.

To use these questions, ask each team member to rate each item from 0 (completely disagree) to 1 (completely agree). The scrum master should then compute the average score and compare with the score in the chart. This will show you if you are doing better or worse than most teams. If your score is below the low quartile score, the score is lower than 75% of other teams. Any outlier, positive or negative, should be discussed and actions should be defined to improve on the lower factors.

We would like to thank Evelien Krooshof for her excellent research work and for sharing her data per question with us. If you would like to use her anonimized data, you can contact her directly or via the form below. 

Image: rugby by Quino Al via unsplash