A Case Study: Increasing Training ROI with DASA

Developing employee competences is an important activity faced by all IT Managers. They must make sure the team has the right knowledge, skills, behaviors and attitudes to do their jobs at the level expected, and also to create the right culture to innovate and keep the work attractive enough to retain employees.

To increase the value of a training investment, we need to focus on a few critical aspects:

  1. Focus the training on developing the necessary competencies so employees can solve problems (learning need).
  2. Present students with a clear objective about what to learn and how it should be applied.
  3. Choose the most effective and cheapest solution.
  4. Create a clear plan for how knowledge is transferred into day-to-day work.
  5. And if possible, measure outcomes.

In this article, I will demonstrate how you can use the 8-fields model to translate a serious learning need into a training result that solves an actual problem. I will apply this model to a real life DevOps situation using the DASA Competence Framework.

The Case: A New DevOps Endeavor

An IT department has decided to start applying DevOps. They have created two (2) teams to experiment with and want to kick off these teams with The Phoenix Project simulation. They contact a training provider with the question:

Can you make an offer for 2 The Phoenix Project Simulations?

A great question and a great opportunity for the training provider. The training provider wants to deliver an effective solution to the customer, but what is the best way to do this?

Case ToolBox: The 8-Fields Model

The 8-fields model is an instrument for designing effective learning interventions and measuring the effects of the training intervention.

8-Field Model

In order to design an effective learning intervention we have to know the Goal or the Problem we want to solve using the learning intervention. We also want to have a clear picture of how the ‘customer’ sees the ideal situation (the Wish).

Now, if we move to the right in the model, we come to the box called Functioning. After executing the learning intervention, we want to see training participants (TP) applying the lessons learned in day-to-day work, by behaving and functioning differently and thereby creating the wish situation. We need to measure this effect, because at the end, we want to know if the training investment has lead to a Business Result, meaning we have solved the problem or reached the goals.

Based on the wish situation, we can create a list of required Competences to develop in learners in order to solve the problem and transfer the knowledge into daily work and create business value/outcomes.

If we have a clear picture of these competences, we are in a good position to design the most effective and cheapest Learning Intervention. And to ensure the learning intervention is well designed, we evaluate the Learning Process and we (maybe) run some Tests to demonstrate that the intended competences are being developed.

This model will be the guide to exploring the learning demand of the customer.

Now… let’s go back to the initial question:

Can you make an offer for 2 The Phoenix Project Simulations?

The first step is to figure out what goal/problem we must solve and then we need to determine the wish (desired situation). During initial meetings with the customer, the current business situation as well as the desired situation, or how he wants to see his employees and teams working after the training, are determined.

We encourage the customer to define desirable behaviors in present tense and specific language. Each description of desirable behavior MUST link to one of the problems or goals (on the left). If not, we have two options (1) update problem/goal or (2) skip the desirable behavior.

Problem/Goal (Current Situation) Wish (Desired Situation)
No shared understanding of DevOps Shared understanding, speaking the same DevOps language
Not applying DevOps, no mindset Teams start applying DevOps, experiment, improve
Employees are scared to try new things Team leaders encourage questioning, asking becomes normal
Too focused on doing, not on reflecting, no learning culture Teams organize learning sessions, experiment, improve,
learn in day-to-day work
Management support and motivates the learning culture
Silos and silo-thinking throughout organization Employees enjoy working in teams, management encourages this

Case ToolBox: The DASA Competence Framework

Next, we need to determine with the team which competences they still need to develop. This is the moment where the DASA Competence Framework becomes important. We will look at the 4 Skill Areas to demonstrate how we can use the framework in combination with the 8-fields model.

DASA Competence Model

Simulations are a perfect instrument for developing skills. So, during the intake with the customer we can explore which of the specific skills we need to develop during our Learning Intervention.

Using DASA’s Competence model, we can explore the following questions about the training participants.

Courage Teambuilding
  1. How proactive are the employees?
  2. What about self-reflection skills?
  3. How do people deal with uncertainty?
  4. Are managers stimulating open discussions?
  1. How well do people work in teams?
  2. Is there a common purpose?
  3. Are people interested in each other?
  4. Do we see management promoting teamwork?
DevOps Leadership Continuous Improvement
  1. Do we see managers facilitate high performing teams?
  2. Is management transparent?
  3. Does management show desirable behavior?
  4. Does management trust the teams?
  1. We are always looking at making our work better.
  2. It’s normal to have improvement sessions.
  3. Quality is on the mind of most of our teams.

Reference: GamingWorks

Ensuring Transfer of Learning

We also focus on developing something called Transfer Skills. These are the extra skills needed to integrate the competences developed within the teams during the Learning Intervention into day-to-day work.

From this exercise, we determine the need to develop the following Transfer Skills within individuals in the teams:

  1. TP is able to implement new lessons learned and new ideas into day-to-day work.
  2. TP is able to influence colleagues to start to participate in this new way of working.

Designing the Learning Intervention

Based on our understanding of the necessary competencies to develop in these teams in order to move them from the current situation to the wish situation, we decide upon the following learning intervention.

But first, we want to keep in mind some of the key principles for designing effective learning interventions:

  • Continue with where we are now.
  • Involve roles we already have.
  • Focus on transferring knowledge.

The customer has a few coaches who are active in helping teams to use more Agile ways of working. We like to involve those coaches in our learning intervention. To get going, we take the following steps:

  1. We organize 2 The Phoenix Project sessions.
  2. The IT Manager introduces each session with goals and introduction and his commitment and expectation of this workshop.
  3. Each session is facilitated by a game leader supported by an Agile coach. The coach facilitates the improvement cycles between the rounds.
  4. Each round we discuss the aspects as defined in competences. We ask the participants how we can apply the discoveries in our day-to-day work.
  5. We give teams room to organize their own processes, and to improve them. We reflect on team competences and skills and make participants aware of their own role on the team.
  6. After the session we have a team debriefing in which we focus on the transfer of knowledge. We ask the team questions like:
    • What did you learn?
    • What can we use in day to day work? How?
    • What stops us from doing this?
    • What is our plan? To fix this? And to implement our ideas?
    • How are we going to do this?
  7. After this debriefing, we motivate the team to start their plan and we follow up with 3-5 short action learning session in which we help the team to work on their questions and experiences, also with the help of the coaches.

How Do We Know if We Increased Training ROI?

To learn if we made the right design decisions, we have to evaluate the learning process we have designed. We don’t only want to do a superficial check; if the trainer was ok, the location was nice, the coffee was ok, the lunch was ok. We also need to check if the balance between doing, listening and reflecting was effective.

Case Study Increasing Training ROI DASA

To see the real value of the training, we need to measure on the levels of both functioning and business results. We ultimately want to know if all our training efforts have provided value. We monitor and measure this by observing the new behaviors and results of the team in the workplace. And this should all be part of the continuous improvement cycle. We check if we are moving the right direction (toward the wish) and when we are, we celebrate. If we see that progress is moving too slowly, we adjust or take appropriate actions to get back on track.

As a part of the new culture we create by developing these competences, we should also always focus on making things better.

Wrapping Up the Case

I hope this article will inspire you with a new perspective on learning and development. The key is to focus on return on investment and to make training more effective as well as more fun. We believe that simulations are the perfect instrument to allow employees to learn by experiencing a situation where they can experiment, fail, learn, reflect, celebrate, and improve.

And along with the simulation, we see that The DASA Competence Framework is a valuable tool for assessing the current level of competences (current situation) in teams and comparing them with our ambition (wish situation). Using this tool along with the 8-fields model, we are able to create an effective training that really solves organizational problems.

Jan Schilt

Owner / Director, GamingWorks

Jan is the co-founder of GamingWorks, a simulation training company based in the Netherlands and a DASA Training Partner. His background is in Human Resource…