MEASURING THE IMPACT OF TRAINING
DEMONSTRATE MEASURABLE RESULTS AND RETURN ON INVESTMENT Why do some Training & Development functions have increasing influence and impact, while others constantly battle for more resources, time, and management attention? Included with this Participant Workbook is the following guidebook, published by Richard Chang Associates, Inc.
•
Measuring The Impact Of Training
Have you ever heard some manager or executive say … “Send my people to training—not me! It’s a waste of time, it costs too much, and it doesn’t work! I need my people to do some real work, not sit around and talk about some theoretical ideas that they can’t apply on the job!” The answer to both of these issues frequently has to do with the way that Training & Development is approached. Successful Training & Development functions typically use an impact- or results-based approach, rather than an activity-based approach. Measurement is a key driver in establishing a successful, resultsbased approach. Strategic use of measurement and evaluation techniques can improve the value, quality, and impact of your effort; demonstrate the need for the effort; and increase management’s support, involvement, and resource allocation to the Training & Development function or system.
Learning Objectives
Upon completion of this session, you will be able to: 1. Demonstrate the business-focused alignment and linkage between Training & Development efforts and your organization’s overall performance 2. Develop evaluation tools/processes that demonstrate the employee response and performance improvement from the training (Participant RESPONSE and On-The-Job ACTION) 3. Develop evaluation tools/processes that demonstrate the businessfocused RESULTS and organizational IMPACT of the training
© RICHARD CHANG ASSOCIATES, INC.
1
MEASURING THE IMPACT OF TRAINING
Measuring The Impact Of Training (cont.)
Agenda
Developing An Aligned Purpose What To Measure In Training & Development Efforts Selecting RESPONSE Measures Selecting ACTION Measures Selecting RESULTS Measures Selecting IMPACT Measures Calculating IMPACT
© RICHARD CHANG ASSOCIATES, INC.
2
MEASURING THE IMPACT OF TRAINING
WHAT TO MEASURE IN TRAINING & DEVELOPMENT EFFORTS
Answering these questions keeps measurements valueadded and aligned with strategic business goals. If you do not know what is important to measure, you may not be measuring the right things!
QUESTIONS TO ASK
MEASUREMENT OUTCOME
1. What impact should this Training & Development effort have on the organization?
Organizational IMPACT
2. What business-focused results should be attained?
Business-Focused RESULTS
3. What on-the-job action should take place as a result of the Training & Development effort?
On-The-Job ACTION
4. What does the participant response to and from the Training & Development effort need to be?
Program RESPONSE (Did they like it? Did they learn?)
What you measure depends on (1) your organization’s needs and (2) the size, breadth, and depth of a given Training & Development effort. You will need to look at each Training & Development effort individually, and decide which measurement(s) will bring the most value.
Levels Of Measurement
In 1959, Donald L. Kirkpatrick revolutionized the Training & Development industry when he defined four levels of measurement (reaction, learning, behavior, and results). Using Kirkpatrick’s widely-accepted framework, RCA has developed a practical approach to demonstrate the organizational impact of your Training & Development efforts.
KIRKPATRICK’ LEVELS 1. Evaluating Reaction 2. Evaluating Learning
⇒ Program RESPONSE
3. Evaluating Behavior
⇒ On-The-Job ACTION
4. Evaluating Results
⇒ Business-focused RESULTS
5. < © RICHARD CHANG ASSOCIATES, INC.
RCA’S APPROACH
> 6
⇒ Organizational IMPACT MEASURING THE IMPACT OF TRAINING
SELECTING RESPONSE MEASURES RESPONSE evaluation enables you to: RESULTS
ACTION
IMPACT
RESPONSE
•
Get feedback from the participants regarding the program’s quality and helpfulness (response TO the program).
•
Find out if participants have learned what the training attempted to convey (response FROM the program).
•
Build a foundation for other training measurements.
As you work through this step of the measurement process, consider the following: 1. Allow time for evaluation. 2. Create a simple, short evaluation. Measuring The Impact Of Training Pages 26–29
Remember Alfonso …?
3. Don’t skew results. 4. Include an evaluation form for the trainer/facilitator/coach.
MANAGEMENT TEAM RESPONSE IDEAS Have participants rate their response to the program through a Session Questionnaire. For example, how well did the participants like … what was their reaction to … Program Materials • Depth • Exercises
Trainer • Knowledge • Motivation
Process • Pace • Schedule
Have participants rate their response from the program (their “new” knowledge) through self-report (session questionnaires, role-play, demonstration, etc.) … For example, during Management Training, how well did participants learn to: • Plan Employee Performance • Manage for Quality •
Coach and Develop Others
•
Value Diversity
•
Focus on Customer Needs
•
Evaluate Employee Results
•
Foster Teamwork
•
Demonstrate Adaptability
Measuring The Impact Of Training Pages 30–35
© RICHARD CHANG ASSOCIATES, INC.
7
MEASURING THE IMPACT OF TRAINING
Selecting ACTION Measures (cont.)
Remember Alfonso … ?
MANAGEMENT TEAM ACTION IDEAS •
Within one week of Management Training (and continuing on thereafter), participants will be: Completing and using Feedback Planners in 80% of the coaching of employees Tracking department efficiency using the described process management system Applying motivational tools real time Etc.
•
Within 90 days of Management Training, participants will have: Performance Plans in place for all direct reports Orchestrated at least one Voice of the Customer Survey Initiated a Diversity Ranking Scale Etc.
Alfonso’s Survey
Recently, a member of your team completed Management Training. Please evaluate this person’s use of the following tools on the job. Use
Competency
Topic
Rating Scale of 1 to 5
Feedback Planner
1 = Little to no use/ Little to no competency
Process Management System Motivational Techniques
5 = High use/High degree of competency
Performance Plans Voice Of The Customer Surveys Personal Profile System® (DiSC®) to guide in flexing styles to build team relationships CPI Processes Diversity Communication Planner
Measuring The Impact Of Training Page 43
© RICHARD CHANG ASSOCIATES, INC.
17
MEASURING THE IMPACT OF TRAINING
Selecting RESULTS Measures (cont.)
Results Example— New Employee Orientation MEASUREMENT CATEGORY: TIME
BASELINE MEASURE
PROJECTED TARGET
30 working days
10 days
15 days
60 days
30 days
25 days
500 hours per year
100 hours
75 hours
MEASUREMENT CATEGORY: OUTPUT
BASELINE
TARGET
ACTUAL
4. Number of new employees going through New Employee Orientation per month 5. Number of partnerships established to support New Employee Orientation process 6. New Employee Orientation feedback sessions per year
25
50
45
0
10
9
0
12
4
MEASUREMENT CATEGORY: QUALITY
BASELINE
TARGET
ACTUAL
7. Customer satisfaction with New Employee Orientation—as reported back by new employees 8. Customer satisfaction with New Employee Orientation—as reported back through Business Unit Management 9. The degree to which the New Employee Orientation process addresses basic benefit questions—as validated by the Benefits Specialist Group
5 out of 10
9.5 out of 10
9 out of 10
3 out of 10
9 out of 10
9 out of 10
20%
90%
80%
MEASUREMENT CATEGORY: COST
BASELINE
TARGET
ACTUAL
$1,200
$200
$335
$30,000
$10,000
$15,000
20% over
5% over
2% over
1. Average New Employee Orientation time (from general company introduction through basic, on-the-job training) 2. The time required for new employees to be performing at 90 percent efficiency 3. Average time spent by Benefits Specialists answering basic policy questions—after New Employee Orientation
10. New Employee Orientation cost per new employee 11. Average monthly costs of a New Employee Orientation 12. Operating budget variance per New Employee Orientation program © RICHARD CHANG ASSOCIATES, INC.
26
ACTUAL RESULT (AFTER TRAINING AND DEVELOPMENT EFFORT)
MEASURING THE IMPACT OF TRAINING
CALCULATING IMPACT Cost/Benefit Calculation Sample Cost/Benefit Analysis
Net Benefit = Total Benefits – Total Costs
BENEFITS
AMOUNT
Additional application fees
$ 500,000
Additional closing fees
300,000
Additional productivity savings
65,000
Reduction in processing errors
250,000
etc.
etc.
Reduced system technicians billing for repair of old system Total benefits
$ 2,400,000
Total costs
876,893
Net benefit
Return On Investment
125,000
$ 1,523,107
ROI = [(Total Benefits – Total Costs) ÷ Total Costs] × 100
= ROI = [(Net Benefit) ÷ Total Costs] × 100
ROI =
Net Benefit Return = = Total Cost Investment
$ 1,523,107 $ 876,893
= 1.74 = 174%
Measuring The Impact Of Training Pages 80-90
© RICHARD CHANG ASSOCIATES, INC.
33
MEASURING THE IMPACT OF TRAINING
MEASURING THE IMPACT OF TRAINING
DEMONSTRATE MEASURABLE RESULTS AND RETURN ON INVESTMENT Personal Action Plan Key learnings: _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ I need to contact and involve the following: Name
How
When
_______________________
____________________________________
______________
_______________________
____________________________________
______________
_______________________
____________________________________
______________
_______________________
____________________________________
______________
The specific action steps I will carry out on the job include: What 1. ______________________________________________________________
When ______________
2. ______________________________________________________________
______________
3. ______________________________________________________________
______________
4. ______________________________________________________________
______________
5. ______________________________________________________________
______________
Additional issues/concerns I need to address: _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ © RICHARD CHANG ASSOCIATES, INC.
37
MEASURING THE IMPACT OF TRAINING
