Linking Training Objectives to Measures of Training Success

Before training evaluation begins, program objectives must be developed. Program objectives are linked to the needs assessment. Let’s look at how objectives can be developed within the context of the five-level ROI framework. Setting clear objectives at each level is important because success with training objectives helps answer basic questions regarding program effectiveness.

Level 1 Objectives (Reaction, Satisfaction, and Planned Action)

These describe issues that are important to the success of the program, including facilitation, relevance and importance of content, logistics, and intended use of knowledge and skills. The best Level 1 objectives should:

• Identify issues that are important and measurable
• Be attitude-based, clearly worded, and specific
• Specify how participants have changed their thinking or perceptions as a result of the program

For example, participants’ perceptions of program relevance is a strong predictor of how well they will apply learned knowledge and skills. So, a good Level 1 objective is:

• At the end of the course, participants will perceive program content as relevant to their jobs.

Reaction data can be collected with feedback questions such as, “Was this program useful, necessary, motivational, and important to your success?” Measures of success with Level 1 Objectives include:

• 80 percent of participants view the knowledge and skills as relevant to their daily work activity as indicated by rating this measure 4.5 out of 5 on a Likert scale.

Level 2 Objectives (Learning)

Level 2 objectives communicate expected outcomes from instruction. The best learning objectives:

• Describe outcome-based behaviors that are observable and measurable
• Describe competent performance that should occur as the result of learning
• Spell out what the participant must be able to do as a result of learning

As with Level 1 objectives, Level 2 objectives should be clearly worded and specific. A typical learning objective may be:

• At the end of the program, participants will be able to implement Microsoft Word.

Sounds reasonable. But, how will you know you have achieved success? You need a measure, such as:

• Within a 10-minute time period, participants will be able to demonstrate to the facilitator the following applications of Microsoft Word with zero errors:

• File, Save as, Save as Web Page.
• Format, including font, paragraph, background, and themes.

Now, you can evaluate the success of learning.

Level 3 Objectives (Application and Implementation)

Where learning objectives and their success measures tell you what participants can do, Level 3 objectives tell you what participants are expected to do when they leave the learning environment. The best Level 3 objectives are observable and measurable, outcome-based, clearly worded, specific, and:

• Emphasize applying what was learned
• Describe the expected outputs of the training program
• Provide the basis for evaluating on-the-job performance changes

A typical application objective might be:

• Participants will use effective meeting behaviors

Again, you need specifics in order to evaluate success. What are effective meeting behaviors and to what degree should participants use those skills? With Level 3 evaluation, you can also follow-up to assess success with learning transfer. Here you look for barriers to application as well as enablers that support learning. Gathering data around these issues allows you to take corrective action when evidence of a problem exists.

Level 4 Objectives (Business Impact)

Level 4 objectives provide the basis for the questions that you ask during the overall evaluation process. They measure the consequences of participant’s applied skills and knowledge and place emphasis on achieving bottom-line results. The best Level 4 objectives:

• Are results-based, clearly worded, and specific
• Spell out what the participant has accomplished in the business unit as a result of the program.

A sample Level 4 Objective might be:

• Increase market share of young professionals by 10% within nine months of new ad launch.

Success with Level 4 objectives is critical when you want to achieve a positive ROI.

Level 5 Objectives (ROI)

Level 5 objectives target the specific economic return anticipated when an investment is made in a program. Some organizations are satisfied with a 0% ROI—break-even. This says that the organization got the investment back. Remember, not all programs are suitable for ROI -- typically only 5–10% of all learning programs are evaluated at this level.

Summary

The value of training occurs when participants react positively to an event; acquire new knowledge, information or skills; apply those on the job after a program; and, as a result of these applied actions, positively influence targeted business measures. Defining specific success measures across multiple levels of evaluation will help you move beyond simple tracking of reaction data and make it easy for your audience to understand the results reported. Each level of evaluation provides important, stand-alone data. Reported together, the five-level ROI framework gives data that tells the complete story of program success or failure. 

About the Authors

By Jack Phillips, PhD and Patti Phillips, PhD

Jack Phillips, Ph.D., is chairman of the ROI Institute. Using more than 20 years of professional experience, Dr. Phillips has developed a sound, tested methodology for calculating true financial ROI for training, leadership and performance programs. He is the author and editor of more than 30 books and 100 articles.

Patti Phillips is president and CEO of the ROI Institute and co-author of Show Me the Money, published by Berrett-Koehler. The ROI Institute is a research, benchmarking, and consulting organization that provides workshops, publications, and consulting services on the ROI Methodology.

This article was re-uploaded from our internal archive.