Ensuring Training Effectiveness – Evaluating Good Manufacturing Training
Introduction
Wash your hands before entering production! Wear your hair net properly! We just completed a Good Manufacturing Practices (GMP) training last week! Ahhh! Wouldn’t it be nice in a dream world to walk around a food-manufacturing facility and not see any violation of proper GMP? Where all employees would inherently know how to manage their locker and would know not to bring candy and gum with them when entering the production floor?
Alas, many of us are not able to live in this dream world and so we must train, train, and train. But how do we know that the training is effective? Many food-manufacturing facilities spend a tremendous amount of time developing and delivering training, but if employee competencies don’t improve, is the money well spent?
The Four-Step Model
That’s why training evaluations are so crucial. Without them, we don’t know if the issue is the material, the person giving the training, or the motivation of the employees. In 1954, Donald Kirkpatrick developed a very simple four-step model for evaluating training effectiveness and learning experiences in manufacturing that by the 1970s became an industry standard. In the early 2000s, research consultant Jack Phillips adjusted it by modifying the fourth step and adding a fifth. As currently constructed, the Kirkpatrick-Phillips Model builds successively, with each step dependent on the previous one, like blocks founding a pyramidal structure. The steps include:
Level 1 – Reaction
Level 2 – Learning
Level 3 – Application
Level 4 – Business Impact
Level 5 – ROI (return on investment)
Using this model, let’s look at the steps in the context of GMP training. For our example we use a formal annual training event, but the steps can be modified for informal or quick-learning experiences as well.
Applying the Model
Reaction. Level one, or the “smile sheet,” requires employees’ quick reaction to a training. On a scale of one to four, with one being completely unsatisfied and four being completely satisfied, they must answer the following questions: Was the training room comfortable? Were you able to understand and hear the instructor(s)? Was the snack food acceptable? Was the training effective? Do you feel you can use this knowledge in your day-to-day activities?
These are just some examples of the questions to ask at this stage. However, be careful to only ask questions whose conditions you can change. For example, if no other training location is available or employees don’t like the chairs, don’t ask them about those subjects. Indeed, if training coordinators don’t craft their questions carefully, they might inadvertently create suspicion and doubt among employees about the relevance of the entire exercise. Workers might decide that giving their opinion is pointless. They may become disillusioned with the training, making them less likely to buy into the next session level.
Learning. Level two assesses whether or not learning occurred during the training via a knowledge test or skills assessment of some sort. Some examples include being given a color code and using it to indicate the proper color to clean a drain. Or the proper color to clean a piece of product contact equipment. After the participant completes the exam, the facility keeps a record of their performance as proof that they have been trained. The level two assessment is completed immediately after the training or before the participant leaves the training area. It is a quantitative measure verifying that learning occurred in terms of gaining knowledge, developing proper skills, or changing work-safety attitudes.
Application. Usually, trainings result in a manufacturing facility achieving level one and level two competencies, either formally or informally; however, that does not indicate whether or not an employee has internalized the training and is going to use it in their job. Level three directly addresses competencies and their retention via auditing. About three to six months after employees have completed the training classes, a food-safety or quality-management team will routinely conduct internal inspections to confirm if employees are using the GMPs which they’ve learned. Some of the yes/no questions that tend to make up level three might include, Are all employees wearing their hair nets appropriately? Is the chlorine concentration adequate in the dip stations?
Level three evaluations occur on a scheduled basis, triggering corrective action when the results are unsatisfactory. Although it’s tempting to link the results to ineffective training, it’s common practice to do a comparative analysis instead: consider how many GMP nonconformities occurred in audits before and after training. If you discern a positive change in GMP compliance, you can then attribute it to training effectiveness.
Business Impact. Level four assesses the impact of GMP training on an organization. Examples of some of the questions include the following: Was there less downgrade from elevated microbiological contamination? Did the sanitation time decrease? Were there fewer customer complaints? Some of the positive results can be attributed to employees following their GMP training.
Return on Investment. Analysis of the training’s monetary benefits and costs defines the last level. Use the following equation to begin:
ROI = (Program Benefits – Program Costs × 100) ÷ Program Costs
Examples of some of the program benefits include monetary savings from decreased downgrade or rework; or the increase in product gained from less sanitation time, etc. Program costs include money paid to employees during the training (wages), trainer costs (fee or wages), or expenses from the venue (like snacks provided), etc.
Conclusion
Training is a must. It is thus crucial that you evaluate it for effectiveness to ensure that your company is getting a return on the time, resources, and effort it requires—ultimately, that you are manufacturing a safe product. Indeed, in today’s globally competitive economy, when we are constantly looking for ways to cut costs, these are very important goals. In food manufacturing, there is always room for improvement, including training regimens. Good luck and make sure you are wearing your beard net correctly!
About the Author
Janna Verburg-Hamlett—University of Idaho Extension/TechHelp, Assistant Professor and Food Safety Specialist, Department of Animal, Veterinary, and Food Sciences
BUL 987 | Published October 2020 | © 2022 by the University of Idaho
