Theory-to-practice and Education Controlling

Essay by Annette Bouzo (CELM), Dr. Dirk Bade

According to the Institute of German Economy (Institut der deutschen Wirtschaft IW, Köln) German Enterprises invested more than 28 Billion Euro in employees' education in 2008. (1)

But how much do those corporations actually profit from this invest? Why does theory-to-practice fail and how can the success quota be increased? Which aspects of education controlling can accompany trainings and thus support target achievement?

Lack of effectiveness of training programs is certainly a common critic, as the overview of invest and benefit is not always clear. When investigating these fields, a common argument is, that some aspects can simply not be measured. Other regular arguments indicate deficits in organizational development or corporate culture. Complaints that employees do not make use of the newly acquired knowledge, training contents are not up-to-date or simply not what the learning requires, show the broad scope of perspectives.

But how is to be proofed, that trainings, especially those which are seen as mere incentives, actually have an impact on the strategic development of employees?

Progress after trainings can in effect not be measured at all if no desired training objective has been defined, which not traceable.

Also, if employees are to take part in change management trainings, but the management prevents any changes or is not aware, that changes are necessary a transfer to practice cannot be exercised. In addition to cost also motivation of employees will be destroyed.

Staff development can also be initiated without involving the respective departments and selecting single persons who are interested in initiating changes (Bottom -up). This can be damaging specially to complying and ambitious employees, who will be confronted with irrevocable structures, traditional processes and management behavior. These initially highly dedicated employees will in fact or inwardly resign or suffer a burnout due to frustration. Is education not far too expensive anyway if the theory-to-practice-quota cannot be measured?

Cost of implementing education controlling is not to be underestimated. However, education controlling does not only cover mere financial aspects and is not understood as a chance of regulated navigation.

This is a fundamental misunderstanding: even without proven application of knowledge to everyday work, educational controlling documents formal qualification of simple trainings, whilst of course successful application is possible without education controlling. The latter might be difficult to reproduce and cannot structurally transferred to another training group, as key indicators might be ignored.

More complex and coherent trainings require educational controlling in any case for selection of participants, measuring of success rate and further motivation of learners.

To be successful, strategic education programs require an open corporate culture, which is supported by management and employees.

Without a transparent communication of training objectives education campaigns will just be passively consumed but not actively adapted.

Furthermore, the conceptional integration of educational campaigns in the operative enterprise strategy on all levels is mandatory to control the position of the corporation in competition and implement changes and corrections in time, based on education controlling data.

Relevant prerequisites for successful change processes are suitable operative structures within the corporation, matching qualification and skills of employees and the serious intention of management to support the changes (2).

To define the needs assessment a learning scorecard, matching the strategic roadmap can be employed. Goal definition and the desired target achievement rate of single training programs must be integrated in an overall concept of organization and personnel development, which is to be communicated to the individuals involved. The integration and acceptance and their resulting commitment is mandatory for a sustainable training result. Only when goals and methods for measuring success has been defined, the results and -hopefully- success can be proofed.

Typical tools for evaluation of trainings are usually:

• Participants' feedback directly after training
• Informal feedback from participant and/or manager
• Assessments and certification routines
• Controlling of target achievement, defined in the trainingsplan
• Statistics by monitoring KPIs (decrease of complaints, increase of productivity...)
• Evaluation before and after training by supervisor

The overall result can be integrated in scorecards, which visualize educational progress and target achievement. This has to be seen as a continuous, iterative process.

Potential barriers should be identified, assessed and kept in mind even before defining the training needs, so the reduction of those handicaps can be started right away. Indications for transfer barriers can already be detected by employees' surveys and management interviews within the approval workflow. Accompanying the whole training process from the start facilitates the minimization of transfer barriers.

Even critical (and therefore usually unpopular) comments should be impartially embraced and assessed. Anonymized surveys can be carried out over encrypted links or even in classic paper form. Employees participating in surveys should be informed about outcomes and resulting steps.

Surveys have been detected reasons for lack of economic allocation of training programs in corporations, which can typically be assigned to (i) individual participant or (ii) in a suboptimal work environment:

• lacking preperation of participant / instructor
• lacking motivation of participant / instructor
• no time to transfer learned skills to actual work
• fear of participants from novelties
• skeptical colleagues / managers
• processes do not allow for employing new skills
• supervisor scared of competition / decrease of authority
• lack of communication after training event
• training is incentive but not job related
• training content irrelevant for tasks (3)

Therefore, a part of the problem of inadequate transfer from theoretical knowledge to successful employment of daily routines is not lack of quality of the training programme as such, but in the work environment of the participant. The professional magazine 'managerSeminare' quotes 2007 from a survey, that 77%(!) of participants cannot integrate the training contents in their working environment. (4)

Training programs resulting from strategic company's goals can be assigned to verifiable target dimensions. In any case an indicator and the method of measurement has to be defined. Learning Management Systems offer a structured framework for data within their training administration unit.

Documented objectives and the desired target achievement level must be transparent to enable a fast and clear assessment. Hierarchic structured target profiles, which should be evaluable by gap analysis are monitored automatically on a daily basis. This way they can confirm the strategy or indicate the need for readjustment.

Need for evaluation might differ from training to training. Skills such as operating certain tools or machines, a compliance training or a training for analyzing KPIs differ greatly e. g. in the way the participant can engage and act independently from his supervisor, his team or his enterprise unit. Also in terms of training format and media an evaluation of theory to practice transfer will differ.

According to Dave Ulrich HR and personnel developers are to be management partners and are to support them by achieving the enterprise objectives. (5)

Thus, a close cooperation and information exchange between staff development / academy, HR and functional management is mandatory. Ironically many managers see themselves as operative and functional supervisors and conveniently delegate training and qualification issues to HR. In companies, such as Microsoft managers' success (and their financial compensation) is closely connected to the effort they make, in supporting qualification of their team.

It has also been discussed to disestablish personnel development as a department, to force supervisors to get involved themselves. An alternative way would be to limit the tasks of training departments to subjects such as training communication and service orientation.

However, there can be no discussion that the training strategy must be integrated in the overall corporation's strategy. - no matter who 's is to take the responsibility for it.

An approval workflow offers fantastic chances to start a productive dialogue. Expectations of employee and supervisor can be documented and integrated in the training process.

Usually approval workflows have been implemented as very convenient one-click-solutions. But the process can be enhanced right at the start by integrating a questionnaire to give room for concerns and motivation of the participant-to-be. Also, potential transfer barriers when it comes to putting theory to practice after the training unit might be detected and abolished from the start.

Also, the supervisor is to make his statement concerning the actual status-quo of the training need as well as a brief comment how he is planning to support the employee when it comes to changes after training. Actual or mental barriers can only be tackled after they have been identified. This way the need of management support is formulated and addressed as an actual to-do of the supervisor.

A process as described offers a chance to evaluate the communication process of the enterprise, as the approval process, the documented rating of participant and employee which are both important to the transfer. The complete communication and documentation process from administration to evaluation of data can be supported by a suitable Learning Management System.

A participant can not only be provided with a feedback form of the academy, but also from staff development and supervisor. The key issue is not only to ask questions about the training session right after the course, but also some time later concerning potential barriers, the managers support and the advantage the course has had in the working environment.

Cross institutional (as in contrast to isolated) evaluations can be used to improve data basis for decisions and controlling related processes. Recognized details concerning training quality can lead to different strategic decisions. In general, it can be said, that open information policy and culture results usually in higher quality awareness.

Web portals can provide statistics and KPIs of education campaigns and training programs to different entitled target groups. Examples are e. g. participant statistics, assessment results, project progress, feedback evaluations and others. Data can be displayed and evaluated in various granularities.

Supply-Chain-Management-Systems have proofed a long time ago, that cross-enterprise efforts increase efficiency. Training as such is not entirely different.

This applies also for trainings focusing on organizational development and have therefore a long-term perspective for the corporation and the individuals involved. Organizational development, such as the transformation to a digital corporate culture, means the development of everyone by gaining experience and sharing them in a productive was. The overall improvement of efficiency, competitiveness and quality of the corporation are the central drivers of change processes (6).

Development of staff and organization are closely connected. Change processes lacking an integrative component miss the chance of a dialogue, cannot state their current progress and thus make it impossible to take control.

The obvious consequence of unmonitored progress is a future education programme and budget which is defined by gut feeling. Cutting of budgets is a frequent consequence if transfer from theory to practice and thus efficiency and relevance of staff training cannot be proofed.

To define the grad of target achievement a verifiable and comprehensible target definition is a prerequisite.

Kirkpatrick (7) differentiates five different target levels in training (Level 0-4).

Enrolment and participant numbers serve as an indicator concerning popularity of certain courses or demand of content. They show the workload of the academy and are relevant for quantitively or qualitative planning of the future course offer.

In more regulated education and certification campaigns those numbers are indicators for project status and progress of qualification campaigns. Significant though, that they do not give any information about knowhow transfer or training success.

Customer satisfaction is usually evaluated straight after the training unit. Feedback questionnaires provide information for quality assurance and (positive) rating of the academy. Feedback surveys via questionnaires allow insights of learners' experience and judgement of trainer's quality, learning environment, catering and service.

To allow participants to remain anonymous is often a key element for getting authentic feedback. Therefore, questionnaires can be only be not provided in the personal learning account but also via an encrypted link in an email.

The evaluation of feedback must be flexible in terms of granularity. Evaluation should be possible 'cross event', over various periods and as detailed as broken down to individual questions, question groups or provide insights in general concerning educational campaigns.

Customer satisfaction is an indicator, but not a sufficient criterion for trainings quality. Participant satisfaction can serve as a trend monitor. A positive and pleasant atmosphere increases initially the consumer acceptance, which is why learning surrounding is important. Sufficiently large and light rooms, modern equipment and catering support a positive attitude as participants feel appreciated. A consistent flow of information about the training, the 'reason why', agenda, trainer's CV and e. g.  a roadmap to the academy facilitate the start. To support training organization Learning Management Systems provide automated correspondence via mail.

An open issue, which may not be detected just after the training is, whether the trainer has created a nice and entertaining day for the participants, or if he has triggered changes in behavior according to company's goals.

It may also be considered, that events breaking through the lethargy of participants and therefore serve as a wake-up call, are more efficient as they suggest a certain urgency. However, such method may not be generalized, but must be tightly connected to a certain audience and cause.

Training content and objectives must be coordinated in detail with the trainer, who must be evaluated on these issues accordingly, not only by his popularity amongst his participants.

The key question according to Kirkpatrick's level 2 is, whether knowhow or skill could be transferred successfully to the learner.

Tests assessing the degree of knowledge transfer must be tailored to training content and objectives. If the training is focused on soft skills such as ability to work in a team or social interaction a multiple choice is likely not to be sufficient. In this case an assessment in a serious game scenario might be more valuable.

Knowledge tests must also be positioned correctly concerning time and content
issues. Assessing knowledge levels prior to training can help designing a more homogenous group and speed learning transfer, as every participant starts out on the same level. Heterogenous groups often turn out to be more efficient, when it comes to solve problems or create new solutions.

Another consideration might be how extensive test scenarios must be, to make deliver sound results. Is it sufficient to have one final assessment or might it be more valuable to have intermediate tests to check progress continually? In addition, an assessment can take place sometime after the training to check how sustainable the training has been, how much the learner could apply in his daily work surrounding and if new questions turned up in the meantime.

Online tests and SCORM-WBTs are able to deliver clear results, as they continually document learners progress and allow also quality improvements within a unit. If certain test questions are failed by a significant percentage of learners, it might be that the course content at this point is insufficient and should be improved.

Easy configurable test generators are usually integrated in training administration software or learning management systems. Usually a variety of question types such as multiple choice, sort-order or drag&drop tasks are included and allow for rapidly designed tests. Test generators evaluation can be directly connected to the participant's learning portal and the course. Results are therefore immediately visible to the learner and entitled individuals such as training instructors, superiors or training office.

But do tests make sure that acquired knowledge can be transferred to everyday work tasks?

Could the training content, the knowledge or skill be applied at the assigned tasks? This is where transfer questionnaires with the expectations of participants and supervisors come in. Those might be combined with interviews with the supervising manager or a team workshop.

Behavior modification and change in attitude, better working results concerning quality or efficiency or also less complaints can be indicators.

Another relevant point is the feedback of the supervisor who has approved training and possibly documented his expectations at the time. Where these expectations fulfilled and if not so, what might be the reason?

In general cost of data generation and evaluation must be adequate to the use the results bring. The detail of evaluation must reflect the importance of the training. In consequence course-specific questionnaires and evaluation sheets must be defined and provided.

Helpful indicators for praxis transfer could be:

• a definite need analysis according to the midterm training strategy
• a precise definition of training content
• assignment of training units to training objectives (e. g. according to Kirkpatrick, Phillips or own classification)
• measures and defined target achievement degrees for training classification
• design of questionnaires, possibly integrating circular question types for training centre, HR and department
• definition of evaluation components and questions by HR and departments
• evaluation of target achievements degrees and training objectives by personnel development integrating employees.
• potentially integration in personal target- and bonus system
• Reflection of transfer questionnaire especially focusing on enterprise strategy, monitoring developments within the corporation and market

Company's success cannot be proofed by training evaluation in direct cause-and-result diagrams, as various processes and external influences have an impact on the enterprise.

A product related sales training can only be successful, if the product matches the market requirements and the market is not saturated yet. If training is provided for a sales person leaving the enterprise shortly or the market demand increases suddenly due to the resignationt of a competitor, consequences cannot be ascribed to a training effort.

Only explicitly defined training need, e. g. indicated by the employee's perspective (learning) of a balanced scorecard or tightly focused set-up of strategic competencies can indicate with some certainty that training had the desired impact or if first assumptions have to be rechecked right from the start.

Which effects did a training actually have? Typically, the progress and approach towards company's goals are measured employing KPIs such as production output, quality, failure rate, cost and time units. Seen independently they do not indicate how cost-effective measures were.

Training is not an isolated process. Enterprises have to be seen in a wide context of influences, trends and development. Especially soft-skills influence on enterprise progress can only be measured with some difficulty, as they are also subject to partialities of team, supervisor and setting. However softskills can be critical success factors, when it comes to dealing with customers, employees and suppliers.

Philipps (8) concretizes Kirkpatrick's work and adds whether training outcome can be measured also in financial terms, comparing input (cost) with output based on training evaluation. The resulting ROE (Return on Education) is related closely to the ROI - concept (Return on Investment).

To introduce detailed methods might not directly result goes far beyond possibilities of this text, but we will try to sketch the starting points of Phillips work, which is worth to read.

Whilst the direct cost of training such as trainer fee, cost for equipment are comparably simple to add up, indirect cost such as and 'loss of productive working hours of the participant are more difficult to define. The Outcome ('Net Program Benefits') is already subject to estimation. Apart from relatively easy to assess training programs Phillips recommends mainly two measures:

1. Clear isolation of 2nd level influences
2. Deduction of changes which are hard to measure (simplify)

Possible other external influences on the turnover which are likely not have anything to do with your training programs such as:

• Interest advantages
• Changes within competition or competitive products
• Marketing programs
• Bonus programs for employees
• New technology
• Seasonal effects

Positive and negative results of training programs which cannot directly be expressed in monetary values:   

• Customer Satisfaction
• stress free work surrounding
• Employees satisfaction
• Degree of identification
• Employee motivation

What is especially remarkable about 'Intangibles' is, that an evaluation how much and which training contents employees have integrated in der every day working life and which of the mentioned are seen as the most advantageous.

Prerequisite for an evaluation like this is on open corporate culture and the knowledge of the employee concerning the training objectives. Employees must feel committed to those. At times the participation on a training is part of the individually arranged employee's objectives, which connects to the monetary advantage. This does not facilitate the isolation of cause and effect as recommended by Phillips.

Solely the improved qualifications in connection with the participant's possibilities to apply the new skills to his tasks can be seen as a variable statement concerning the training success. But this does not mean, that a positive training result is a mere obligation to deliver from the employee. The enterprise must provide and maintain an infrastructure and culture which embraces new ideas and processes in accordance with company's objectives.

Especially training units required for external proof of instruction are often internally belittled and not taken seriously. This attitude can affect the whole training organization. Employees' absence due to mandatory, abstract training courses will not be looked upon by supervisors with a friendly eye. However, if relevant training content is passed on or discussed within the team including management, training advantages can be multiplied and easily integrated win the corporation processes.

Data for KPIs (Key Performance Indicators) must also be collected within the training surrounding, but is to be aligned with particularities of a corporation or department. An indicator for differences might be the degree of collaboration in terms of time and precision. When comparing the results of different departments, it must be considered, if these tasks are seen as desirable, are tolerated or even irritating.

A refusal of participation on one side and an unlimited demand of details on the other side can result in a complete paralysis of the project.

A solution mihgt be the prioritization on measurement on a certain limited level, such as a department. Supported by appropriate software a number of valuable KPIs can be defined and evaluated. Particular findings or developments might require clarification, which is then an unquestionable and justified reason to drill deeper. It is helpful to display a number of reasonable KPIs as graphics in a training dashboard / cockpit. This helps to get a speedy insight of developments and trends.

Successful staff development cannot be seen as an isolated process, especially when training is seen as more than just single measures. Staff development must be a part of strategic corporation policy. Organization development, corporate culture, approval- and communication processes must be included in the company's visions and be supported by management competencies.

Trainings must be deliberately assigned to those levels which they are supposed to influence. These levels must be continually evaluated, to monitor developments and measure the degree of target achievement.

A balanced score card concept offers a transparent, dynamic perspective and can be altered in a learning scorecard. A continuous gap-analysis can accompany developments and assist in aligning them to enterprise objectives.

Before declaring training results successful those have to be adjusted to a target level and scales. An indispensable starting information is the feedback of employees concerning their training experience. Also, negative critics must be embraced as a chance for improvement. If training content is not seen as valuable for everyday tasks or incompatible with working environment the question of theory-to-practice quota and achievement level of company's objectives arises.

To help bridging the gap between organization and personnel development a professional Learning Management System can be employed. Trainings are strategically valuable tools. An LMS can reduce the operative workload by completing routine tasks fast and without failure. At the same time processes can be accelerated and information such as controlling and evaluation data is provided for education controlling.

Also, training objectives and their close fit with the company's objectives have to be continuously monitored. Education controlling is necessary to control developments. A suitable LMS collects a number of valuable data and provides significant insights.

1. Behrendt, P., Pritschow, K., & Rüdesheim, B. (Januar 2007). Transfercoaching. zfo. Führung + Organisation .  
2. Bimmler, S., Bonhage, M., & Kleinert, A. (12 2008). Nachhaltigkeit von Personalentwicklungsmaßnahmen. Abgerufen am 27. September 2014 von www.mittelstandsbund.de: www.mittelstandsbund.de/media/raw/Nachhaltigkeit_Personalentwicklungsmassnahmen.pdf  
3. EuPD Reseach Sustainable Management GmbH / TÜV SÜD Akademie. (2013). Bildungs- und Talentmanagement - Jahrbuch Deutscher Bildungspreis 2013. Bonn / München: Selbstverlag.  
4. Frank, G. P. (2000). Personalentwicklung - Organisationsentwicklung: Rolle des Managements. In A. Clermont, W. Schmeisser, D. Krimphove, & (HrsG.), Personalführung und Organisation. München: Vahlen. 
5. Gris, R. (2008). Die Weiterbildungslüge. Frankfurt / M.: Campus Verlag.  
6.Hapkemeyer, J., & Scheibner, N. (Oktober / November 2012). www.statistik-evaluation.de. abgerufen am 15. Oktober 2014 von www.statistik-evaluation.de/wp-content/uploads/2014/05/PidO_10_2012.pdf
7. Kauffeld, S. (2010). Nachhaltige Weiterbildung. Heidelberg: Springer Verlag.
8. Kirkpatrick, D. (1998). Evaluating Training Programs: The four Levels; 2nd Edition. San Francisco / CA: Berrett-Koehler Publischers Inc.  
9. Olfert, K., Steinbuch, & Pitter, A. (1998). Personalwirtschaft, 7. Auflage. Ludwigshafen: Kiehl.  
10. Philips, J. J. (1997). Handbook of Training Evaluation and Measurement Methods. New York: Butterworth-Heinemann.  
11. Seufert, S. (. (November 2007). managerSeminare , S. S.11.  
12. Staehle, W. H. (1991). Management, 6. Auflage. München: Vahlen.
13. Trost, A. (2012). Talent Relationship Management. Berlin-Heidelberg: Springer-Verlag.
14. Ulrich, D. (1997). Human Resource Champions: the next agenda for adding value and delivering results. USA.  
15. Von Rosenstiel, L. (2003). Führung von Mitarbeitern, 5. überarbeitete Auflage. Stuttgart: Schäffer-Poeschel.  
16. Von Rosenstiel, L. (2003). Führung zwischen Stabilität und Wandel. München: Vahlen.    
17. Von Rosenstiel, L. (2003). Gurndlagen der Organisationspsychologie, 5. überarbeitete Auflage. Stuttgart: Schäffler-Poeschel