Level 3 analysis

Casey, M.S.& Dover, D. (2005) Training Needs Analysis and Evaluation for New Technologies Through the Use of Problem-based Inquiry. Performance Improvement Quarterly 18(1) pp. 110-124

Casey and Dover detail an interesting method of determining level 3 analysis data: analyzing the frequency and type of help desk calls. This makes a lot of sense - these calls are made when recipients of technical training attempt to apply that training to their jobs. The calls reflect the inadequacies of the trainees' memories and whatever job aids they constructed or were given. One caveat mentioned in the article was that, from their data, they couldn't tell whether the trainees had even attempted to use their job aids. This would indeed be interesting as it would suggest whether the gap lay in the aid, or whether it lay in the trainee's belief as to the effectiveness of the job aid; two quite different training challenges.

Of course, I don't do technical training but it is interesting to think of how to apply this to soft skills. One thought we've been throwing around in my department is the idea of providing some sort of electronic bulletin board for our students. That way they could ask questions of each other, share stories and tips, etc. Essentially keep the learning experience going beyond training. It remains to be seen how effective such a board might be - according to the literature, technology like that is rarely used - but it would be a rich source of feedback for the designers. The questions and stories would pinpoint deficiencies in our training.

I was also thinking of a limited "training help desk," where our trainers would respond to any emails or phone calls about applying training to the field. I think that might undermine the role of their manager.

I'll continue to ponder this way of looking at level three data. It's much more cost efficient than a field analysis and probably more accurate than a survey.