All too often people reviewing a web-based training (WBT), including subject matter experts (SMEs), request the course printed for them. If it is the absolutely only way they will review it, then I do accommodate them. Keep in mind this is after I have exhausted all other attempts of getting them to do a proper online review.
Here are reasons not to print courses for a review:
- It is important for anyone reviewing a course to not just look at content, but to review the entire learning experience including the delivery medium.
- If they themselves are not willing to participate online how can they expect, or request, our audience to participate.
- Online courses are very often non-linear. Thus, do not fit in a printed, linear format.
- Courses are interactive. They may contain anything from simple rollovers to complex games or simulations. Interactivity does not translate to a printed page.
- Once printed it is occasionally handed around for others to review without the designer’s knowledge. This can result in not being able to identify the origins of edits, if needed. It can also result in draft content mistakenly being distributed to the end user. This can all be prevented by setting appropriate access in an LMS.
- Depending on the authoring tools used, it can be time consuming to print a course. For example, a course that contains many interactive Flash elements will require many screenshots to be taken. Time is better spent on on design and development.
- It is more environmentally friendly to review online. As a fellow e-learning designer said to me recently, “I killed many trees with “WBT to be printed out” for SMEs, higher ups, etc.”
The reality is people reviewing courses are going to push for a printed version and sometimes the only way to get them to review it will be to comply. However, I am not going to comply without at least explaining the importance of an online review. In the end, even if I send them a printed version, or screenshots, I always supply easy access to the online course along with several reminders of how important it is to also review it online.
Benjamin Martin has published “Beta Testing an Online Course“ in Learning Solutions Magazine. It details his approach to beta testing online courses and provides practical advice for what is a very important stage of e-learning development. If you are creating e-learning, then you are probably involved in beta testing and will find this article helpful. If you are not beta testing your courses, then you should be and this article can help you get started.
You will need to subscribe to Learning Solutions Magazine or have a membership to the e-Learning Guild to read the article in its entirety. However, associate membership is free and in my opinion an absolute must for anyone in the e-learning field.
I just read a comment on a blog where a someone was very frustrated by bland, unengaging page turners. It got me thinking. How do you get a an organization out of the rut of making page turners and to start creating more engaging and effective courses? Here are my first thoughts:
- Put on your instructional designer hat and do everything you can to educate all involved (SMEs, clients, managers, and audience too) on what effective e-learning is and how all involved can benefit from it.
- Show all involved what effective e-learning looks like, actual examples. Here is just one place where you can find examples –http://minutebio.com/blog/free-e-learning/ (this my Free e-Learning collection ).
- Find case studies, articles, evaluations, etc. that support your case.
- Create a prototype to demonstrate the level of interactivity and engagement your organization can produce in a course. Get your co-workers involved so they will be vested in the “new approach.” This will earn you supporters and people who can rally against the archaic page turners the organization still wants to produce.
- When you launch your prototype/course and your audience provides positive feedback. Be quick to send that feedback to the powers to be along with any evaluation you have done. They will have a hard time arguing against more interactive courses then.
- Continue to evaluate your courses even after you have been given the go ahead and resources to create more interactive courses. If you can demonstrate positive results for all 4 levels of evaluation, especially “results,” they will have little argument for ever implementing a page turner again.
What else can be done to address the organization stuck in page turner mode? Please feel free to make suggestions. Thanks.
I just released the first draft of a new WBT course and as usual I have a slew of people reviewing the course. This includes Subject Matter Experts (SMEs) among others. In the past I have provided a general list of what aspects of the course should be reviewed (e. g. grammar, accuracy of content, navigation, technology, etc.). This time around I compiled a far more detailed list of concerns reviewers should be attentive to during their review. It is meant more as guide to what they should be looking for, but can also be used as a questionnaire.
Here is what I included:
- Check spelling, grammar, and consistency of language.
- Does the course answer your questions/concerns about the subject?
- Do you feel prepared to begin applying the new knowledge/skills learned?
- Does the course meet the objectives presented at the beginning of the course?
- Do you feel you now have a better understanding of the subject at hand?
- Were there any links or buttons that did not work?
- Were all navigational elements marked appropriately?
- Were you able to navigate through the course with ease?
- Do you find the graphics helpful?
- Do the graphics appear properly?
- Was text in the graphics clear and visible?
- Does the animation appear properly?
- Was text in the animation clear and visible?
- Do you find the animation helpful?
- Are the soft skill simulations reflective of realistic scenarios?
- Do the simulations, interactive exercises and/or pop-ups function properly?
- Are the software simulations/demonstrations realistic and appear to reflect the actual “live” system?
- Do the questions measure your understanding of the content presented?
- Are there questions that address content not presented in the course?
- Are the questions/answers accurate and pose no potential exceptions that could make an answer incorrect?
- Is the feedback provided helpful?
- Does the assessment provide correct scoring results?
- Does the audio function properly?
- Do the videos function properly and appear professional?
I am sure as time goes on questions will be added and some will be eliminated. What would you include, eliminate or change on this list? Any input would be great.
Here is a nice, quick overview of Donald Kirkpatrick’s Four Levels of Evaluation provided by Kirkpatrick Partners, LLC. Also a nice addition to the Free e-Learning page.
Update on 1/26/2010 – the slides are no longer available online. Sorry for the inconvenience
Slide 12 have a helpful matrix of tools that can be used for measuring each level.