In my prior post, The Other Stakeholder Wants It Now Too, I had focused on the learners’ need for “on-demand” learning. This month’s Big Question at Learning Circuits is “How do we need to change in what we do in order to address learning/performance needs that are on-demand?”
At this moment I happen to be in a situation where a major software system upgrade is going to occur in a matter of weeks. Staff do not have the luxury of waiting for me to design, develop and implement a thorough set of learning events. Here is one real life example of what is done in an “on demand” situation.
- An immediate sit down with the SMEs to identify the audience and the changes to the system (luckily, I have worked with this audience and system before) and to gain access to the new system.
- Use the new system myself to further identify learning needs and challenges the audience may encounter, while grabbing screenshots and creating job aids for the tasks the audience will need to learn in the upgraded system. If you are a fan of the ADDIE model, this would be like trying one’s best to lump the analysis, design and development phases together into a very, very short period of time… It is not pretty, but it is what happens in an “on-demand” situation.
- Get a blog on the Intranet where I can provide ongoing support, additional learning materials, tips and tricks and a place for staff to share their knowledge of using the new system.
- Communicate details about the upgrade including its features, benefits and motivate learners to master the new system. Oh yeah, get them the job aids they need.
- If time allows, I will build some impromptu simulations in addition to the job aids.
None of this is pretty, but this is what I have had to scratch together to address on-demand learning needs in this particular situation. Luckily, job aids and on-going support via the blog should get the learners up to speed for this upgrade… Wish me luck.
FYI: This post may not directly address the question of how we need to change but rather be an example of what is done in one particular “on-demand” scenario. I will say we cannot completely skip analysis and design phases, but need to be able to think on our feet and do our best to conduct very quick, informal analyses and design on the fly in these situations.
Signs you are in e-learning hell (as a designer/developer):
- The Help Desk calls and tells you they are receiving too many calls regarding your course and the __________ (pop-up blocker, Flash Player and/or LMS sign-on).
- Someone shows up at your office and says “I’m here for the online training.”*
- The stakeholder proposes that all staff take the online course in a computer lab where it can be proctored.*
- People call you and ask, “How long does it take to convert their eight hour classroom training into an e-learning course” and you answer,”I don’t know. How long will it take you to turn my phone into a ’67 Dodge van?”
- Your subject matter expert (SME) tells you the course looks great and has no edits. The LMS report shows the SME did not even launch the course.*
*Yes, this actually happened.
Signs you are in e-learning hell (as a participant):
- The course starts with five pages on how to take an online course.
- The quiz questions have absolutely nothing to do with the course you just took.
- Every answer on the quiz is either “all of the above” or “none of the above.”
- The content is not exactly concise. In fact, it may have been written by Proust.
- You scored 100% on the quiz but are not marked “complete” because you skipped page 42.
- You toss your most snarky remarks at the people pictured in the course, but they still ignore you.
Of course the above is all in good fun and just a way for me to vent some frustrations of working in our field. Thankfully, the above are resolvable issues and are becoming less and less common. Do you have signs that tell you when you are in e-learning hell? Please feel free to share in the comments section. Thanks!
When creating e-learning courses I ask stakeholders to sign-off on the final product. This is only after the review process is complete. Very often the stakeholders are also the subject matter experts for the course. Either way, I find that a sign-off provides confidence for me and my team that the course meets the approval and needs of the stakeholder, the content is accurate, and that we have a commitment from the stakeholders/SMEs for future reviews.
Here is what I include in a typical sign-off form. Granted, this may change depending on the project and any addendum requested by the stakeholder/SME. I keep it as simple and succinct as possible. The following points are included in the agreement.
- That the design and content for the course provides accurate information and complies with the organization’s current policies and procedures.
- To conduct future reviews on a specific scheduled basis to verify its accuracy.
- To inform the course designer(s) of any future content, policy and procedure changes that affect the on-line learning program.
- The course will not be released on the learning management system (LMS) or Intranet without the completion of this form/agreement.
Please keep in mind, I am referring to a sign-off for courses created for internal use. A sign-off for an external client would be approached with greater detail.
Are there additional things you include in a sign-off? Please feel free to share in the comments section.
All too often people reviewing a web-based training (WBT), including subject matter experts (SMEs), request the course printed for them. If it is the absolutely only way they will review it, then I do accommodate them. Keep in mind this is after I have exhausted all other attempts of getting them to do a proper online review.
Here are reasons not to print courses for a review:
- It is important for anyone reviewing a course to not just look at content, but to review the entire learning experience including the delivery medium.
- If they themselves are not willing to participate online how can they expect, or request, our audience to participate.
- Online courses are very often non-linear. Thus, do not fit in a printed, linear format.
- Courses are interactive. They may contain anything from simple rollovers to complex games or simulations. Interactivity does not translate to a printed page.
- Once printed it is occasionally handed around for others to review without the designer’s knowledge. This can result in not being able to identify the origins of edits, if needed. It can also result in draft content mistakenly being distributed to the end user. This can all be prevented by setting appropriate access in an LMS.
- Depending on the authoring tools used, it can be time consuming to print a course. For example, a course that contains many interactive Flash elements will require many screenshots to be taken. Time is better spent on on design and development.
- It is more environmentally friendly to review online. As a fellow e-learning designer said to me recently, “I killed many trees with “WBT to be printed out” for SMEs, higher ups, etc.”
The reality is people reviewing courses are going to push for a printed version and sometimes the only way to get them to review it will be to comply. However, I am not going to comply without at least explaining the importance of an online review. In the end, even if I send them a printed version, or screenshots, I always supply easy access to the online course along with several reminders of how important it is to also review it online.
When I read e-Learning Magazine’s article by Bob Little, Rapid e-Learning Polarizes Opinion, I was very irked by it. Especially when I read the following excerpt.
“While purists sneer that e-learning produced via rapid tools may lack quality in terms of adhering to instructional design principles and may just be brain dumps by subject matters experts, if such e-learning materials improve workers’ performance, who can criticize their place in the learning and development armoury?”
I will say this, if they are not adhering to instructional design principles, then they are far less likely to improve workers’ performance. I have never been a fan of the term rapid e-learning. I believe there are some great rapid development tools, but these still require sound instructional design, which takes time and effort, starting with a needs analysis.
As stated, I am not fond of the term rapid e-Learning, but I do not wish to be negative or come across as bashing efforts made under only good intentions. Here is the reality, not all organizations have the luxary of employing an instructional designer or perhaps enough instructional designers. However, they do wish to offer their staff online training. So, the subject matter experts gets a hold of the new fangled software that says “create e-learning in only a matter of hours” or some such thing. They then do their very best, but because they did not identify the learner’s needs and create an effective course design it falls short and does not result in learning or impact behavior.
So, here is what I suggest.
First off, be willing to dedicate more time to the analysis and design phases of your project. Learn as much as you can about instructional design, more specifically e-learning design. Here are a few ways to get started:
- Follow e-learning blogs (the eLearningLearning blog community and eLearningPulse are chock full of great blogs)
- Attend conferences (there are plenty of great e-learning conferences out there)
- Join e-learning and instructional design societies and groups (the eLearning Guild and ASTD both have plenty of books, articles, research, webinars, conferences and more)
- Network with e-learning designers and ask for advice and reviews of your work (LinkedIn has plenty of e-learning and ID groups and Twitter is perfect for connecting with people in the e-learning world)
The more time you invest in instructional design, the more effective your courses will be and your audience will appreciate it too. And remember, sloooow doooown and spend some time in instructional design!
This month’s Big Question at Learning Circuits is on working with subject matter experts (SME). First off, thank you Tony at Learning Circuits for using my suggestion as this month’s question. In my opinion, working successfully with SMEs is the most challenging part of instructional design. I have written quite a bit about this subject on my blog. Here is a link to my posts with SME tags and my tips on working with SMEs (pasted below from a prior post along with an addendum).
Originally posted November 16, 2008
When working with subject matter experts (SME), you must keep them involved and engaged in every step of the e-learning process. Here are some tips to accomplish this goal.
Lay the foundation
Introducing e-learning to the SME lays the foundation of a successful collaboration. E-learning is still new to many people and many misconceptions exist. Someone not exposed to quality e-learning may think it will be a page turner, which is not good e-learning. Explain what e-learning is, when it is appropriate, and its capabilities. Suggest they look at courses already available. This will provide some reference to what they may expect in a course.
Introduce their role in the process
Inform SMEs of your expectations of them, allowing them to schedule their time and prepare for the tasks ahead. They will not just hand over some form of subject matter and that’s that. They will help identify training needs, learning objectives, etc. Also explain other expectations you have of them as it relates to the different phases of the course’s creation (proofing prototypes, drafts, clarifying subject matter, etc.).
Ask the right questions (Analysis)
Never assume the SME knows the training need. A needs analysis is very important. Reality is that needs analyses are often informally conducted. If a needs analysis has not been conducted by you or anyone else then it will be your job to identify the training need. And your SME will be your first resource.
Share the plan (Design)
Unless your SME has worked with an instructional designer before, a training design plan will be new to them. So, prepare them on how to interpret it and make sure you explain the difference between terminal objectives and enabling objectives. Once you and your SME have identified the appropriate objectives and agree to the design, get their approval. Remember, write your design plan to a non-training audience. You do not want to confuse them with a lot of training jargon.
Put them to work (Development)
Keep your SME updated during the development stage. Development takes a significant amount of time and they may wonder, “Whatever happened to that e-learning designer?” Do not let SMEs forget about the project while you are busy creating it. Let them know of your progress. If possible upload what is available so far and call it a prototype. And get their feedback.
Provide detailed instructions for reviewing drafts. I also provide a checklist. They should not think they are to look at grammar or content only. Be sure they also look at the flow of the course, accuracy of questions, usability, and identify any technical problems. Also request SMEs to provide names of anyone else familiar with the subject well enough to provide useful feedback.
Market the course (Implementation)
You may release the course on your learning management system, but there is more to implementation. It must be marketed. Participation requires strong communication efforts and buy-in from supervisers. Have your SME help communicate the importance of the course. SMEs can help promote the course via e-mails, intranet announcements, employee newsletters, presentations, etc.
Is it effective? (Evaluation)
SMEs are subject matter experts because they apply the knowledge the course will teach, or work closely in some form with those applying it. Thus, they have an inside view of seeing the knowledge or skills applied and have a relationship with those applying it. This will be helpful in connecting you to the people that will provide evaluation data. SMEs can also support the process by selling the importance of working with you in evaluating the course.
In the end, effective collaboration with your SMEs will compliment your project. The key to this collaboration is keeping your SMEs informed, invested, and involved throughout the process. And always give them appropriate credit and a big thank you.
Addendum – September 1, 2009
I will add that recently I started a new position with the bank that acquired my prior bank/employer. One thing I learned during this transition is that when joining a new team of instructional designers it is very important to get a good grasp of how they approach working with SMEs. Although the team works well with SMEs some aspects of my approach created shifts in a process in which the SMEs had become very accustomed. However, using my new team’s approach while introducing aspects of my own worked well, but I made it a point to explain to the SMEs why I was approaching certain tasks differently than they experienced in prior projects. SMEs can become very accustomed to specific ways of working with their instructional designers. So prep your SMEs well, especially if they already have pre-conceptions of the SME-instructional designer relationship. This is not to say one way is wrong or right, but it is what works well for each person and for the needs of the project.
I just read a comment on a blog where a someone was very frustrated by bland, unengaging page turners. It got me thinking. How do you get a an organization out of the rut of making page turners and to start creating more engaging and effective courses? Here are my first thoughts:
- Put on your instructional designer hat and do everything you can to educate all involved (SMEs, clients, managers, and audience too) on what effective e-learning is and how all involved can benefit from it.
- Show all involved what effective e-learning looks like, actual examples. Here is just one place where you can find examples –http://minutebio.com/blog/free-e-learning/ (this my Free e-Learning collection ).
- Find case studies, articles, evaluations, etc. that support your case.
- Create a prototype to demonstrate the level of interactivity and engagement your organization can produce in a course. Get your co-workers involved so they will be vested in the “new approach.” This will earn you supporters and people who can rally against the archaic page turners the organization still wants to produce.
- When you launch your prototype/course and your audience provides positive feedback. Be quick to send that feedback to the powers to be along with any evaluation you have done. They will have a hard time arguing against more interactive courses then.
- Continue to evaluate your courses even after you have been given the go ahead and resources to create more interactive courses. If you can demonstrate positive results for all 4 levels of evaluation, especially “results,” they will have little argument for ever implementing a page turner again.
What else can be done to address the organization stuck in page turner mode? Please feel free to make suggestions. Thanks.
“You put together a committee to build a horse and get a camel.”
That is what I have been going through with my SMEs. I had the right subject matter experts (SMEs), all with the expertise needed, and not too many to be productive or reach a reasonable consensus. Somewhere along the line they took it upon themselves to invite more into the mix and now reaching a concensus is becoming difficult.
The bigger problem is they all want to throw more content into the course. The SMEs want everyone to be subject matter experts too. While the learners need the skills to do to do THEIR job effectively, not to master all the skills they are not expected to ever apply.
Today the SMEs surprised me. They wanted to push the training up 3 weeks due to an earlier than expected system release. This is the first time I have ever experienced an early system release. Usually these are delayed, maybe on time, but never early. Anyhow, what is an e-learning guy to do?
It was an opportunity to make sure we built a horse!
Here is what I did:
- Shaved 1 week from the amount of time participants have to take it prior to the system release. This means I have 2 weeks less development time and participants have 1 less week to take the course. It’s a compromise between the learners/participants and me, and the SMEs still get to keep there training prior to the system launch date. I hate giving participants less time, but it is a necessary evil.
- Revisit the storyboards and determine what can be eliminated from the course that is not essential to participants being able to use the new system. This will accomplish the following:
- Shave a lot of development time off. Especially if it includes eliminating unnecessary simulations. I know for a fact there are numerous topics, demos and sims that are what I call “nice to knows.” It is great to provide background info or more about the system’s inner workings, but for the sake of time on our part and the participants part too, they can use the system effectively without this some of this info. And believe me, my audience will appreciate a succinct course, while still being able to learn exactly what they need in order to use this system.
- Redesigning some of the interactive sims (try me sims) into demos (show me sims) instead. The very simple tasks will transfer and be retained just as well with out the practice, in my opinion. I will still invest the time needed to develop the interactive “try me” sims for the more complicated tasks.
Yes, it is a pain to have less time to develop this course, but at least it can be a more succinct, focused and hopefully a more effective course.
I just released the first draft of a new WBT course and as usual I have a slew of people reviewing the course. This includes Subject Matter Experts (SMEs) among others. In the past I have provided a general list of what aspects of the course should be reviewed (e. g. grammar, accuracy of content, navigation, technology, etc.). This time around I compiled a far more detailed list of concerns reviewers should be attentive to during their review. It is meant more as guide to what they should be looking for, but can also be used as a questionnaire.
Here is what I included:
- Check spelling, grammar, and consistency of language.
- Does the course answer your questions/concerns about the subject?
- Do you feel prepared to begin applying the new knowledge/skills learned?
- Does the course meet the objectives presented at the beginning of the course?
- Do you feel you now have a better understanding of the subject at hand?
- Were there any links or buttons that did not work?
- Were all navigational elements marked appropriately?
- Were you able to navigate through the course with ease?
- Do you find the graphics helpful?
- Do the graphics appear properly?
- Was text in the graphics clear and visible?
- Does the animation appear properly?
- Was text in the animation clear and visible?
- Do you find the animation helpful?
- Are the soft skill simulations reflective of realistic scenarios?
- Do the simulations, interactive exercises and/or pop-ups function properly?
- Are the software simulations/demonstrations realistic and appear to reflect the actual “live” system?
- Do the questions measure your understanding of the content presented?
- Are there questions that address content not presented in the course?
- Are the questions/answers accurate and pose no potential exceptions that could make an answer incorrect?
- Is the feedback provided helpful?
- Does the assessment provide correct scoring results?
- Does the audio function properly?
- Do the videos function properly and appear professional?
I am sure as time goes on questions will be added and some will be eliminated. What would you include, eliminate or change on this list? Any input would be great.