Computer-Supported Collaborative Learning
Development of online distributed training: Practical considerations and lessons learned
September 1, 2009
[The following is a chapter from the book Computer-Supported Collaborative Learning: Best Practices and Principles for Instructors, by Kara L. Orvis and Andrea L.R. Lassiter (eds.). It is reprinted here with permission from the authors.]
"Development of online distributed training: Practical considerations and lessons learned"
Chapter by Eileen B. Entin and Jason Sidman, Aptima, Inc., and Lisa Neal, eLearn Magazine
The value of peer learning is well known, especially for domains in which people will apply what they learn in collaborative settings, but it is challenging to design courses that effectively incorporate and support peer learning. When learners are co-present in the classroom, it is easier to devise exercises that facilitate peer learning. Currently most training outside of the classroom is self-paced, eliminating peer and instructor contact, due to the perceived cost reduction and the greater ease of implementation. It is a challenge to develop online training that incorporates these rich human interactions while respecting the time constraints under which learners operate, yet it can lead to deeper learning that is more memorable and more easily applicable (Neal and Miller, 2005; Notess and Neal, 2006).
This chapter discusses considerations and tradeoffs in designing, developing, and evaluating online training for an instructor and students who are geographically distributed. The discussion will take readers through the development process, starting with analysis of audience and goals, to the challenges in acquiring and adapting course material to the online format, and finally the implementation and evaluation. We focus on considerations involved in supporting active engagement of learners and meaningful and thoughtful learner-learner interactions appropriate to the subject matter (Neal and Miller, 2006).
Training is successful only when there is a demonstrable performance improvement; for that to occur requires opportunities for demonstration, practice, and feedback as well as for declarative learning Cannon-Bowers, Tannenbaum, Salas, and Volpe (1995). Furthermore, training that involves teams, rather than individuals, requires that the practice and feedback is conducted in a team environment. Whereas in the past training has been traditionally delivered in a face-to-face classroom-like environment (Neal and Miller, 2006), our goal was to develop a distributed training program that includes elements of didactic instruction and demonstration as well as the opportunity for practice and feedback in a team-based environment. The training program we developed was designed to train civilian and military emergency medical team members in teamwork skills and in methods for enhancing teamwork; the approaches used are applicable to any training where learners will not be practicing in isolation.
This chapter is written through the lens of our experiences in developing the teamwork training program. The program was developed for medical professionals and in that sense is different from the academic environment where students are still in a learning mode, and have not yet had experience in their chosen profession. Nonetheless, we suggest that many of the points discussed in this chapter are relevant for academic instructors teaching courses that involve the students in practice and application as well as for instructors in a professional training program.
In discussing the issues involved in developing this online training course, we organize the discussion around two major phases of development: research leading to the development of substantive materials and factors considered during the development of the course materials and process. Although the factors are not totally independent of one another, for the purposes of exposition they can be usefully separated.
We note that while our approach is based on both sound theory and substantial experience as well as feedback we received during formative testing of the program, at this writing the specific techniques we implemented in our training program have not been validated by a thorough training evaluation. Therefore, while this chapter can make training developers aware of some of the key considerations that led to our implementation approaches, it cannot offer empirical proof of their effectiveness. Additional research is required for that.
Teamwork Skills Training
Teamwork skills (Sims, Salas, & Burke, 2004) are valuable in many settings, but in emergency medical settings they are crucial to the safety and well-being of patients. Many medical teams are ad hoc and they, in particular, need to rapidly increase team proficiency when brought together by implementing effective leadership, backup behavior, and the other components of teamwork. Teamwork Training and Assessment in a Remote and Networked Environment (T-TRANE) is a teamwork skills training program for emergency medical teams that is delivered using web-enabled collaborative technologies (see Entin et al., 2007 for a more complete description of the program). The program assumes students are skilled in clinical techniques but have minimal formal knowledge of teamwork.
The goals of T-TRANE are to train emergency medical teams, be they ad hoc or cohesive teams, to understand and use teamwork skills. Due to the difficulties of arranging lengthy training for such learners, the focus of the course is not on a theoretical understanding of teamwork skills but on the components of teamwork as applied to emergency medicine. The course is specifically designed to do this through the use of real video footage and authentic textual scenarios culled from emergency medical personnel. Furthermore, the course objective is the immediate application of skills in the workplace, and this is accomplished through questions about and reflection on daily practice.
In order to achieve the course goals, T-TRANE is comprised of information about and examples of teamwork skills and scenario-based training exercises that provide practice in strategies to promote teamwork such as conducting pre-planning and debriefing sessions. The program includes four modules, with both live (synchronous) interactive sessions and self-paced (asynchronous) sessions that students complete within scheduled intervals. The first module provides definitions and examples of five critical teamwork skills: communication, monitoring, back-up, leadership, and team orientation (Sims, Salas, and Burke, 2004). It discusses internal and external threats that may disrupt effective teamwork, and the use of planning and debriefing as strategies for promoting effective teamwork. Modules 2 through 4 reinforce and apply the concepts and strategies discussed in Module 1, and provide opportunities for the students to apply these concepts in their current work. Scenario-based exercises ask the students to identify the teamwork skills that were displayed in a video-based or verbally-described situation and the threats that emerged to disrupt effective teamwork, and show how these threats could be addressed in a debriefing.
The training is led by an instructor who is experienced in the emergency medical domain, but who is not necessarily an expert in teamwork training. An extensive Instructor's Manual supports the instructor in planning and delivering the course. The approach used in this program can be adapted to any domain in which ad hoc teams are formed or when teams need to perform well together.
Pre-development Considerations
Planning any online program requires consideration of many factors, most importantly the target learners and the topic; the constraints that arise due to these factors; and the course objectives that will satisfy stakeholders (Neal and Miller, 2005). In this section we discuss some of the issues that we grappled with in program-planning stage, when certain decisions that would frame the subsequent development needed to be made.
Identifying the Target Learner Population and Other Stakeholders
Many of the decisions about training system development occur before any curriculum is drafted or any courseware is designed. While content and functionality are obviously critical components to any training system, they can only be optimally designed when the training consumer is considered (Neal and Miller, 2005). Consumers of training are not only the trainees themselves, but instructors, administrators, and potentially other people who may purchase, assign, deliver, or study training content. These stakeholders may vary widely with respect to their level of expertise in the training domain, level of familiarity with the training technology, and so on. Therefore, determining who the consumers of the training are, and what unique relevant characteristics they will possess, is a critical first step in training development.
For our training program, we identified three categories of stakeholders: administrators, instructors, and students. All three groups were medical professionals in emergency medical units: department heads (administrators), attendings (instructors), and residents (students). As emergency medical providers, their defining common characteristics were their education level and their extremely busy schedules; and these factors demanded three requirements of our training system. First, we needed to find a way to allow time- and location-independent access to the training. If their schedules suddenly permitted some time for training, then we wanted learners to be able to access the course and use their time productively. Second, the course needed to be efficiently designed, and instructors well-prepared, so that everyone's valuable and limited time was optimally used during training sessions. Finally, and most importantly, course materials needed to be appropriate, relevant, and immediately applicable, capitalizing on existing knowledge and experience.
Identifying Course Objectives
As any course is being developed, it is important to be clear about the course objectives. For the teamwork training course we developed, we had several key objectives, including: enabling distributed (rather than co-located) training; providing opportunities to practice the teamwork concepts being trained; promoting interaction and reflection among the course members; and circumventing the need for a teamwork trainer to deliver the course. All aspects of the program were developed in light of these objectives.
Identifying How Training Is Delivered
Before development of a distributed training program starts, the development team must ascertain what options are available in terms of "where" in space the team will meet - in particular whether it is desirable to use an asynchronous or synchronous method only, or whether both technology options are blended into a solution that is in harmony with user requirements and constraints (Shneiderman, 2002). Once that is decided, the challenge is to use each mode to its best pedagogical advantage (Neal and Miller, 2006). Relevant considerations here, due to the nature of the topic, were the need to present realistic scenarios, to provide opportunities to practice together, and to promote and reinforce training through independent reflection and group discussion (Notess and Neal, 2006).
Synchronous sessions allow for dynamic interchange of ideas and promote social networking since learners have better opportunities to get to know each other (Neal and Miller, 2005). This is especially important for teaching teamwork, where there are limited opportunities for practice through role play or sharing of insights without real-time interaction. Even with the value of synchronous sessions, they can be challenging to incorporate into training since they require all participants to be available at the same time. Participants may be distributed across multiple time zones, further complicating the task of identifying suitable times to meet. In our case, this was further constrained by busy schedules that were often out of the participants control since they could be paged at any time. Furthermore, synchronous technology is more complex to introduce.
Asynchronous learning is the most common way of delivering e-learning (Neal and Miller, 2005). It circumvents the scheduling difficulties because learners can choose a time that is convenient and can work at their own pace. On the other hand, the only interaction is through discussion forums, which are less dynamic, and there is less of a sense of belonging to a class since the class never "meets" (Neal and Ingram, 2001). In addition, because the time commitments are not rigid and harder to reinforce, it is easier for students to lag behind, which creates problem when students are out of sync.
In order to leverage the advantages of both self-paced and group learning, we decided upon a blended solution that would interleave synchronous and asynchronous training modules (Neal and Miller, 2005). For a few hours over the course of the entire training program, the students would be required to participate in synchronous modules. They could complete asynchronous training modules at their convenience within a specified range. Defining a time window in which each asynchronous module had to be completed was necessary because the modules had to be completed in a specified order and the students' responses to exercises in the modules needed to be integrated and distributed to the class members by specified times. Given the busy schedules of our training consumers, we believed that this was an effective compromise between demands on time and the desire for dynamic interchange among students.
Technology Usage
The delivery of the synchronous sessions proved to be a considerable challenge, both logistically and practically. Logistically, coordinating synchronous sessions requires finding a common meeting time, a challenge with varied schedules and time zones, ensuring that all computers have the necessary hardware (speakers, microphones) and software, providing login names and passwords to access the website, and so on. While advances in technology, as evidenced by the growing popularity of remote meetings and e-learning and the sophistication of supporting technology, provide solutions to all of these logistical challenges, whether learners know how to use the latest technology is another matter.
We identified several web-conferencing platforms capable of running synchronous sessions with distributed users. While most people in our target population know how to access the internet and interact with a website, fewer are knowledgeable about and comfortable using web-conferencing tools, such as voice-over-Internet Protocol (VoIP), as a method of communication. Consequently, our pilot test of one web-conferencing platform barely got past introductions. Our untrained group demonstrated a dismal display of what military personnel refer to as "comms discipline." People were unaware of when other people were talking and therefore interrupted communications, of whether what they were saying was being heard by others, and so on. Indeed the technology itself was functioning fine; the people, however, did not know how to use it appropriately and were not being adequately facilitated.
In order to allow consumers to avoid the frustrating and unproductive pilot test experience, we decided to integrate within the training program a structured introductory synchronous session in which people could a) test that they could connect to the web-conferencing platform, b) ensure that their computers contained the necessary hardware and software to view training content and to communicate with others and c), be taught protocols for online communication. Although this required an additional synchronous session (and the time commitment and logistical preparation that goes along with that), the benefit of knowing how to use the technology outweighed the cost of not being able to conduct the sessions at all, or to conduct ones that would not have accomplished their pedagogical objectives.
Instructor Preparation and Support
One of the goals we established for the training program was that the instructor did not have to be a subject matter expert in teamwork, since that would have required bringing in an outside person to deliver the training, for which hospitals may not have the resources . Rather, we felt it was more important that the instructor was experienced in the application domain — in this case emergency medicine — and could therefore apply the training concepts to a range of teamwork situations he or she experienced. Having identified ways to allow anytime, anywhere access to training (and being trained on the tools necessary to conduct the training), we then faced our second dilemma with respect to our training: preparing non-expert instructors to deliver our training content. The defining feature of this training was the focus on teamwork in emergency medicine, not on the delivery of the medicine itself. While emergency medical units are populated with qualified medical instructors, these medical experts are not experts in teamwork. Our challenge, therefore, was to implement measures to allow non-expert instructors to be sufficiently prepared to facilitate discussions of teamwork.
We created several methods to prepare instructors to facilitate the course. First, from a logistical perspective, we needed enough preparation time for instructors to become familiar with the course prior to delivering it. We therefore hard-coded triggers within the training system that would alert instructors to upcoming deadlines well in advance. For example, when an administrator assigns an instructor to a course, an email is automatically sent to the instructor weeks in advance of the class.
While instructors had plenty of advance notice prior to leading sessions, they still required the requisite domain knowledge to truly facilitate sessions. One of the key resources we included within the training system was an Instructor's Manual that included background information on teamwork skills including concepts like communication, monitoring, and back-up. The Instructor's Manual also contained a guide through each of the training modules, highlighting key teaching points along the way.
Additional aids were implemented within the synchronous sessions themselves to provide instructors with timely notices of what to talk about and when. Within the synchronous platform, the instructor could see notes within the training content that the trainees could not see. The notes within the Instructor View might point out a key instance of teamwork from a video clip, or suggest a critical question to ask to get trainees to think about teamwork instead of taskwork. Again, this feature of the training allowed non-expert instructors to confidently and competently facilitate discussion on teamwork.
Despite our best efforts to reduce the workload on an instructor, there were still significant preparation and facilitation responsibilities for the designated instructor. All of these responsibilities, of course, do not include actually providing any feedback to students on their self-paced exercises. This task, as any school teacher or college professor knows quite well, is time-consuming and labor intensive. Given our priority to minimize the time commitment of all of our consumers, we decided to diffuse the responsibility of reviewing student work across the trainees themselves. We included an administrative functionality within the training system by which Instructors could designate trainees to be Peer Leaders for a given training module. Peer Leaders would be responsible for reviewing completed assignments and identifying key themes within them that would promote discussion in subsequent synchronous sessions.
Lessons Learned
In sum, many decisions were made about training development prior to developing any training content. The following lessons learned may be useful for future instructional designers developing distributed training:
- Tailor your training to your consumers: Consumers may be more than the trainees themselves. There may be administrators and instructors as well who will purchase, facilitate, or interact with your training system. These populations need to be identified from the start and their needs considered for training to b successful.
- Blended learning is not just an effective pedagogical approach, but it is a practical compromise between achieving learning objectives and acknowledging real-world time constraints.
- Instructors do not have to be experts: However, the burden is on the training developers to implement measures to allow non-experts to effectively deliver training.
- Students can be more than just students: Diffusing some of the instructor's responsibility to Peer Leaders can alleviate some of the burden on the instructor. Providing students with additional responsibilities may increase their level of commitment to the course and enhance their educational experience since they think more deeply about the training materials.
- Just because technologies exist that enable distributed learning does not mean that people know how to use them: The capabilities of some technology may exceed people's ability to use it. Be sure to allow adequate preparation time to ensure that consumers of all ages, education levels, and technical expertise can be comfortable with and fully exploit the training technology, and so that it does not get in the way of the learning it is supposed to deliver.
Substantive Development Considerations
How course material is designed and delivered is dependent on the learners and topic, as well as on the course objectives. While a decision to offer a blended course, incorporating synchronous and asynchronous interaction, fosters many pedagogical approaches beyond the self-paced "electronic page turner" model, many decisions and some creativity needs to be applied to design a course that meets learners needs and meets course objectives.
In this section we explain some of the considerations and issues that arose as we developed the training program content. As noted previously, the program we developed used a blended approach, but we focus here primarily on the challenges in development of asynchronous training modules, where engagement and student interaction are more difficult to achieve than in synchronous modules.
Materials for exercises
The training program we developed is grounded in the principles of scenario-based learning in which operationally realistic scenarios are used to engage students in actively forming links between classroom and real-world applications of key concepts. Scenarios place students in a particular context and encourage them to think about cause and effect (Neal, Miller, and Perez, 2004). Therefore, our first challenge when developing training materials was to create engaging scenarios upon which the training content would be based.
Our preference was to use real experiences within emergency medicine as the source for examples and training exercises. This was possible because our medical subject matter experts had independently developed a library of video recordings of cases treated at a major medical trauma center. Our plan was to use the video clips both to provide illustrations of teamwork skills and give the students practice in applying teamwork skills by giving them the opportunity to recognize, assess, analyze, and perhaps even role play aspects of teamwork from multiple points of view.
The challenge was in how to select and use the video clips effectively. However, the key dilemma was that the video recordings we drew from were made for different purposes than the training program we were developing. As a result, we found that they were not indexed according to teamwork concepts captured in the videos, so the development team had to spend many hours going through videos to find ones that were relevant for the examples and exercises. A further problem we encountered was in finding clips in which both the video and audio portions were of sufficiently high quality that they can be understood both visually and aurally. While for their original purpose there was not a need for clear audio, for teamwork training, where verbal communication is a central skill, the audio portions were critical. As a result we had to work through the videos that we selected multiple times to insure that we understood what was being said and captured it in a transcript that could be played along with the video. Where that was not possible, we needed to find other alternatives.
An alternative in scenario based training is for the developers to generate their own exercise materials. The advantage of creating original scenarios from scratch is that training developers can tailor them to the specific training objectives. Although it was not possible to create videos of our original scenarios, we found that a scenario could be described effectively in text format. We therefore worked with both civilian and military emergency medical practitioners to develop and integrate several authentic textual scenarios along with the video clips.
Creating Social Presence
Social presence is a variable that may affect learner-instructor interactions in distance education courses and is an important aspect for effective online learning (Volery, 2001). Because of the lack of physical, face-to-face contact in Web-based courses, students may not feel the instructor's presence in the course, and they are less likely to be aware of the presence of other students. The absence of the visual cues that normally exist in the traditional classroom may lead to feelings of isolation or lack of connection with students and instructors in the Web-based environment (Atack & Rankin, 2002; Billings et al., 2001). Getting to know the instructor is more difficult in a Web-based environment because of the absence of the face-to-face interaction and the lack of visual cues. Furthermore, students' perceptions of social presence are significant predictors in students' perception of overall learning (Richardson & Swan, 2001).
As a way of creating social presence at the outset of the course, we provided the ability for both the instructor and the students to enter profiles in which they provided information about themselves related to the course and their professional background. In addition to the text-based information, the instructor and students can upload a picture, which could increment the sense of knowing one's fellow students. Participants are urged to upload their profiles before the course begins, so that everyone has an understanding of who the other course members are. This is important because sharing personal information at the beginning of team formation boosts member satisfaction and communication (Kahai and Cooper, 1999) and increases feelings of social cohesion (Zaccaro and McCoy, 1998).
Engaging Learners
The active engagement of learners in forming connections between existing and newly presented material is known to be an important aspect of facilitating learning, retention, and comprehension, and is supported by a large body of literature (e.g., Mayer, 1997; Soraci et al., 1999; Slamecka & Graf, 1978; Wittrock, 1989). Active involvement of students in the learning process can be ensured by cuing important concepts with open-ended questions, providing multiple examples in different formats, and perhaps most importantly, providing opportunities for discussion which help in actively forming links between classroom and real-world experiences.
The first module in the training program we developed was asynchronous in format and primarily didactic in nature. Especially because it was the first one and set the tone for the rest of the course, we felt is was important to ensure that it was engaging as well as informative. We provided audio clips from high-profile and experienced medical personnel testifying to the importance of effective teamwork. We used pictures and video clips to provide examples of the teamwork skills we were training. While most of the examples were taken from the emergency medical domain, we also included examples from other domains - including, for example, aviation and sports — to illustrate the importance of teamwork skills in a wide range of domains.
But still we were concerned that the module did not require the learner to become actively engaged. We explored a number of available online courses and found that, for the most part, they provided useful lessons in what not to do. The courses we reviewed tended to foster what might be called a 'read and click' mindset, in which the learner passively reads the information that is presented and clicks to move on, without any deep processing of the material. We were concerned that the first module in the training program, which is primarily didactic in nature, could foster this kind of behavior. Yet we wanted to establish at the outset that the training program required active participation.
To clearly establish the interactive nature of the program, we inserted reflection questions early in the first module. These questions required the students to think about the material being presented in terms of their own experiences, and to periodically reformulate their ideas as new concepts were introduced. To insure that students actively engaged with the material, they were required to submit their responses to the instructor and, in addition, to provide a relevant work experience in an online student forum. The requirement to submit their responses to the instructor established accountability; it was made clear that the instructor would be reading the students' responses and would discuss them with the group in the next module, which was conducted synchronously. The requirement to enter their experiences in the forum and then to read the other students' entries reinforced the reflective nature of the training; in addition it set the expectations for learner-to-learner interaction.
Promoting Learner-Learner Interaction
In learner-learner interaction, students help themselves to learn, by sharing ideas and discussing problems, often in a real or virtual group setting (Moore, 1996). LeBaron and Miller (2004) describe the successful infusion of various instructional techniques, including the use of discussion forums and role plays, implemented within an online graduate level course, to provide for the social construction of peer knowledge and facilitate an overall sense of course community. Further, when an online course facilitates the structured discussions that enrich and provide context to learning, learning is more enjoyable and more social (Neal, 2002). Orvis and Lassiter (2006) recommend that instructors focus on learner-learner interactions as early as possible in a course.
In the synchronous sessions the instructor can facilitate learner-learner interactions. Our challenge was how, in a distributed, asynchronous environment, to create interactions among the students in a timely and effective manner. One solution was to use a process whereby each student's responses to the exercise questions were submitted to one of the students in the training program who was assigned by the instructor to be the peer leader for a particular module. The peer leader was responsible for reviewing all the students' responses, summarizing them, and providing comments to all the trainees. This mechanism served several purposes. It motivated the students to provide thoughtful responses; it engaged the peer leader in the training; and it provided an efficient way that the students could share the other students' perspectives. In addition, the use of the peer leader shifted a task away from the instructor.
A second mechanism for encouraging learner-learner interaction was a "forum" where students could post ideas and comments and respond to other students' postings (Neal and Miller, 2005). At the end of each asynchronous module, the student was asked to think about a situation in his or her experience where issues relevant to the concepts and techniques trained in that module had arisen, and to describe that situation in a posting to the forum. The students were then encouraged to read the other students' postings and respond to them. These ideas were available to all the students who could then comment upon them, in a "chat" dialogue about a particular topic. Although the text-based forum does not have the spontaneity of a live on-line session, research suggests that online asynchronous discussions possess the ability to invite a higher level of learner-learner interaction than occurs in face-to-face settings (Chih, 2004).
In addition to its use in discussing the exercises, the discussion forum was available for student discussion on any topic. This opportunity is particularly useful for student teams that will be working in field settings, where it is useful, and even sometimes necessary, to know about team members' skills, interests, and concerns that fall outside the medical domain. For example, in a field setting, it may be important to know who is able to drive an ambulance or who has had experience with specialized field equipment.
Together these mechanisms allowed students to respond to questions, reflect on what they were learning, and exchange relevant experiences and ideas with peers. The fact that the postings were public served as a motivator to engage in the conversation. That students were describing real experiences generated "deep" responses from their peers based on how the description resonated with their own experiences (Notess and Neal, 2006).
Lessons Learned
In sum, many decisions were made to achieve training that engages learners and promotes learner-learner interaction. The following lessons learned may be useful for instructional designers developing distributed training:
- Involving subject matter experts as a resource eliminates some of the burden of developing training materials. The developers' role becomes tailoring the materials to meet the specific training objectives.
- Engage the student immediately. Set the precedent that active learning will take place at the very first training module.
- Learner-learner interaction can be synchronous or asynchronous. Do not be of the mindset that interaction has to be synchronous. Asynchronous interaction can be achieved through the use of email and discussion forums, and can set the stage for future synchronous interactions.
Conclusions
In this chapter we have discussed some of the considerations and challenges that arose as we planned and developed a teamwork training course, how we resolved them, and our lessons learned. Any course has unique requirements for which a unique solution must be crafted to provide an optimal solution to training on a given topic for a group of learners. As we have described, in our case, decisions about delivery methods and approach were based not only on existing theory, but also on the abilities and constraints of the targeted learner population and the other stakeholders, and the requirements imposed by the topic. Furthermore, we needed to be realistic about the opportunities provided and limitations imposed by the technology in order to insure that the selected technology served to facilitate learning rather than impede it. Given these considerations, we tried to be creative in developing a training program that met our course objectives, engaged learners, and led students to immediately apply what they learned in their professional practice. While our recommendations are based on research, theory, and experience, future research is necessary to empirically validate our approach.
References
Atack, L., & Rankin, J. (2002). A descriptive study of registered nurses' experiences with web-based learning. Journal of Advanced Nursing, 40, 457-465.
Billings, D. M., Connors, H. R., & Skiba, D. J. (2001). Benchmarking best practices in Web-based nursing courses. Advances in Nursing Science, 23, 41-52.
Cannon-Bowers, J.A., Tannenbaum, S. I., Salas, E., & Volpe, C.E. (1995). Defining team competencies and establishing team training requirements. In R. Guzzo & E. Salas (Eds.), Team effectiveness and decision making in organizations (pp. 330-380). San Francisco: Jossey-Bass.
Chih (2004). Research in Interaction and Durations of Online Asynchronous Discussions. Presentation at American Educational Research Association Meeting, April 2004.
Entin, E. B., Lai, F., Sidman, J., Mizrahi, G., Stewart, B., Neal, L., Mackenzie, C., & Xiao, Y., (2007). A web-based teamwork skills training program for emergency medical teams. In Westwood, J. et al., Medicine meets virtual reality 15: Amsterdam: IOS Press.
Kahai, S. & Cooper, R. L. (1999). The effect of computer mediated communication on agreement and acceptance. Journal of Management Information Systems, 16, 165-188.
LeBaron, J. & Miller D. (2004). The Teacher as Agent Provocateur: Strategies to Promote Community in Online Course Settings. The Teacher as Agent Provocateur: Strategies to Promote Community in Online Course Settings. In T. Latomaa, J. Pohjonen, J. Pulkkinen, & M. Ruotsalainen (Eds.), eReflections - Ten years of educational technology studies at the University of Oulu. Essays contributed by the network builders, pp 109-125 Oulun. Retrieved January 19, 2005 from http://herkules.oulu.fi/isbn9514276329/.
Mayer, R.E. (1997). Multimedia Learning: Are We Asking the Right Questions? Educational Psychologist, 32, 1-19.
Moore, M. G. & Kearsley, G. (1996). Distance education: A systems view. Belmont: Wadsworth Publishing Company.
Neal, L. (2002) Talk to Me: Discussion is the Key to Engaging Online Courses. eLearn Magazine. Retrieved March 7, 2005 from http://elearnmag.org/subpage.cfm?section=opinion&article=48-1 .
Neal, L. & Ingram, D. (2001) Asynchronous distance learning for corporate education. In 2001/2002 ASTD Distance Learning Yearbook, K. Mantyla and J. Woods (editors). McGraw-Hill.
Neal, L., & Miller, D. (2005) Distance Education. In R. Proctor and K. Vu (Eds.), The Handbook of Human Factors in Web Design, New Jersey and London: Lawrence Erlbaum Associates, p. 454-470.
Neal, L., & Miller, D. (2006) The Use of Technology in Education. In O'Neil, H. F. and Perez, R., (editors), Web-Based Learning: Theory, Research, and Practice, Lawrence Erlbaum Associates.
Neal, L., Miller, D., & Perez, R. (2004) Online Learning and Fun. eLearn Magazine. Retrieved March 7, 2005 from http://elearnmag.org/subpage.cfm?section=articles&article=4-1 .
Notess, M & Neal, L (2006) Deep Thoughts: Do Mandatory Online Learning Activities Help Students Leave Surface Learning Behind? eLearn Magazine. Retrieved February 19, 2006 fromhttp://www.elearnmag.org/subpage.cfm?section=opinion&article=75-1 .
Orvis, K.L. & Lassiter,A.L.R. (2006) Computer supported collaborative learning: The role of the Instructor. In Ferris, S. P. & Godar, S. H. Teaching and Learning with Virtual Teams. Hersey, PA: The Idea Group.
Richardson, J. C., & Swan, K. (2001). The role of social presence in online courses: How does it relate to students' perceived learning and satisfaction? World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2001(1), 1545-1546.
Shneiderman, B. (2002), Leonardo's Laptop: Human Needs and the New Computing Technologies, Boston (MA), MIT Press.
Sims, D. E., Salas, E., & Burke, C. S. (2004). Is there a big five in teamwork? 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
Slamecka, N.J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning & Memory, 4, 592-604.
Soraci, S.A, Carlin, M.T, Chechile, R.A, Franks, J.J, Wills, T., & Watanabe, T. (1999). Encoding Variability and Cuing in Generative Processing. Journal of Memory & Language, 41, 541-559.
Volery, T. (2001). Online education: An exploratory study into success factors. Journal of Educational Computing Research, 24, 77-92.
Wittrock, M.C. (1989). Generative Processes of Comprehension. Educational Psychologist, 24, 345-376.
Zaccaro, S. J. & McCoy, M. C. (1988). The effects of task and interpersonal cohesiveness on performance of a disjunctive group task. Journal of Applied Social Psychology, 18, 837-851.
