Quality Matters: Research and Standards Support

II. LEARNING OBJECTIVES (COMPETENCIES)

Literature Support for the General Review Standard:

• The value of learning objectives in describing measurable outcomes has long-standing support in the literature (e.g., Bloom, 1956; Mager, 1962, 1975). As the QM Rubric annotations note, "measurable learning objectives help teachers precisely describe what students are to gain from instruction" and "accurately assess student accomplishment." Because they are an outgrowth of behaviorist learning theory, learning objectives have been criticized as being excessively focused on behavioral outcomes and inadequate for specifying affective or cognitive outcomes. For this reason, the QM Rubric annotations also direct reviewers to "check for clear indications that the learning objective is meaningfully assessed" for cognitive or affective outcomes.
• The importance of developing critical thinking skills has wide acceptance in the field. It is deemed particularly important in adult education to the point of being considered a template for practice (Brookfield, 1997) but is also commonly stated as an institutional policy goal for many undergraduate and graduate programs. Garrison, Anderson, and Archer (2001) suggest that appropriate teaching and social presence in a computer-conference environment can create and support "cognitive presence" (i.e., critical, practical inquiry). Bruning (2005) describes one of many possible techniques for incorporating critical thinking skills into an online course.
• Gunawardena, Lowe, & Anderson (1998) identified phases of knowledge construction: including, cognitive activity, arguments, resources that explore arguments, supporting literature, data and dissenting ideas, and evidence of changes in understanding.
• Achtemeier, Morris & Finnegan (2003) found consensus among more than thirteen best practices instruments and the accompanying literature review that the text-based questions in online education should be worded clearly, simply, logical, not biased or leading, and each should stand-alone and address only one issue.
• Phipps & Merisotis (2000) include analysis, synthesis, and evaluation activities as key activities that should be part of course requirements.
• Murphy, Mahoney, & Harvell (2000) found that students without clear direction can be easily confused about their obligations. Group goals especially require clear and achievable objectives.
• Spallek, Berthold, Shanley, & Attstrom (2000) surveyed dental professionals on quality assurance criteria for online courses. The importance of courses being scientifically based, regularly updated, easy to navigate, have defined educational objectives, stimulate learning, be created by appropriately qualified academics, and be at an appropriate level for the intended participants were deemed important.
• Shea et al. (2002) also found that students in online courses who reported that communicating clear expectations on how to succeed in the course correlated highly with levels of satisfaction and perceived learning.
• Gunawardena, Lowe, & Anderson (1998) identified phases of knowledge construction: including, cognitive activity, arguments, resources that explore arguments, supporting literature, data and dissenting ideas, and evidence of changes in understanding.
• Sims, Dobbs, & Hand (2002) pointed out that "unit or course content should be considered in terms of the interaction with major design issues" (p. 138).
• Pawan, Paulus, Yalcin, & Chang (2003) pointed out that "it is important to first provide clear participant requirements in terms of length, content expectations, and timeliness" (cited in Garrison & Cleveland, 2005, p. 145).
• Lux & Davidson (2003) identified within the scientific literature best practices strategies for a "successful CBI [computer-based instruction] modules should contain, i.e., modules that are received favorably by students and which result in a demonstrated higher level of learning" (p. 125). They summarized the guiding principles for module design to include: Goal for users (teach a skill set); presentation (simple, clear straightforward); architecture (default path for novices, high degree of learner control for advanced users); navigation (direct access to all content); interface (intuitive with high level of interactivity); and design method (iterative, solicit feedback from both professionals and end-users) (p.127).
• Trigano & Pacurar-Giacomini (2004), building on the literature in human machine-interface, web ergonomic, pedagogical structure, and pedagogical environment, noted that "because of the non-linearity of the information representation in a digital support...(sequence)...the information in hypermedia should be based on information-units (emphasis in original) corresponding to computation-units...[so that the reader/learner will be able to identify] which is necessary and sufficient in order to understand a concept" (pp. 22-23).
• Stewart, Hong, & Strudler (2004) in preparing for quality evaluation of online courses highlighted Driscoll's 1998 and Khan's 1997 work on web-based instruction - "attention to education details (e.g., clear guidance and direction for each lesson, clear objectives, adequate practice, and meaningful feedback)" (p. 133) was among those considered important.
• Koszalka & Ganesan (2004) considered information, instruction, and learning design elements as identified in the web-based online learning literature and applied those to a course which was "initially a failure" (p. 243). Issues addressed were confusion of the learner when "haphazard integration" of CMS features "did not match course objectives"(Oliver, 1999; Kearsley, 1997; Collis, 1999; Grabowski & Small, 1997 were cited); "practice components were often weak or missing (Gilbert & Moore, 1998; Kidney & Puckett, 2003 were cited) (pp. 244-245) ; activities and resources did not closely match instructional purposes (Kidney & Puckett, 2003; Koszalka & Bianco, 2001; Simonson et al., 2003 cited); and "learners did not see a connection between the activities they were completing" and the overall objective (p. 251).

Literature Support for Specific Review Standards:

III. ASSESSMENT AND MEASUREMENT

Literature Support for the General Review Standard:

• Eanes (2001) provided a "task-oriented question construction wheel based on Bloom's taxonomy" what that provides support for well constructed online assessment.
• McLoughlin (2001) suggested that evaluation tasks be associated with both learning outcomes and teaching approaches in order for the numerous characteristics of pedagogy to be supported in a cross-cultural setting.
• Wisher, Curnow, & Seidel (2001) looked at knowledge retention in two distance learning course sections for the military and found it comparable with other classroom training. They conclude that distance education offers the potential of improving knowledge retention if frequent testing and spaced practice are incorporated.
• Youngblood, Trede, & DeCorpo (2001) identified a number of essential tasks for an effective teacher: make student welcome, clarify expectations for contributing online, clarify grading for the online participation, monitor participation in online discussion, keep discussion on track, contact students offline, bring closure to discussion, use questions to stimulate discussion, move discussion forward, stimulate reflection on students' comments, encourage students to build on others' contributions, and divide students into groups for specific tasks. Findings revealed that students felt clarification of grading and of expectations were most important.
• Tuzum, H. (n.d.) cited Friedrich & Armer (1999); Kuchinke, Aragon, & Bartlett (2001) and others regarding assessments in WBI (Web based instruction). He pointed out that assessments in WBI "should be different from traditional classroom techniques and include "authentic tasks as written reports" and should be "performance-based mode and should include fewer objective tests" (p. 364)
• McVayLynch (2002) noted the "dualistic debate over whether assessment should focus on accountability or improvement," and suggested that a misconnection in evaluation can still occur "because teachers or course designers fail to create a direct relationship between instructional objectives and assessment measures" (117-118).
• Macdonald & Twining (2002) looked at the relationship between assessment, student participation, and the development of skills. They suggestedconcluded with key issues for assessment of activity-based learning: assessment must reflect course philosophy, assessment is essential in creating learning opportunities at critical points, assessment provides a vital opportunity for feedback, helping to complete the reflective learning cycle.
• Lewis' (2002) dissertation found the importance of "in-course practice exercises are aligned with the final evaluation tests in terms of both the content and the style" (abstract). LewisShe (2002) looked at the efficacy of quizzes vs. discussions in online learning in regard to the impact on learning and. She concluded that students do better in the final assessment tests if in-course practice exercises aligned with content and style, that intensity and frequency of a student's participation in group discussions also seems to have a possible influence on final performance.
• Thurmond et al. (2002) found that the when students' belief believe that their learning was being assessed in a variety of ways and that they were receiving timely feedback were among the strongest predictors of student satisfaction. The Annotations for Standard III.3 direct reviewers to look for evidence that students "receive frequent, meaningful, and rapid feedback" lists a variety of examples of how such feedback can be provided. This study supports the view that the online environment influences students' satisfaction rather than being solely a function of student characteristics.
• Hannafin et al. (2003) noted that "Assessment practices have largely been mapped over from traditional teaching-learning approaches, but may not provide either suitable evidence of student learning or may simply emphasize those aspects of learning that are easy to assess"
• Thurmond et al. (2002) found that the students' belief that their learning was being assessed in a variety of ways and that they were receiving timely feedback were among the strongest predictors of student satisfaction. The Annotations for Standard III.3 direct reviewers to look for evidence that students "receive frequent, meaningful, and rapid feedback" lists a variety of examples of how such feedback can be provided. This study supports the view that the online environment influences students' satisfaction rather than being solely a function of student characteristics.
• Shea et al. (2002) found that students in online courses who reported the highest levels of prompt, high quality, and constructive feedback from their instructor also reported the highest levels of satisfaction and perceived learning. Shea et al. also found that students in online courses who reported that communicating clear expectations on how to succeed in the course correlated highly with levels of satisfaction and perceived learning.
• Youngblood, Trede, & DeCorpo (2001) identified a number of essential tasks for an effective teacher make student welcome, clarify expectations for contributing online, clarify grading f the online participation, monitor participation in online discussion, keep discussion on track, contact students offline, bring closure to discussion, use questions to stimulate discussion, move discussion forward, stimulate reflection on students' comments, encourage students to build on others' contributions, and divide students into groups for specific tasks. Findings revealed that students felt clarification of grading and of expectations were most important.
• Jensen, Self, & Rhymer (2002) tested for improved effectiveness of multimedia instructional modules in a basic engineering course at the US Air Force Academy. They discovered the importance of keeping the content in a module focused on the objectives (no extraneous material) and "emphasizing that the concepts will be tested on exams" (" (¶30).
• Hannon et al. (2002) used Gagne's & Laurillard's instructional models to design and evaluate five courses in an online curriculum. Gagne's model evolves from cognitive theory with an emphasis on "enhancing learner performance"; while Laurillard's model emphasizes teacher-learner interaction. Hannon, et al noted, "In contrast with Gagne, Laurillard's model of instruction was designed with interactive technology in mind" (¶3). See http://www.irrodl.org/content/v3.2/hannon7.png for table of Gagne & Laurillard instructional events. Students were generally satisfied with the online course, but students' perceptions of the teacher-learner relationship appeared to support the Laurillard approach to design.
• Hannafin et al. (2003) noted that "Assessment practices have largely been mapped over from traditional teaching-learning approaches, but may not provide either suitable evidence of student learning or may simply emphasize those aspects of learning that are easy to assess"
• Achtemeier, Morris & Finnegan (2003) found consensus among more than thirteen best practices instruments and the accompanying literature review that the text-based questions in online education should be worded clearly, simply, logical, not biased or leading, and each should stand-alone and address only one issue.
• Koszalka & Ganesan (2004) considered information, instruction, and learning design elements as identified in the web-based online learning literature and applied those to a course which was "initially a failure" (p. 243). Issues addressed were confusion of the learner when "haphazard integration" of CMS features "did not match course objectives"(Oliver, 1999; Kearsley, 1997; Collis, 1999; Grabowski & Small, 1997 were cited); "practice components were often weak or missing (Gilbert & Moore, 1998; Kidney & Puckett, 2003 were cited) (pp. 244-245) ; activities and resources did closely match instructional purposes (Kidney & Puckett, 2003; Koszalka & Bianco, 2001; Simonson et al., 2003 cited); and "learners did not see a connection between the activities they were completing" and the overall objective (p. 251).

Literature Support for Specific Review Standards:

Comments

IV. LEARNING RESOURCES AND MATERIALS

Literature Support for the General Review Standard:

Course content is more than just an aggregation of learning resources and materials; the design of course content is an essential aspect of courses, particularly online courses.  This is an issue because, as Kanuka, Collett & Caswell (2002) found, some "experienced distance education instructors tend not to design their courses with a great deal of flexibility - even though they acknowledge that Internet communication technologies can support it" (p. 166).   However, the literature provides conceptual support for the importance of this standard as well as some studies which illustrate successful examples of practice:

• Liu & Ginther (1999) suggested that designers should base content on a diversity of learning styles; teachers should match course materials and assessments to those same learning styles. For example, visual images and diagrams should be used for analytical, visual learners; cooperative activities for field dependent learners, and individual projects for field independent learners.
• Oliver (1999/2002) cited content as one of three critical elements teachers and designers need to consider when designing online courses.
• Swan, Shea, Fredericksen, Pickett, & Pelz (2000) investigated the relationship between student perceptions and course design factors and found that consistency in course design was one of the key factors.
• Spallek, Berthold, Shanley, & Attstrom (2000) surveyed dental professionals on quality assurance criteria for online courses. The importance of courses being scientifically based, regularly updated, easy to navigate, have defined educational objectives, stimulate learning, be created by appropriately qualified academics, and be at an appropriate level for the intended participants were deemed important.
• In one well-accepted conceptual framework (Rourke et al., 2001; Garrison, Anderson, & Archer, 2001) learner-content interaction is one of three principal types of interaction, corresponding to one of the overlapping elements ("cognitive presence") which need to be considered in designing an online course focused on critical inquiry.
• Sonwalkar (2001) describes "pedagogically driven design principles for online education." Among those he suggests that course content is seen as "a set of simulations that support discovery-based learning"; taking advantage of "media assets" to allow for "learner's cognitive preferences"; consider "technological constraints" (¶22-25) of user bandwidth. Additionally, he alerts the important role of experts in content, as well as instructional and web design and the necessity of obtaining copyright permissions.
• Conrad (2002) found that learners judge instructors based on how clearly and completely online course materials present the details of the course. A well-organized course with a clear overview and introduction including a clear statement of expectations, explanation of the course outline, clear timelines, and well-written course notes helped learners feel that they were getting off to a good start in their course.
• Perrin & Mayhew (2002) analyzed data, including learning outcomes from two online distance courses and concluded that online educators need to know more than course content, design issues are crucial.
• Nicol, Littlejohn, and Grierson (2005) found that well-structured learning resources facilitated team collaboration and the learning of engineering design. Students in the course were positive about the value of a well-structured (accessible, easy to use) shared workspace to support their collaborative learning projects in engineering design courses.
• Swan, Shea, Fredericksen, Pickett, & Pelz (2000) investigated the relationship between student perceptions and course design factors and found that consistency in course design was one of the key factors.
• Oliver (1999/2002) cited content as one of three critical elements teachers and designers need to consider when designing online courses.
• Nakos, Deis, and Jourdan (2002) found that "some of the important issues from the students' point of view for the success of an on-line class are the availability of class notes, study guides, interaction with the instructor, ability and ease of asking questions, and ability to get assistance when needed." (However, only 77% of survey respondents (n=121) had actually taken an online course, which dilutes the usefulness of the study's findings.)
• Liu & Ginther (1999) suggested that designers should base content on a diversity of learning styles; teachers should match course materials and assessments to those same learning styles. For example, visual images and diagrams should be used for analytical, visual learners; cooperative activities for field dependent learners, and individual projects for field independent learners.
• Sims, Dobbs, & Hand (2002) pointed out that "unit or course content should be considered in terms of the interaction with major design issues and their impact on the learning community" (p. 138); that the learning design "requires careful thinking about the learner and the options provided for interacting with the content and their learning partners" (p. 140).
• Shirathuddin, Hassan, & Landoni (2003) based on a critical reviewed of four well-known usability design guidelines identified seven factors of usability that should be considered in preparing content for electronically delivery in higher education courses: useful and relevant content with depth and breadth was one of the factors. They pointed out "unnecessary media should be avoided to present long downloading time:" (p. 119).
• Koszalka & Ganesan (2004) considered information, instruction, and learning design elements as identified in the web-based online learning literature and applied those to a course which was "initially a failure" (p. 243). Issues addressed were confusion of the learner when "haphazard integration" of CMS features "did not match course objectives"(Oliver, 1999; Kearsley, 1997; Collis, 1999; Grabowski & Small, 1997 were cited); "practice components were often weak or missing (Gilbert & Moore, 1998; Kidney & Puckett, 2003 were cited) (pp. 244-245) ; activities and resources did not closely match instructional purposes (Kidney & Puckett, 2003; Koszalka & Bianco, 2001; Simonson et al., 2003 cited); and "learners did not see a connection between the activities they were completing" and the overall objective (p. 251).
• Garrison & Cleveland-Innes (2005) found "defining clear expectations and selecting manageable content, structuring appropriate activities (collaborative and individual), and conducting assessment congruent with intended goals" foster a deep approach to learning (p. 145). Reconfirmed that, "The goal in deep learning is to move discussion from exploration to integration and then to resolution (Garrison & Anderson, 2003)" (p.145).
• Nicol, Littlejohn, & Grierson (2005) found that well-structured learning resources facilitated team collaboration and the learning of engineering design. Students in the course were positive about the value of a well-structured (accessible, easy to use) shared workspace to support their collaborative learning projects in engineering design courses.
• Zhang (2005) noted that simply converting "paper-based multimedmultimedia instruction into its digital equivalent and making it available on the Internet will not lead to effective learning due to a lack of an appropriate mix of content richness, interaction, and engagement" (p. 160)

Winegar (2000) explored the impact of those "lessons learned" [from awareness training of Chickering & Ehrmann; Graham, Cagiltay, Lim, Craner & Duffy; Chickering & Gamson] and discovered that while teachers agree in principles but few actually made changes in their practice.

Williams (2000) identified thirteen roles and thirty general competencies necessary for quality distance education in higher education. Instructional competencies were found central to all roles including student support roles. Interpersonal and communication skills dominated the top of the general competencies.

Standard IV.5 is not a research-based standard, but rather a practical one designed to address the issues of intellectual property and copyright.

Literature Support for Specific Review Standards:

Comments: *[re class notes, study guides, but see above disclaimer]
 

V. LEARNER INTERACTION

Literature Support for the General Review Standard:

The QM rubric reflects the research findings that several types of learner interaction are important:

• Moore (1989) identified three types of interaction - student/teacher; student/student; and student/content.
• Anderson (2002, 2003) suggested an expansion of interaction possibilities to include teacher/teacher; teacher/content; content/content.
• Others, coming from an instructional technology perspective, suggested learner/interface interaction (Hillman, Willis, & Gunawardena; 1994) and learner/the virtual world interaction should be considered (Chase, Macfadyen, Reeder, & Roche; 2002).

Because of the intense interest in the role of interpersonal interaction in online learning, there is a substantial amount of conceptual and empirical research supporting this review standard:

• Gunawardena & Zittle (1997) identified social presence ["The degree to which a person is perceived as a ‘real person' in mediated communication" (p.9)] and reported correlation between perceptions of interaction and quality/quantity of learning.
• Vrasidas & McIsaac (1999) found interaction in an online course was related to structure of the course, class size, feedback, and prior experience with computer-mediated communication interaction. Socially constructed meanings from the students' perspectives were emphasized.
• Fredericksen et al. (2000) found that instructor-student interaction was the most significant contributor to perceived learning.
• Hislop (2000) reported that a substantial proportion of working professionals in a graduate degree program felt that they interacted more with their instructor in their online courses and that these learners achieved comparable grades to those in traditional classroom courses, even after controlling for variance by instructor.
• Gunawardena & Zittle (1997) identified social presence ["The degree to which a person is perceived as a ‘real person' in mediated communication" (p.9)] and reported correlation between perceptions of interaction and quality/quantity of learning.
• Anderson, Garrison, & Archer (2001) noted that social presence (the ability of learners to project themselves socially and affectively into a community) was shown to strongly predict learner satisfaction in online education and challenges the widely held assumption that face-to-face, non-verbal behaviors are necessary for establishing student and teacher rapport. Social presence was found to support cognitive presence [critical thinking].
• Sims (1999) listed functions of interactions: allowing learners control, facilitating program adaptation based on learner input, allowing various forms of participation and communication, and aiding the development of meaningful learning
• McIsaac, Blocher, Mahes, & Vrasidas (1999) found that teacher in online course appeared to be more concerned about encouraging student participation than in traditional, classroom based courses. Instructors perceived more effectiveness of their personal interactions to individual student messages in enhancing learning.
• Hislop (2000) reported that a substantial proportion of working professionals in a graduate degree program felt that they interacted more with their instructor in their online courses and that these learners achieved comparable grades to those in traditional classroom courses, even after controlling for variance by instructor.
• Roblyer & Ekhaml (2000); Roblyer & Wiencke (2003) present a rubric to determine the interaction levels in an online course. The model is based in learning theories, instructional theories, instructional design models, and instructional delivery systems.
• Berge (1999), Liaw & Huang (2000), Weller (1988) pointed out the necessity of closing the communication loop.
• Murphy, Machoney, & Harvell (2000) investigated the use of contract in promoting learner-centered project-based learning in a web course. They concluded with a suggested list of instructor responsibilities to facilitate this form of active learning.
• Muirhead (2000) provided six types of interactive activities that might encourage learner-learner interaction: sharing of relevant personal experiences, reference to appropriate materials (besides assigned readings), comments on the opinions of others, introduction of new issues for discussion, questions posed to the group by students, instructor acting as guide and facilitator.
• Jung, Choi, Lim, & Leem (2002) investigated the effects of different types of interaction (academic, collaborative, and social) on learning achievement, satisfaction and participation in web-based instruction. Results indicated that the social interaction group outperformed the other groups; the collaborative interaction group expressed the highest level of satisfaction with their learning experiences; the collaborative and social interaction groups participated more actively in posting their opinions that the academic interaction group.
• Tu & McIsaac (2002) revealed that social presence is complicated by the social context of the learners' and learning environment, by the nature of online communication [including keyboarding and language skills], and interactivity [for example, response time to messages, communication styles, group size].
• Anderson (2003, 2002) pointed out that increased learner control, ability to tailor courses to learners' needs based on their input, as well as opportunities for meaningful collaboration among learners is a result of increased interactive possibilities in online education.
• Belanish, Wisher, and Orvis (2004) found that use of a collaborative question-generation tool produced a 7% increase in the comprehension of course material.
• Vrasidas & McIsaac (1999) found interaction in an online course was related to structure of the course, class size, feedback, and prior experience with computer-mediated communication interaction. Socially constructed meanings from the students' perspectives were emphasized.
• Vandergrift (2002) identified the concept of a faculty member's ‘restrained presence' in an effort to facilitate students' personal responsibility for their own learning and for community building in an online learning environment." Restrained presence of the instructor might encourage self-directed learning.

There is significant support in the research literature that correlates instructor-student interaction with perceived student learning and satisfaction, for example:

• Swan (2001) cited "an instructor that interacts frequently and productively with students" as one of three factors which "contribute significantly to the success of asynchronous online courses."
• Studies conducted by the SUNY Learning Network have repeatedly found that students' positive perception of interaction with their instructors correlates strongly with perceived learning and satisfaction (Fredericksen et al., 2000; Shea et al., 2001; Shea et al., 2002).
• Richardson and Swan (2000) also reported a significant correlation between students' perceived learning and satisfaction levels with their instructors, while Jiang and Ting (2001) also found correlations between perceived learning and perceived interactions with instructors.
• Hannon et al. (2002) used Gagne's & Laurillard's instructional models to design and evaluate five courses in an online curriculum. Gagne's model evolves from cognitive theory with an emphasis on "enhancing learner performance"; while Laurillard's model emphasizes teacher-learner interaction. Hannon, et al. noted, "In contrast with Gagne, Laurillard's model of instruction was designed with interactive technology in mind" (¶3). See http://www.irrodl.org/content/v3.2/hannon7.png for table of Gagne & Laurillard instructional events. Students were generally satisfied with the online course, but students' perceptions of the teacher-learner relationship appeared to support the Laurillard approach to design.
• Chang (2003) identified five types of online facilitation by using content analysis of students' threaded messages: (1) assignments and grades (for example, questions about due dates, instructor's expectation, grading criteria), (2) network access (for example, questions about online course materials in the course site), (3) online discussion (for example, questions about clarification, reflections), (4) group activities (questions about those activities), and (5) other course materials access (such as textbook, study guides, the university, and local libraries) (pp. 496-497).--

There is also significant support in the research literature that requiring student-student interaction can result in improved perceived student learning and satisfaction, for example:

Beaudin's (1999) survey from 135 online instructors regarding how to keep asynchronous online discussions on topic revealed (1) questions must be carefully design, (2) guidelines must be provided to students, (3) questions might need "reworded when responses are going in the wrong direction," and (4) regularly provided summaries of the discussions by the instructor (p.41).

Swan (2001) cited "a constructive and dynamic discussion between students and their peers" as one of three factors which "contribute significantly to the success of asynchronous online courses."

Wu and Hiltz (2004) found that required asynchronous online discussions improved students' perceived learning.

The SUNY Learning Network studies also have repeatedly found that students in online courses who reported the highest levels of interaction with their coursemates also reported the highest levels of satisfaction and perceived learning (Fredericksen et al., 2000; Shea et al., 2001; Shea et al., 2002).

Jeong (2003) took on the challenge of quantifying "how message sequence and group processes affect subsequent discussion and cognitive outcome" (p. 26) and found that "interactions involving conflicting viewpoints promoted more discussion and critical thinking, and the evaluation of arguments was more likely to occur as conclusions were being drawn-not as arguments were being presented" (p. 25).

However, there is also significant support in the research literature for online course designs which do not require student-student interaction, for example:

Indiana University's Learning to Teach with Technology Studio (LTTS) effectively uses a self-paced, one-on-one mentored format for courses designed to serve K-12 teacher professional development needs (Duffy and del Valle, 2005; Malopinsky, Kirkley, & Duffy& Duffy, 2002).

Anderson (2003) suggests that deep and meaningful formal learning is supported as long as one of the three forms of interaction (student-teacher; student-student; student-content) is at a high level. The other two may be offered at minimal levels, or even eliminated, without degrading the educational experience.

Sener (2001) reported case studies which utilized a "tutorial model" of instruction featuring strong reliance on tutor-like instructor-learner interaction without requiring learner-learner interaction.

Ragoonaden & Bordeleau (2000) found that two problems emerge from blanket designs for collaborative interaction: technical and interpersonal. Highly independent learners often prefer to work alone, especially when group members worked at a different pace.

Kramarae (2003), Cook (1989), & May (1993) suggested "more is not necessarily better" and warned of potential dangers resulting from imposed interaction as it might interfere with student autonomy in managing time, place, and pace of learning. This is also consistent with the American Council on Education's Distance Learning Principles, which state that "the learning experience [should be] organized to increase learner control over the time, place, and pace of instruction" (Learning Design Subprinciple 5).

For this reason, the QM rubric directs reviewers to look for evidence of course design which supports student-student interaction as appropriate, recognizing that required or even optional student-student interaction is not necessary for all online courses.

A word of caution: as Swan (2003) notes, while the research results to date establish indirect connections between learner interaction and perceived learning, direct connections between instructor-learner interaction (and by extension, student-student interaction) and learning outcomes have not yet been documented.

Literature Support for Specific Review Standards:

VI. COURSE TECHNOLOGY

Literature Support for the General Review Standard:

While there is plenty of research related to the concepts of active learning and learner interactivity (also see General Standard V), the QM team has uncovered scant research to date which pertains directly to the link between active learning and/or learner interactivity and course technology:

Sonwalkar (2001) describes "pedagogically driven design principles for online education." Among those he suggests that course content is seen as "a set of simulations that support discovery-based learning"; taking advantage of "media assets" to allow for "learner's cognitive preferences"; consider "technological constraints" (¶22-25) of user bandwidth. Additionally, he alerts the important role of experts in content, as well as instructional and web design and the necessity of obtaining copyright permissions.

Sims (2003) found that study participants were able to identify that interactivity ("the inherent quality of the medium and learning environment" (p. 87) included engagement, control, communication, design, the individual, and learning. Those "nodes" can be connected directly to theoretical literature as "providing benefits to learning" (p. 101). Nicol, Littlejohn, and Grierson (2005) found that well-structured learning resources facilitated team collaboration and the learning of engineering design. Students in the course were positive about the value of a well-structured (accessible, easy to use) shared workspace to support their collaborative learning projects in engineering design courses.

Shirathuddin, Hassan, & Landoni (2003) based on a critical reviewed of four well-known usability design guidelines identified seven factors of usability that should be considered in preparing content for electronically delivery in higher education courses: They pointed out "unnecessary media should be avoided to present long downloading time: (p. 119).

Johnson (2004) found that faculty generally "decide to implement tools based on whether they believe students can easily use them" (¶18) and therefore information on their use needs to be considered.

Nicol, Littlejohn, & Grierson (2005) found that well-structured learning resources facilitated team collaboration and the learning of engineering design. Students in the course were positive about the value of a well-structured (accessible, easy to use) shared workspace to support their collaborative learning projects in engineering design courses.

Literature Support for Specific Review Standards:

VII. LEARNER SUPPORT

Literature Support for the General Review Standard:

Many of the previously-cited literature references about the importance of clear descriptions also pertain directly or indirectly to the need for clear descriptions of technical support offerings.

Shin (2001) found that the perceptions of psychological presence a distance student holds in relation with teachers, peer students, and institution can be significant predictors of learning outcomes. An institution's transactional presence may be relatively more important than teachers' and peer students' transactional presence (the degree to which a distance student perceives the availability of, and connectedness with, other parties involved in a given distance education setting" (2002, p.121).

As Ludwig-Hardman and Dunlap (2003) note, "the positive influence advising can have on distance learners' ability to successfully fulfill their educational goals has been well documented (Feasley, 1983; Hezel and Dirr, 1991; Paulet, 1988; Thompson, 1989)."

Visser and Visser (2000) noted a general lack of empirical research regarding the effectiveness of student support services in distance education. However, Krauth and Carbajal (1999) provide a comprehensive guide to developing online student services, including evidence that provision of one-on-one access to advisors is an essential student support service.

Chang (2003) identified students wanted access to other course materials access (such as textbook, study guides, the university, and local libraries) (p. 497).

Johnson's (2004) case study indicated that unfortunately providing information to learners about the CMS's [course management systems] was] was "typically neglected...[... [it was] "often assumed that students will figure out CMS skills on their own" (¶18).

Literature Support for Specific Review Standards:

VIII. ACCESSIBILITY

Literature Support for the General Review Standard:

This General Review Standard has recently been revised to be more encompassing (accessibility vs. ADA compliance), so additional review of the research literature is necessary. Literature references which supply such support include the following:

Coombs and Banks (2000) suggested that when designers and teachers are preparing an online course they should step back and try to access the material and move through it as a student who is blind.

Shirathuddin, Hassan, & Landoni (2003) considered four well-known for guidelines of web design usability design guidelines to look at implementing good course design in higher education courses and noted: "Having good design and useful content are inadequate without considering the accessibility factors" (p. 122).

Schwartz (2004) looks at using internet audio to enhance online accessibility and credited Rowland, Burgsthaler, Smith, & Coombs (2004) with calling attention to the "under-utilization of DE program by disabled students, possibly due to the failure of those programs to adapt to their needs" (¶2). Nielsen (2000) and Zaborowski (Shattuck, 2004) commented "accessible features can result in enhanced learning for all student" (¶3).

Dr. Zaborowski, director of Special Programs for the National Federation of the Blind, described Section 508 of the Rehabilitation Act (http://www.section508.gov) as a better piece for online designers to consider because ADA "does not directly deal with the access issue....Section 508 will really help you understand what you have to do with links and frames and radio buttons and forms and all the different components of an online education [course]" (Shattuck, 2004, p. 261).

Kinash, Crichton, & Kim-Rupnow (2004) reviewed literature that concentrated on disability and online learning published from 2000 through 2003. They noted 22 pieces were didactic; 12 were descriptive pieces; three were opinion pieces; and only five were research studies. Importantly they noted that this literature was often found only within the disability journals and not the distance learning journals. They urged further study with questions such as, "How can we promote universal design of the online environment, with particular emphasis on complementing the visual with the auditory interface?" (p. 12) and suggest "the potential of multimedia to support multiple learning styles and needs" (p. 13).

Edmonds (2004) called attention to "first-generation" and "second-generation" issues of access. He pointed out that first-generation access issues revolve around HTML accessibility and are usually addressed by developers of courseware and Web designers. Second-generation issues revolve around the non-HTML applications, such as non-text elements used with Word and PowerPoint. These are "generally in the hands of the faculty member creating the course- that is, a person without the technical background to provide accessibility" (p. 59). He provides directions to make PowerPoint presentations into HTML which are accessible. Edmonds noted two studies (Rowland, 2000; Schmetzke, 2002) which found that only a same small percentage of distance learning providers and organizations provided accessible.

Weir (2005) "collected observations and anecdotal information to gain insight into the experiences of special-needs students who take online courses" (p. 30). As a result she developed a set of guidelines for developing accessible online courses, including instructional strategies and course design considerations.

See http://www.access-board.gov/508.htm; http://www.webaim.org for design recommendations (Moore & Kearsley, 2005, p. 119).

Learning style is of great interest to those interested in distance education with some reporting a learning preference of visual learning, but most support Hannafin's position that learning style is not a good predictor of whether a student will succeed. Neuhauser (2002) compared face-to-face and online sections of the same course and found there was no evidence that either learning style or type (e.g. visual, auditory, or kinesthetic/tactile) is a good predictor of educational success. Aragon, Johnson, & Shaik (2002) found there were significant differences in learning style preferences between online learners and face-to-face students, but those differences were no significant when success factors were controlled. Thus these studies support the notion that learning style is a less significant accessibility issue.

Shattuck (2005) noted that culture plays a part in making education fully accessible since many non-Western cultures emphasize different dimensions of teaching and learning (Collis & Remmer, 1997; Robinson, 1999; Robbins, 1997; Chen, Mashhadi, Ang & Harkrider, 1999; Chen & Mashhadi 1998a, 1998b; McLoughlin, 1999; Chase, Macfadyen, Reeder & Roche, 2002; Tu, 2001; Gunawardena, Nolla, Wilson, Lopez-Islas, Ramirez-Angel & Megchun-Alpizar, 2000; Sanchez & Gunawardena, 1998; Gunawardena, Wilson & Nolla, 2003). Study participants (all identifying as coming from non-Western cultures) could describe tensions (with teacher-student roles, with course structure, with cross-cultural nuances and real communication) when working within their "Western" online distance education environments.

Literature Support for Specific Review Standards: