Establishing a Quality Review for Online Courses by Tracy Chao, Tami Saj, and Felicity Tessier
WHO IS RESPONSIBLE FOR ASSESSING QUALITY IN ONLINE COURSES?
Measuring quality in online courses is a contemporary topic of discussion in our field. This is a concern of faculty, students, administrators, and instructional designers.
This article presents a practical approach to implementing a quality review of online courses. The article reveals that measuring quality of online courses is a complex task; there are many variables in producing a quality course that goes beyond the teacher and the curriculum (but of course we all know that…).
Quality in online courses is typically measured in terms of courses evaluations, perceptions of teachers and administrators and peer teaching observations. Typically, for program accreditation, the state requires academic programs to conduct faculty observations to demonstrate quality teaching. These measurements are typical of what is used in a traditional face to face classroom but they may not be best for the online classroom.
The authors suggest additional quality measurements for online courses: instructional design, course development, and use of technology. However, other measures need to be place for assessing quality of teaching, curriculum design, and experience of the learner. What issues are important in assessing quality in these areas?
What makes a quality course?
According to the authors, quality can be reviewed and measured along six different areas. The authors provide a model for measuring quality. Six areas are identified as the quality framework for web-based courses. The authors present corresponding measurements for three out of the six quality areas (instructional design, web design, and course presentation). See below for a summary of the division of labor for assessing quality measurements presented in the article.
Area of quality and Who assesses quality?
1. Curriculum design: Academic units ensure the curriculum meets quality standards for content and learning standards
2. Teaching and facilitation: Academic programs use interim formative surveys and final course evaluations to help assess the quality of teaching and facilitation.
3. Learning experience: Academic programs use interim formative surveys and final course evaluations to help assess the quality of the learning experience
4. Instructional design: Collaborative relationship between instructional designers and academic units ensures shared responsibility for sound instructional design for a course
5. Web design: The producers of the online courses are responsible for and ensure quality standards in web design. The producers are CTET.
6. Course presentation: The “course writer” or editor proofreads the materials at the predefined stages of development.
The parties that assess quality in four and five raise some important issues:
In assessing the quality of instructional design the collaborative partnership is essential. How is this relationship is constructed between faculty and instructional designer. How responsive is the faculty to feedback on instructional design from the instructional designer? What if the faculty does not have the skills in instructional design, how can it be ensured that the quality is properly assessed?
The producers are responsible for assessing the quality of the web design of online courses. However, what happens when the faculty are the producers of the content? Who should be responsible for assessing?
Some strategies for accessing quality:
1. Team approach to quality review within the online teaching and learning support group for academic programs.
This approach was presented by the authors for the review of instructional design, web design and course presentation quality. This peer review type process could be very helpful to new instructional design groups and to mentor new instructional designers. Some questions I have are:
o Does each team member have the skills to conduct a quality review?
o Who are the team members conducting the review?
o Who reviews the team’s review?
o Should there be a dedicated quality assurance staff to review all courses?
2. Partnerships with Faculty and Academic Programs
There must be a balanced partnership between the instructional design group, the faculty and the academic program. In many cases quality is assessed from the instructional design perspective, but not from the academic program perspective when the program is new to offering online courses or vice versa. Concerns that I have experienced in my professional work are on both ends. Some programs are to understaffed to monitor their online faculty and at times programs are too inexperienced to provide sufficient feedback on instructional design, web design, or course presentation.
The tension that arises in the relationship presented between the two contingencies, the instructional designers and the faculty. Who has authority to assess quality? Both parties may be sensitive to stringent reviews; both parties have expertise in their respective areas.
I raise this issue, because it is very real. Areas four, five and six are areas where a group that works with faculty to produce and design online courses fit in very well. However, the lines are blurred when faculty are responsible for creating their own content and do not have instructional designers to work with (which is the case for many programs).
Teaching, curriculum design, and the learning experience are typically left to the academic department to monitor. When new departments embark on online learning, do they have structures and processes in place for doing this type of quality assessment? What structures, processes, measurements and support do academic programs need to truly assess the quality of their online courses?
posted by sgoss @ 16:44
3 comments
3 Comments:
Check out http://www.qualitymatters.org if you have not done so already! They have a rubric and quality control program that is worth studying in detail. QM really only pertains to the course design process and not the implementation (teaching) and learning experience, so the model presented in this article is more complete.
Great article. I found it amusing that the authors said that “course content dictates learning outcomes.” Most ID models put it the other way around. I’d like to know more about how the Curriculum Quality Insurance Program was put into place. I like that they separate the what and the how of the design process, suggesting that there are many ways to achieve the same thing. Since I am working at a unionized state institution I have to work within highly defined procedures. The idea of using the quality review to determine allocation of resources makes this review like a needs analysis- where should staff time be focused. Great idea. As the authors note, the review takes time, and their model does not scale. QM trains reviewers who can then freelance or work as reviewers at their own institutions, creating a review process that scales with the number of people involved in online teaching and the QM process.
Coming up with scales and rating systems is the easy part. Implementing them can be a very difficult process- are they an additional form of evaluation? Will they be used for tenure and review? Will faculty with a highly rated course receive additional remuneration? Will course with a low score be cut? Should faculty be involved in the rating procedure? Many faculty fear a process driven entirely by staff who are controlled by the administration.
I like the idea of a holistic application of standards - not the separated, piecemeal vetting of grammar, layout, UI functionality, and instructional validity that i see in my daytime job. By setting uniform standards and taking an organized approach, consistent results can be obtained.
The authors were honest: about the time involved for review, and the need for a team approach to bring the necessary skills to bear. Often, I see reviews that are hurried and based on the individual reviewer's area of expertise, with little or no allowance for a number of factors that come together to make a course effective or ineffective. Proper quality assurance involves adhering to a plan and to standards. 'Mature' conventional media like film and certainly print have such processes, though the names put on them may vary. By defining how what elements of online courseware will be examined in the QA process, and by what benchmarks these elemnts will be measured, CTET takes a quality assurance approach to quality assurance.
Assessing program quality and the issues of assessing outcomes has been an ongoing debate in the realm on distance and online education for many years, certainly long before the advent of the Internet and any Web-based programs. While many of those on the institutional side consider assessments and outcomes to be results from exams, projects, and learner participation in class, they cannot or should not be the only means of assessment. One of the strategies that many for-profit distance learning institutions use as a way of assessing the success of their programs is by tracking their students' careers post graduation (or class completion). For those who complete professional programs - i.e. business, education, health, - the question asked is how has the certificate or degree impacted their careers, their earning power, or their social status. Has the program provided them with the skills to succeed in their lives? For those who develop training programs, the questions focus on whether the skills learned have improved a company's profitability? These are strategies that are measurable and I think should be adopted more widely in addition to the other framework components posited by the authors of this article.
-Rebecca
Post a Comment
<< Home