My online course quality rubric has a first name. It’s O-S-C-Q-R!

Today on WCET Frontiers, we welcome Alexandra Pickett, the Director of Open SUNY Online Teaching, to talk with us about the development of the online course design rubric OSCQR. This rubric was developed to help faculty improve the quality and accessibility of their online courses. She also reviews the components of the rubric and the process and gives an update on how the rubric is being used (and includes some tools to help you use it as well!).

Thank you to Alexandra for discussing this wonderful tool and process.

Enjoy the read and enjoy your day,

– Lindsey Downs, WCET

---

The Open SUNY Online Teaching (formerly the Open SUNY Center for Online Teaching Excellence (COTE), developed an online course design rubric and process that addresses both the instructional design and accessibility of an online course that is openly licensed for anyone to use and adapt. The aim of the Open SUNY Course Quality Review (OSCQR) Rubric and Process is to assist online instructional designers and online faculty to improve the quality and accessibility of their online courses.

OSCQR, also provides a system-wide approach to collect data that informs faculty development and supports large scale online course design review and refresh efforts in a systematical and consistent way. The OSCQR rubric and process are currently being used by 56 SUNY institutions and 825 non-SUNY individuals, institutions, and organizations.

The Development Process

We developed OSCQR in response to the lack of adoption/use, cost, and the prescriptive and evaluative nature of other commercially available rubrics. We needed a rubric that would allow greater in both the flexibility of training models and the addition and customization of standards, and that could accommodate both new and mature online courses, as well as a variety of online course quality review models in addition to a peer review model. It was also essential that the rubric and process be non-evaluative. Where other rubrics resulted in scores (and potential faculty failure), it was important that we view and promote the online course quality review and refresh process as a positive professional development experience for online faculty with any level of online course development experience.

We wanted a tool that would do more than merely point out deficiencies. We wanted the tool to be able to provide significant substantive and positive suggestions on how to address each standard well. And to support and facilitate the plan and priorities for improvements to help guide the online course refresh process. We also wanted to incorporate a mechanism to crowd-source additional suggestions and examples for each standard, and to rally and recognize our deeply experienced community of online practitioners around online course quality assurance activities.

The work relied on multi-institutional teams of SUNY online instructional designers, librarians, distance learning directors, and technologists. They reviewed and discussed the California State University, Chico Rubric for Online Instruction, 20 years of SUNY Learning Network (SLN) research-informed best online practices, the SUNY office of general counsel’s memorandum on accessibility considerations, and a gap analysis with Quality Matters, iNACOL’s standards for Quality Online Courses, and Blackboard exemplary courses.

The resulting rubric was also informed by the Community of Inquiry model (Garrison, Anderson, and Archer, 2000), The 7 Principles for Good practice in Undergraduate Education (Chickering & Gamson, 1987), The Adult Learner (Malcom Knowles, 1973, Blooms Taxonomy (Bloom et al., 1956) and How People Learn (Bransford et al., 1999). The findings were mapped to the Open SUNY fundamental competencies for online teaching.

The Result: The OSCQR Process and the OSCQR Rubric

There are two components to OSCQR:

1. The OSCQR Process provides a Framework and Dashboard that supports a campus-tailored and scalable approach to improving the instructional design of online or blended courses.
   • The Framework includes:
     • A Course Review that results in an action plan to improve the design of the online course, that is supported by specific examples and ideas.
     • The Course Refresh based on the things targeted for improvement by the course review.
     • A Learning Review that identifies and prioritizes then next set of improvements for continuous quality improvement.
   • The campus Dashboard is the tool from which all the rubrics at a given campus (or in a program, or department) can be generated, customized, and managed.
     • It provides automations for campus-level management of all course reviews, custom prioritization of the standards, and incorporation of additional standards. It can be used with any rubric and can assign different rubrics to different courses.
     • Analytics are built in to aggregate and track all course review progress, identify and document course design issues and trends, and to aggregate information across courses to inform faculty development initiatives and course development planning.
2. The OSCQR Rubric has 50 integrated online course design standards and accessibility standards. The Rubric is flexible and designed to be used in a variety of course quality assurance approaches.
   • By instructors and instructional designers in faculty development and course design professional development activities to inform and influence the design of new online courses.
   • By an individual instructor to self-assess and prioritize design improvements; to continuously review, revise, and improve the instructional design of their existing online courses.
   • By an instructional designer to conduct a formal course review of an online course as part of an online course quality review process at the program, department, or institutional level.
   • As a peer review process, by a team of instructors interested in a peer-review model of online course review and continuous improvement (the teams can be made up of inter- or intra-disciplinary teams).
   • In a collaborative team model made up of a group of at least 3 people approaching the course review process from their own various specialized perspectives, i.e., instructional designer, course author, and external reviews that might include other subject matter experts (faculty), online librarian, student, instructional technologist, multimedia designer, or other faculty.

Tools to Help Use OSCQR

A complimentary resource to the rubric has also been developed and integrated into the rubric. OSCQR.org provides the online course reviewer with additional resources for each standard, including:

1. A full explanationof the standard, including a micro-learning video designed to explain the standard and share examples.  
2. A list of examples, ideas, and suggestions to improve, or address the standard as a supplement, or to inform the reviewers recommendations.
3. Related research that provides reputable evidenceand scholarly support for the standard.
4. Additional examples of Effective practicesfrom the University of Central Florida’s Teaching Online Pedagogical Repository (TOPR) and OLC’s Effective Practices by Institution.
5. The opportunity to explore additional related resources.
6. The opportunity to submit additional examples, evidence and best practices for consideration to be added to the site.

How the Rubric is Unique

The rubric is unique in many ways. It is not restricted to mature courses. The rubric can be used formatively with new online faculty to help guide, inform and influence the design of their new online courses. It is non-evaluative: Conceptually the rubric and process approach course review and refresh as a professional development exercise, to guide faculty in their understanding of improving course design from an effective practices perspective, rather than as a course evaluation, or quality assurance procedure. The rubric can be customized. Standards can be added, edited, and /or eliminated. There is no license fee for use of the rubric. It is shared with a Creative Commons license: CC BY-3.0 US.

An Action Plan is automatically generated by the course review process that presents recommendations for course design improvements based on the review and assists in prioritization of course revisions based on the estimated time to make those improvements. The rubric also provides suggestions for course design improvements for each standard that can be selected from a menu of options by each reviewer to supplement reviewer feedback.

The action plan is framed from the perspective of the Community of Inquiry (CoI) model, to help reviewers assess and target opportunities to improve the course’s social presence, cognitive presence, and teaching presence, in addition to the overall online course educational experience. It also substantively addresses accessibility. The OSCQR Accessibility Rubric is based on the recommendations of SUNY’s Office of General Counsel in their 2013 memo, “Accessibility Considerations in the wake of SUNY’s Online Initiatives”. The rubric has been reviewed by members of the FACT2 Accessibility Task Force, and address the legal considerations required to be compliant with Section 508 of the Rehabilitation Act of 1973, New York State Enterprise IT Policy NYS-P08-005, Section 504 of the Rehabilitation Act, and the Americans with Disabilities Act.

A partnership with the Online Learning Consortium (OLC) initiated in November 2016 has resulted their adoption of OSCQR as one of their online quality scorecards.

Because the OSCQR Rubric is licensed under Creative Commons, and the Dashboard is licensed under LGPL, the entire process can be shared, used by anyone with no cost, and can be customized to address individual campus environments.

OSCQR Links

About OSCQR:
http://OSCQR.org/about

The Rubric:
http://bit.ly/OLCoscqr

OSCQR rubric examples and annotations resource:
http://oscqr.org/

OSCQR User Group:
http://cotecommunity.open.suny.edu/group/oscqr

Open SUNY Online Teaching
https://commons.suny.edu/cotehub/

 

 

Alexandra M. Pickett
Director, Open SUNY Online Teaching
The State University of New York

 

 

 

---