Overview and Introduction

Today, we conclude the WCET blog series on microcredentials which set out to address the inception, implementation, and evaluation of microcredentialing approaches across the higher education industry. We learned that there is a wide range of reasons for an institution to begin a microcredential initiative, an even broader spectrum of microcredential terminology, and yet two primary outcomes: “to improve student recruitment and retention” and “to align learning with shorter-term outcomes that can more immediately align with career progression” (Niles and Lazarewicz, 2022). Institutions and training providers use microcredentials to adapt to a number of trends such as decreasing student enrollment and workforce degree requirements and increasing acceptance of online learning, employer demand for skills training, and public demand for greater return on educational investment (Higher Learning Commission, 2022). As professionally trained and former instructional designers, we begin by defining what success looks like. In this blog post, we will present what we learned about how to assess and evaluate the efficacy of microcredentialing approaches through a variety of stakeholder perspectives.

Multiple Perspectives on Success

“One of the positives from the pandemic has been the acceleration of innovation and change in higher education – microcredentials have grown in scale and areas of focus to help provide current students or adult learners additional knowledge to help them be more competitive in the job market. The assessment and evaluation of whether these microcredentials are making a difference for students and adult learners and whether employers value these micro credentials. The next phase of the growing area of microcredentials should seek to answer the questions about employer value, industry microcredentials, or higher ed developed micro credentials, measuring learning outcomes, and more,” shared Melissa Vito, Vice Provost of Academic Innovation at the University of Texas at San Antonio.

Key questions: Key Questions: What does success look like for students, employers, faculty, and administrators? What are the needs that institutions can help learners and employers to fill, in terms of gaps in the curriculum for learners to gain more marketable skills and be competitive in the marketplace? What are the hidden needs that employers have not recognized but that institutions can identify through the work of their faculty as experts in the field? Who are the stakeholders that should be included in the microcredentialing process, and how should institutions build a strong governance model? How can microcredentialing programs be intentionally and strategically integrated in the curriculum design processes?

Microcredentials should create positive labor market outcomes in terms of employment and earnings. “If done correctly, microcredentials help students develop skills that today’s employers are expecting from their entry-level talent” stated Scott Shireman, Global Head of Coursera for Campus. “When microcredentials are built and recognized by industry leaders like Google, IBM, and Meta, they reflect the skills and tools most in-demand by top employers.” Shireman also invited us to consider how microcredentials impact students. “From a student recruitment perspective, the fact is that students today, and their parents for that matter, are thinking more critically about the ROI of the traditional four-year degree. We’re seeing that shift in mindset reflected in university recruitment numbers. It’s a very competitive landscape, and universities are increasingly leveraging microcredentials and job-relevant skills training to differentiate their brand and show students that they’ll benefit from multiple signals in a competitive and fast-evolving market – the two or four-year degree they graduate with, and the career certificates they earn along the way.” The new Coursera survey published shows high demand for industry microcredentials from students and employers in tight labor market.

Kate Smith, President of Rio Salado College, reinforced that student success is at the core of an effective microcredential: “Evaluation of microcredentials is tied to outcomes of students – retention, completion, post-completion employment, and wages, along with structured qualitative feedback from learners, employers, and related faculty and staff.” Microcredentials can provide students with early exposure to career choices, helping students to “build confidence to continue their post-secondary education” and also to reinforce or redirect career choices.

For former Mesa Community College student, Sam McFadden, a two-week, full-time “wire harness bootcamp quite literally changed my life, and has put me in a position to be successful in my career, and complete my education. Three years ago, it would have been difficult to imagine that I would be where I am today.” Today, Sam is employed by Boeing, the world’s largest aerospace company.

Jude Chudi Okpala, Professor of Instruction at the Department of Philosophy and Classics at The University of Texas at San Antonio explained what success should look like for students. “Success in a critical thinking microcredential, for example, consists in application of critical thinking on a concrete and real problem. Looking at students’ work in such microcredentials, one sees a correspondence between the objectives and the learning outcomes for the microcredential: for example, evidence of logic in addition to effective understanding of complexity and the relationship between the problem and solution. Invariably, a successful microcredential in critical thinking should reveal independent thinking and the students’ ability to solve a concrete problem. The microcredential should demonstrate without ambiguity the students’ skill in critical thinking/problem-solving.”

Lisa Larson, President of Eastern Maine Community College, Ed Design Lab, Designer in Residence believes “micropathways are the gateway to transformation” because this demonstrates “we are committed to transforming our institution to meet the needs of those who need us most.” Larson emphasized the need to measure the engagement, student and employer value proposition, and impact after completion. For example, getting employers to include microcredentials in their recruitment, hiring, and promotion practices are metrics of success. In short, Larson asks “Who did the microcredential make a difference for, and how can we tell the story of impact?” Microcredentials may also increase access for non-traditional students to “break through ceilings” and “earn a living wage” contributing to equity and economic growth in our communities. Larson advised that microcredentials should be “digitally discoverable”, so students can “own the learning upon receiving validation of the competency and skill.” The digital nature of a microcredential should “create the visibility for students to take every ounce of learning and experience with them.”

Assessment and Evaluation

“Ideally a feasibility study includes a market review of demand, wages, stakeholders, and resources needed to build and implement the credential, combined with a SWOT analysis. The microcredentials are then developed with input and co-design principles when feasible, with local or national industry partners” (Kate Smith, 2022, personal communication). Data analysis plays a key role in evaluating the effectiveness and impact of scope, resources, risks and feasibility of a microcredential project. Consensus-building with campus stakeholders and employer stakeholders is critical to a mutual definition of what success looks like.

Key Questions: How do we measure the efficacy of microcredentials in this evolving landscape? What are the standards that institutions, employers, faculty, and learners need to agree upon to create transparent and relevant pathways for career? How will competencies and skills be assessed and validated by both institutions and employers? How will the institution and student signal the completion of the microcredential? How can the evaluation process drive a cycle of continuous improvement to generate microcredentialing programs that are up-to-date and aligned to the market needs?

Lydia Riley, Chief of Staff of The University of Texas System (UT System) invites institutions to consider the “AGILE planning framework with frequent feedback loops that lay out the foundations of the microcredential programs, identifying strategic goals for revenue-generation or mission-enhancing endeavors”. All stakeholders expressed a desire for marketable skills, relevant, contextualized curriculum, and increased perspective of employers into curriculum. While success may be measured by the value proposition to learners, the microcredential must also be validated by employers. One of the most challenging elements when planning and assessing microcredential approaches “is the personnel capacity and stakeholder capacity; the people on the ground really designing and implementing into the institutional structures” (Lisa Larson, 2022, personal communication).

Leah Palmer, Executive Director at Maricopa Community Colleges Arizona Advanced Manufacturing Institute (AzAMI) emphasized the importance of data tracking and benchmark metrics to measure the impact on student success, costs to the institution, and sector demands for talents. Palmer recommends tracking the number of hits on your microcredential webpage, enrollments and trajectory, value propositions such as course completions, earned badges or certifications, and follow-up emails for employment-related status. We must create opportunities for enhancement and/or implementation of corrective actions during the microcredentials planning phase.

Micheal Voss, Dean of Instruction at Mesa Community College recommends an “annual threshold review of Key Performance Indicators (KPIs) and a more robust program review at least every 5 years. Ideally, program completers should receive follow-up surveys 12-18 months post-completion to gain feedback as to how the knowledge, skills, and behaviors learned have impacted the person’s employment or next step education status.”

Lori Lindenberg, District Director of Data Analytics and Strategy at Maricopa Community Colleges explained how their system of 10-independently accredited colleges shares data through the Student Clearinghouse Postsecondary Data Partnership (PDP) to assess the leading and lagging indicators typically associated with the guided pathways framework, such as credit momentum and time to completion. Measuring economic outcomes has proven a little harder for some institutions. Employment data is notoriously difficult to acquire yet increasingly important to track the impact of microcredentialing. Maricopa Community Colleges has a partnership with Equifax to study wage data (pre- and post-completion), annually for up to 10-years, but this has been traditionally focused on degrees and certificates. They are working to transcribe recognition of microcredentials into their student information system and also experiment with digital badging platforms. The country’s largest accreditor, the Higher Learning Commission, sees an opportunity to provide a quality assurance service for non-degree credentials, yet they note their current limitations in capacity and scope (HLC, 2022). Perhaps third-party validation will promote shared standards of quality and transferability of microcredentials.

Key Takeaways and Summary Points

Through the testimonies and insights gathered in this process, we have identified that different data points will be indicators of success based on stakeholder perspectives (Larson, personal communication, 2022). Institutions and employers should base their microcredential partnership on key values and standards such as transparency, validation, recognition, and reward of skills. Table 1 illustrates this point that the measures of success may be complex and individually defined. The value proposition of a microcredential often depends on one’s perspective. 

Table 1. What success looks like by stakeholder group – words commonly used

Student Employer Faculty Administration Accreditor
Affordable Meet a specific workplace need that directly support industry job roles and job descriptions that are in demand Engagement Meet a specific workforce development need Quality assurance
Short-term Recruit skilled workforce  Learning  Student’s success Curricular alignment
Re-careering Diversity equity and inclusion strategies for recruitment Authentic assessment Enrollment Employability of graduates
Upskilling Retain and reskill/upskill current employees Retention  Retention Transparency of tuition, fees, and costs
Recognition of skills  Career pathways Completion Complete ROI for students
Career progression   On-ramps to educational pathways Completion  
Higher wages   Off-ramps to employment Revenue  
New job or different job   Stackable educational and career pathways Digitally discoverable  
Pathways to economic self-sufficiency   Curricular alignment to employment Portable for the learner  
Gain more confidence in career path and future   Career-ready Valuable to the learner   
Confidence to continue engaging in post-secondary education   Industry engagement and advisory Validated by the employer  
Effective demonstration of the target skills     Equity opportunities for students   
Authentic alignment of the learning objectives and outcomes     Community engagement  
Understanding of how to apply the skills        
Reflection on acquired competence and its corresponding outcomes–what the students should be able to do        
Concrete professional growth        
Relevance of the assessment and partnership with the industry/community        

It became quite evident that assessing and evaluating microcredentials will require rapid prototyping with inclusive feedback loops from students, faculty and employers. Microcredentials have shorter lifecycles and therefore continuous evaluation of KPIs and lead measures are critical. Table 2 shares examples of different KPIs for assessing and evaluating microcredentials that align to the ADDIE instructional design process.

Table 2. Key Performance Indicators to assess along the microcredential lifecycle 

Needs Analysis Design & Development Implementation Evaluation
Institutional capacity  Stakeholder buy-in Time-to-market Student surveys
Personnel capacity Employer engagement Minimally viable product  
Feasibility analysis Co-design between institutions and employers Marketing leads Student completion 
SWOT analysis Standards  Enrollments Student intent fulfilled
Identification of skills gaps Signaling completion Alignment to industry jobs Feedback about the positive outcomes of the microcredential
Technology needs   Transparency, validation, recognition and reward of skills Employment opportunities
      Revenue generation
      How and where the student shares the microcredential
      Employer integrates into recruitment, hiring, retention, and promotion processes
      Content quality 
      Improve the design of the credential

Contributors

A heartfelt thank you to all the expert interviewees lending their time, leading this transformative movement in higher education, designing these critical learning experiences, and for helping us to measure what matters most. Interviewees included:

  • Dr. Lisa Larson, President of Eastern Maine Community College, Ed Design Lab, Designer in Residence
  • Lori Lindenberg, District Director of Data Analytics and Strategy – Maricopa Community Colleges
  • Dr. Jude Chudi Okpala, Professor of Instruction- Department of Philosophy and Classics – The University of Texas at San Antonio
  • Leah Palmer, Executive Director – Arizona Advanced Manufacturing Institute (AzAMI) – Maricopa Community Colleges
  • Lydia Riley, Chief of Staff – The University of Texas System
  • Scott Shireman, Global Head of Coursera for Campus
  • Dr. Kate Smith, President – Rio Salado College- Maricopa Community Colleges
  • Dr. Melissa Vito, Vice Provost of Academic Innovation – The University of Texas at San Antonio
  • Michael Voss, Dean of Instruction – Mesa Community College – Maricopa Community Colleges

References:

Higher Learning Commission. (2022, April). Evolving: Accreditation and credential landscape. Retrieved on September 20, 2022 from https://download.hlcommission.org/initiatives/2022StakeholdersRoundtablePapers.pdf

Lazarewicz, K., Niles, G. (2022, May 26). Considering Your “Why”- How Developing a Microcredential Strategy Will Help Your Students (and Your University) Thrive. WCET Frontiers. https://wcet.wiche.edu/frontiers/2022/05/26/how-developing-a-microcredential-strategy-will-help-your-students-and-your-university-thrive/  

Shireman, S. (2022, September 27). New Coursera survey shows high demand for industry microcredentials from students and employers in tight labor market. Coursera Blog. https://blog.coursera.org/from-higher-education-to-employment/

Claudia Arcolin

Executive Director of Teaching and Learning Experiences, UTSA Academic Innovation


claudia.arcolin@utsa.edu

LinkedIn Profile

Janelle Elias

WCET Steering Committee, Vice President, Strategy & Advancement, Rio Salado College


janelle.elias@riosalado.edu

LinkedIn Profile

Subscribe

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,168 other subscribers

Archive By Month

Blog Tags

Distance Education (272)Student Success (256)Online Learning (201)WCET (193)U.S. Department of Education (190)State Authorization (188)Managing Digital Learning (184)Regulation (183)Technology (159)Digital Learning (133)Innovation (125)Teaching (120)Collaboration/Community (113)Course Design (102)WCET Annual Meeting (101)Access (97)Cost of Instruction (88)Professional Development (88)Faculty (84)Financial Aid (83)Legislation (83)Completion (74)Assessment (69)SAN (67)Instructional Design (66)Open Educational Resources (65)COVID-19 (63)Accreditation (62)Competency-based Education (61)SARA (61)Quality (61)Credentials (60)Data and Analytics (59)Research (58)Professional Licensure (57)Accessibility (55)Reciprocity (48)Outcomes (47)WOW Award (46)Diversity/Equity/Inclusion (45)Workforce/Employment (41)Higher Education Act (41)Regular and Substantive Interaction (39)Policy (38)Negotiated Rulemaking (37)Title IV (36)Virtual/Augmented Reality (36)Practice (35)Disaster Planning/Recovery (34)Academic Integrity (30)Leadership (30)IPEDS (28)Reauthorization (28)Survey (27)Adaptive/Personalized Learning (27)Credits (26)Military and Veterans (26)Disabilities (25)MOOC (23)WCET Summit (23)Every Learner Everywhere (22)WCET Awards (21)Evaluation (21)Complaint Process (20)Retention (20)Correspondence Course (18)Physical Presence (17)State Authorization Network (17)WICHE (17)Member-Only (16)Products and Services (16)Enrollment (16)Forprofit Universities (15)WCET Webcast (15)Blended/Hybrid Learning (14)System/Consortia (14)Cybersecurity (14)NCOER (14)Textbooks (14)Digital Divide (13)Mobile Learning (13)Consortia (12)Futures (11)Marketing (11)Privacy (11)Personalized Learning (11)Prior Learning Assessment (10)Courseware (10)STEM (10)Teacher Prep (10)Social Media (9)LMS (9)Rankings (9)Standards (8)Student Authentication (8)Partnership (8)Artificial Intelligence (7)Tuition and Fees (7)Readiness and Developmental Courses (7)What's Next (7)International Students (6)K-12 (6)Nursing (6)Remote Learning (6)Testing (6)Graduation (6)Lab Courses (5)Proctoring (5)Closer Conversation (5)ROI (5)DETA (5)Game-based/Gamification (5)Dual Enrollment (4)Outsourcing (4)Coding (4)Security (4)Fall and Beyond Series (3)In a Time of Crisis (3)Net Neutrality (3)Universal Design for Learning (3)Cheating Syndicates Series (3)Student Identity Verification (2)Cross Skilling/Reskilling (2)Nontraditional Learners (1)Title IX (1)Higher Education Trends (1)Mental Health (1)

Follow us on Twitter