Leveraging Analytics to Close DEI Access & Attainment Gaps

Today’s post from Adam Cota with the WCET Steering Committee working group on Diversity, Equity, and Inclusion highlights how Adam’s institution, Western Governors University, uses data analytics to accomplish equity and diversity goals. This post continues the series started earlier this month on “Enabling Difference.”

We’re so thrilled to continue this series and focus on DEI throughout this month and thank Adam and the other members of the WCET DEI work group for working with us on these posts. Make sure to register now for our Closer Conversation on Diversity, Equity, and Inclusion in Digital Learning on August 27th. Registration is free and open to current WCET members but limited to the first 50 registrations. All participants are invited to interact via video and or chat.

Enjoy the read and enjoy your day,

Lindsey Downs, WCET

 

---

In last week’s blog post, Janelle Elias provided an analysis of strategic plans from WCET’s membership to understand how they are being used to advance Diversity, Equity, and Inclusion (DEI). With the vast majority of institutions mentioning DEI concepts in their strategic plans, it’s encouraging to see the increased focus and attention this topic is receiving.

In this week’s post, we’ll review how one institution – Western Governors University (WGU) – is leveraging advanced analytics to help drive our access and attainment goals.

We do not believe that one’s ethnicity or income should be predictive of college access or attainment.

At Western Governors University we have set the goal of closing the 4-year graduation rate gap for Black, Latinx, Indigenous and low-income students by 50% within 5 years and eliminating it completely within 10 years. These gaps are measured relative to other student groups. In addition, we want our student body demographics to reflect those of our nation within 10 years. We do not believe that one’s ethnicity or income should be predictive of college access or attainment. This does not mean that success rates for other student groups will remain flat. We want success rates for all student groups to improve, but we want them to improve faster for certain groups to close our achievement gaps.

Advanced Analytics Equity Initiative

To help achieve these objectives, WGU created the Advanced Analytics Equity Initiative in July 2020. This cross-department effort is spearheaded by three academic analytics teams – in Faculty Experience, Learning Analytics, and a dedicated Advanced Analytics team – which work with departments across the university towards a long-term goal of personalizing the student experience at scale to achieve our equitable access and attainment goals.

In the short-term, we are focused on three areas of research:

(1) Outcome driver research

While outcome gaps exist across ethnic groups at many universities, ethnicity is not the driver of these outcomes. Ethnicity is however correlated with many of the underlying drivers of student success. One objective of our Advanced Analytics Equity Initiative is to identify all drivers associated with the gap in performance between ethnicities. At that point, we will be able to adequately characterize the factors correlated with ethnicity that are driving differences in performance.

Many of the drivers identified to date are unsurprising and associated with systemic racism. For instance, prior academic preparation and income correlate strongly with ethnicity and zip code and are indications of unequal academic and economic opportunity. Other drivers – such as students with a COVID illness or household COVID illness, students working overtime, or students experiencing webcam or software issues – are less well-recognized. Our early research indicates not all drivers of student success are correlated with ethnicity while others may be correlated in ways inconsistent with relative student success measures. For instance, individual attributes such as procrastination or persistence are not correlated with ethnicity or may correlate in a way that offsets the gap in attainment.

In addition to individual learner characteristics (e.g., prior academic experiences and outcomes; professional goals; learning preferences; beliefs about self, higher ed, employment; financial resources) we’re also examining historical and contextual experiences (e.g., geographic factors such as internet access and employer demand; life circumstances). Their experiences at WGU, which we classify into friction points and interventions, are perhaps the most important drivers we’re looking at (see below).

While such efforts are complicated by cross-correlations, questions of causation and data availability, we feel there is substantial work to be done in this area before we can truly understand and eliminate our achievement gaps.

(2) Friction points research

Learner characteristics and geographic contexts do not drive student outcomes in isolation. Ultimately, outcomes are determined by the interaction effects between learners and their educational experiences. These experiences may create friction or build momentum towards student success. While we want to eliminate friction and increase momentum and performance for all students, initiatives that will contribute to the elimination of our access and attainment gaps are a high priority for our university.

Our early research indicates that certain experience may create more friction for some learners than others. For instance, after submitting a WGU application, students with prior higher education experience must request transcripts before being accepted and enrolling. We found that far fewer low-income students of color were completing this process driving up to 60% of the gap in app-to-new start conversion. In response, we are piloting efforts to obtain transcripts on behalf of all students. While that should help access for all students, we expect it will have a greater impact on our Black, Latinx, Indigenous and low-income students. In addition, we’re looking for evidence of bias or other friction points in our curriculum. For instance, we are looking for course-level performance gaps between student groups that are larger than that of the average course.

Working with our operational leaders, we have identified over 1,000 potential friction points across student-facing processes. We have organized these friction points into four vectors of research:

1. academic policies and metrics;
2. community of care;
3. curriculum & assessments content and design;
4. operational transactions.

Our analytics teams will be running analyses to prioritize these friction points by expected impact and identify friction points that differentially impact students of color, women and low-income students.

(3) Intervention’s research

Over the last several years, WGU has created a set of mentor or course instructor trigger-based interventions. In addition, departments across WGU have identified over 160 grassroots initiatives to help us achieve our equitable access and attainment goals. We’ll be assessing the effectiveness of many of these efforts through our Advanced Analytics Equity Initiative to identify differential impact for different groups of learners.

Our early results have identified several initiatives that may be having a differential impact. For instance, participating in certain student groups was associated with greater increases in student success for students of color and low-income students. Awards for excellent performance on assessments similarly show greater impact for these student groups.

…participating in certain student groups was associated with greater increases in student success for students of color and low-income students.
Awards for excellent performance on assessments similarly show greater impact…

Our research is just beginning but we are committed to the road ahead. Armed with information of drivers, friction points, and interventions associated with the achievement gap we will change some experiences for all students (e.g., obtaining transcripts on behalf of all students). Other initiatives could lead to a more targeted response (e.g., greater encouragement to participate for students we expect to benefit most from student clubs). Ultimately, we’ll want these insights to contribute to ‘personalization at scale’ through a combination of our Community of Care (faculty and other student supports) and AI-enabled recommendation engines.

We know that many other institutions are pursuing similar lines of research. We’d love to hear about insights from your institution. What drivers of student success have you identified? What experiences are creating friction or momentum for different groups of learners? What have you changed in response?

---