Meet Maura, A Virtual Intelligent Agent Answering Student Services Questions

In the spring semester of 2013, American Sentinel University was experiencing the welcomed issue of high student traffic. The university had back-to-back years of significant growth in enrollment, and new degree program requirements were being implemented in the spring, naturally creating curiosity among the student population. While we encourage and are happy to talk with our students and give them the answers they need, at the time the bandwidth of our student success advisors was beginning to be stretched thin.

Coincidentally, the success of Apple’s Siri had become mainstream. A team within our university, including then-Chief Academic Officer John Bourne and former advisor-turned instructional designer Trevor Rasmusson, were conceptualizing how similar technology could benefit our students’ experience.

The result, a virtual intelligent agent: Maura (My Anytime University Resource Aide)

Their idea:  A virtual intelligent agent, complete with a face and a voice, who lived inside university webpages and could answer the questions of American Sentinel University students. Day or night. Weekdays and weekends. 365 days a year

Introduced to our students in September 2013, Maura’s avatar appeared on our registration site as well as the login page for our classroom. And while Maura received a rather warm welcome from our students (an objective analysis based on the higher than expected questions/inputs that Maura received in her first few days of going live), she did not go without some hiccups in this implementation stage.

The first, and arguably most disadvantageous, obstacle was the agent’s response accuracy.  Since most student questions previously came in the form of emails and phone calls, there was no conceivable way to catalog all questions our students could ask Maura. Therefore, Rasmusson and a team of student success advisors relied on their experience working with students to generate the content for Maura. Many of the agent’s responses were specific to frequently asked questions, such as how to register for a class to the steps necessary to complete a graduation audit. And while these questions were accounted for when Maura went live, it was impossible to anticipate all the questions (not to mention how they would be asked) our students would ask. Therefore, Maura began at a rather low response accuracy rate (around 40%).

We also had to work on acceptance from Maura’s human colleagues

This lead to a second problem: Lack of buy-in within the university. In seeing the low success rate Maura began at, there was understandable skepticism among staff and faculty as to whether this resource was advantageous to the student experience.

So, to enhance Maura’s response accuracy as well as her overall value to the university, we conducted an extensive audit process. We needed a log of as many possible questions Maura could be asked in order to create appropriate responses. We reached out to faculty and advisors, asking them to take student questions they have received and ask those to Maura. We asked for students to challenge Maura with university-related questions they could think of. These, along with other live questions our agent received, helped us to generate a much more extensive catalog of questions that Maura could accommodate for. With this, Maura was able to learn a lot in a short period, building her knowledge base to over 500 unique responses and raising her response accuracy rate to around the 80% mark.

Maura has grown and continues to learn

Since then, we review the log of questions Maura receives on a weekly basis to continue to help her learn. To make these necessary revisions, our team uses Artie™, a product of Healthcare Learning Innovations that allows users to create and revise virtual intelligent agents from scratch. While the 100% response accuracy rate may be an unachievable goal, we now proudly point to Maura’s two years of functioning at over 90% accuracy and happily call her a member of the American Sentinel family.

 

Gregory Dennis
Manager, Writing Center
American Sentinel University