UMBC’s Learning Analytics Program is a community of scholars interested in using analytical data to advance student achievement and success. We’re relaunching the program after, like so many things, Covid-19 interrupted our plans.  The program has two parts, a community of practice and a competitive mini-grant program supported through the Provost’s Office.  

Meetings of the community of practice are open to all faculty, full and part time.  During these meetings, we’ll share ongoing research, have invited speakers, and learn about the data, tools, and platforms that faculty can use to improve student success and learning.  We’ll have infrastructure experts, data scientists, and others on hand to help interested faculty with access to data and to help design analysis or interventions to better understand the factors that improve student learning or student success in face to face, hybrid, or online classes.  

Our first meeting of the community this semester will be held on March 11, where we’ll have virtual visitors from Indiana University show some of their work on grade surprise followed by a discussion about using learning analytics from both a practitioner and research perspective.   More details about the seminar are available here.

The mini-grant program is open to all faculty (full and part time). Each successful applicant will receive $2,000 in research funds and when needed, a license for our visualization software to enable the exploration of data and the sharing of discoveries. The Analytics and Business Intelligence Teams in DoIT and the Provost’s Office will provide support to those who need assistance with data (and will coordinate support with IRADS when needed) and in creating visualizations or helping design and implement the analysis of data.  

We are interested in proposals surrounding student success and retention through graduation, especially those that explore relationships between activities and learning in one course and outcomes in others.  We are also interested in proposals from instructors of courses that have low success rates, are critical progression points for students, or that impart fundamental skills required for success in the major or program. Collaborative proposals are allowed and encouraged. Multiple proposals with different goals may be submitted from the same department.  Participants who identify course or practice redesign as part of their work are eligible and encouraged to compete for a Hrabowski Innovation Award.

Interested in more details?  Here are links to the WebEx recording of our kickoff meeting and the PowerPoint presentation that went along with it.  You can also contact me (Bob Carpenter) or John Fritz directly.

Almost everyone has found themselves in a position where they need to ask a question, but aren’t sure who they should ask or how to ask it.  It can be frustrating (and inefficient) to ask several people for the same thing with the hope that one of them can help.

The Analytics and Data Science team, IRADS, and the REX team have made some simple changes to take the confusion out of asking for help with reports, analytics, or data requests.  We’ve added links to the sites people generally go to find data.  Look for a link that says “REQUEST HELP WITH DATA, REPORTS, OR ANALYTICS.”   Clicking on that link takes you to a page which helps you to refine your request and to make sure the proper people see it.  The requests become part of the familiar campus RT ticketing system.

Here’s what happens when you submit a ticket.  Your request automatically becomes part of a queue reviewed by the Data Warehouse Group at its regular Friday meetings.  At that meeting, your request is triaged, the appropriate member of the group is assigned to be responsible for it, and a due date is assigned.  It will be tracked until it’s closed.  

Not sure exactly what to ask?  Look for my favorite link “None of the above fit or I don’t know how to ask for what I need”   Our teams will work with you to help you define your question and help you get an answer.  

The enhanced analytics, data, and report request system takes the worry about who to ask off your plate, and the RT ticket you generate takes away the worry about your request slipping through the cracks or being lost in an email inbox.

In this first of a series of updates, we’ll give a description of what HelioCampus (Helio, for short) is and why we’re investing in it, provide an overview of UMBC’s goals for the project, and summarize what we’re currently working on and what’s next.

HelioCampus was established as an independent company after being formed at UMGC. It provides a data analytics platform covering the student lifecycle, institutional finance, and advancement.  Designed to run using Amazon Web Services, Helio provides UMBC with the next-generation decision support services we need to operate more strategically and sustainably (from a financial perspective).  

Put simply, Helio integrates our enterprise data warehouse with advanced reporting and visualization tools.  With it, we’ll be better able to share information from our systems and answer key questions through analytics, data storytelling, and data science and do it easier and faster.  Helio’s a key component of an integrated infrastructure for academic planning, assessment, and analytics and it helps to support our strategic plan, strategic enrollment plan, and the recommendations of our Middle States accreditation Self-Study.  

Our goals for Helio are to broaden and deepen our use of analytics to increase enrollment, improve student success, shorten time to degree, ensure financial sustainability, and accelerate gift giving...all key elements of UMBC’s plans to weather the current environment and position ourselves for growth and success going forward.  

Helio uses Tableau visualization software.  The first set of visualizations and tools, often called dashboards, has already been deployed to a small group of campus users to allow them to test the platform.  These dashboards focus on metrics related to admissions, enrollment, retention, and graduation.  People using Tableau have had very positive reactions to its ease of use and power.

In the inaugural episode of the Educause Exchange Podcast, Associate Provost and Deputy CIO Bob Carpenter contributes his views on how UMBC should use analytics and predictive models to support students.  In this episode, entitled The Ethical Issues Around Learning Analytics(10 minutes long), he argues that we must not use learning analytics to foreclose opportunities to students.  We must act as an agent for the student, supporting them in the pursuit of their educational goals.  A student’s “grit” matters for their success; our models won’t always capture it.

Blackboard Predict (BB Predict) is a subscription-based predictive model designed to identify students at risk of receiving a grade of D or F in a course.  UMBC has a long partnership with Blackboard and had an active advisory role in the development of this tool.  While the product produced was high quality, it has been lightly used (although more recently DoIT has partnered with individual faculty and staff to explore ways to use it as part of our student success efforts and as a complement to our First Year Intervention Alert Program).  

Given current usage and financial constraints, we have made the decision to sunset BB Predict and will not renew our subscription.

One factor that limited the usefulness of BB Predict to improve student success through better early alerts is simple.  The model’s predictions only become precise enough to use with confidence at the semester's midpoint and are not “early enough.”  Recognizing that the earlier at-risk students can be identified with precision, the more time we have to help them and the greater the likelihood of their success, the Data Science Team within DoIT and supported by the Provost’s office constructed our own model, now called UMBC Predict.  The team’s goal was to increase the precision of the model’s estimates earlier in the semester, in this case, four weeks from the first day of classes, to allow for greater opportunities for students to seek assistance or change their approach to the class, and for these changes or assistance to have an impact.

UMBC Predict does not attempt to replicate the model and platform of our vendor partner.  The strategy employed by the Data Science Team was to focus on fewer courses in the initial stages of the work.  In particular, the team focused on courses with elevated DFW rates, large numbers of students, and importantly, courses where instructors make relatively heavy use of our learning management system (Blackboard) to engage students and to record assignments and grades throughout the semester.  It has always been true that instructors’ use of the learning management system provides valuable information to students about their progress in the course, but it is also the case that these interactions provide important information that the predictive model can use to sharpen predictions at an earlier point in the semester.

A second important factor underlying our decision to sunset BB Predict is pragmatic and necessary.  With the current pandemic and its associated negative effects upon the state budget, reducing our expenditure on a subscription-based model in favor of an internal model with equivalent or better performance for certain courses allows us to maintain our support for students while freeing up resources to support people and programs.

UMBC Predict is already being integrated into department-level enhanced early alert programs on a pilot basis.  This term, faculty in the Mathematics and Statistics Department and the Dean’s Office in the College of Mathematical and Natural Sciences plan to partner with DoIT and the Analytics and Business Intelligence Group to use targeted communications to students who may be at risk after four weeks.  The communication, delivered over the course instructor’s signature, encourages students to take advantage of their office hours along with other tips developed by the Academic Success Center to improve students’ chances of success both in the course and at UMBC.

We in DoIT know that numbers alone can’t capture the time and effort UMBC students, faculty and staff have put into coping with the disruption caused by the Coronavirus global pandemic. However, after the start of campus-wide online instruction last week, we wanted to provide a “bird’s eye view” of the various systems and tools that now make up everyone’s virtual UMBC.

For example, consider the following changes we’re seeing in just a few related tools (Collaborate, Webex and Panopto) compared to the first week of March, when “academic continuity” and “business continuity” were still becoming common terms like “social distancing.”

In terms of campus Blackboard usage, UMBC already started from a strong position. Apart from averaging 25k total logins per day last week, prior research has shown that UMBC’s Blackboard usage is higher compared to most institutions. Obviously, this reflects the hard work and dedication of faculty (especially preparing for last week), but we also want to extend special thanks to faculty serving as Academic Continuity “Ambassadors.” Through a brief survey last week, we learned Ambassadors have already provided support to more than 100 colleagues and peers, primarily through one-to-one direct assistance. The most popular topics include Collaborate, Assessments and Panopto. 

DoIT’s Instructional Technology & New Media group has also seen similar high volume interest in these topics. To date, we’ve hosted 216 and 194 participants in virtual “drop ins” and “pop up” focused training workshops, respectively. For a glimpse of what these sessions are like, check out this brief (3 min) clip of DoIT's Collin Sullivan demonstrating how to use OneNote in Blackboard for shared annotations, taken from a recorded virtual drop-in workshop for the Math Department last Friday, March 27. In addition to several DoIT ITNM staff responding to a robust "back channel" chat, the workshop was attended by 16 math faculty. Special thanks again to Math faculty member (and Academic Continuity Ambassador) Liz Stanwyck and Department Chair Animikh Biswas for the invitation and support. More info, including new evening drop-ins twice a week, are available on the DoIT Academic Continuity and Training sites.

UMBC’s Frequently Asked Questions (FAQ) knowledge base has also increased 45% from March 2019 (66,154 pageviews) vs. March 2020 (95,974 pageviews). By contrast, phone calls and Request Tracker (RT) ticket volume for DoIT’s Technology Support Center (TSC) have been relatively flat compared to FAQ usage. Tip: See the FAQs about FAQs, including why campus support offices and staff should use the FAQs to help users help themselves.

Beyond everyone's continued health and safety, we know there is obvious concern with how well students are able to make the transition to online learning. We know there are challenges, but if you haven’t seen it already, please check out (and share) DoIT’s tips for students about getting online, getting a computer, and getting software. 

In particular, in terms of software, we’ve just added a faster, more user friendly Virtual Lab Environment for all students and faculty to use for classes and instruction. The new system allows students to use all University software without having to download or install those applications on their own computers. It leverages UMBC’s computing resources, and can be accessed from any type of device, including Chromebooks, tablets, mobile devices or regular computers, making it easier than ever for students to get access to the tools they need for their classes. Instructions for accessing the system can be found at umbc.edu/go/vde-faq. 

To help faculty quickly determine if, and how frequently, students are keeping up during the move to campus-wide online instruction, DoIT’s Analytics and Business Intelligence group has also recently provided new Report Exchange (REX) reports focused on days since last login, dependence on mobile devices, and “Student at a Glance” reports showing patterns and trends across all courses a student is enrolled in that use Blackboard (Reminder: accessing REX reports from off campus requires logging in with the Virtual Private Network (VPN) to emulate an on-campus connection). Similar, more detailed reports the new reports are derived from are also available inside every Blackboard course. 

Finally, as more of us move to work and life online, DoIT’s IT Security group has provided important updates about such topics as home network security, telework security resources, and the rise in COVID-19-related scams, including insurance offers, unexpected calendar invites, and fake Docusign phishing messages. Similarly, DoIT’s Business Systems Group (BSG) made sure Human Resources has what they need to track leave, process payroll and make sure employees still get paid. BSG also quickly developed and added Docusign exception request forms so researchers can come to campus to maintain facilities and experiments, and students can seek approval of applied learning experiences.