Before the Spring 22 semester started, four Biological Sciences professors -- David Eisenmann, Hua Lu, Jeffrey Leips, and Kevin Omland -- reached out to DoIT to see if they could (and should) try an experiment: proctor their first semester exam in the University Center (UC) Ballroom and online (via Blackboard). 

All four have been teaching in the UC as part of the pandemic-driven retrofits for teaching designed by DoIT and managed by Events & Conference Services (ECS). Also, after teaching remotely for nearly two years, they'd come to like the benefits of online exams other faculty have discovered with Blackboard exam question pools, including randomization, item analysis and quicker student feedback. So, as more classes were offered on campus, the faculty wondered if they could leverage student laptops and mobile devices that made UMBC's pandemic pivot to online learning possible. 

"Initially, our biggest concern was power for student laptops to make it through a 50-minute class period and exam," says Omland, who reached out on behalf of the group, and has co-taught BIOL 142 "Foundations of Biology: Ecology and Evolution" with his colleague Leips for 20 years. "Then, we wondered if the campus WiFI would melt, so we thought maybe we should check with DoIT first." They were joined by colleagues Eisenmann and Lu, who co-teach BIOL 302 "Molecular and General Genetics."

After several emails exploring potential issues -- and two RT tickets to schedule DoIT support on their first exam dates -- the faculty briefed their students and conducted a "dry run" test of the UC's power and WiFi 1-2 weeks before the exams. "We appreciated that step very much," says DoIT's David Toothe, Director of Enterprise Infrastructure Solutions, who leads the AV and Desktop Support units that collaborated with the faculty. It also helped that DoIT's Networks group had just upgraded the UC WiFi as part of a larger campus wireless initiative in January.

Interestingly, each course had similarities and differences in their approach to online exams proctored in the UC:

• In addition to using Bb exam question banks, both courses enroll over 200 students, and planned to use adjacent rooms (301, 310 and 312) for overflow or so students could plug into extra power outlets provided by DoIT, if needed. They also had paper back ups of the exam, in case there were issues that could not be fixed during the test.
• BIOL 302 required the use of Respondus Lockdown Browser (not Respondus Monitor) to access and take the test, and asked students to update their browsers/software before hand. It also uses Bb Original. However, BIOL 142 used an "open note" and "open web" (but not "open neighbor") approach, assigned students to specific UC rooms, and uses Bb Ultra.

Results

During the last week of February, each course administered its first exam of the Spring 22 term. Results included the following (excerpted with permission from emails provided by Eisenmann and Omland):

BIOL 302 "Genetics" (1st SP22 exam on 2/24/22)

• No students reported Wifi connectivity issues to [instructors]
• Several students had problems with LDB but went away upon hard restart.
• ~6 students (out of 200) needed to use outlets, the rest were fine.
• The calculator in the menu bar does not appear for students using LDB on a Chromebook [a known issue DoIT has reported and documented in a related FAQ]. Otherwise no Chromebook or iPad issues.
• 2 students had problems that could not be fixed and took paper copies of a similar exam.
• For a class of [302's] size we would have used 2500 pieces of paper to print this one exam. This way we printed 10% of that number (back up paper copies and scratch paper to give out).
  
  

BIOL 142 "Foundations of Biology" (1st SP22 exam on 2/25/22)

• 4 students needed paper copies of the exam
• One student said he always has trouble connecting in rm 301, so we moved him to 312 where he seemed to have no problem.  
• 5 students used outlets in rm 312.
• The only fairly annoying problem was that one student was unable to submit her exam at the end. [Took photos of each exam page, No idea what happened]
  
  

"After this experience I say we SO need a testing center," says Eisenmann, a sentiment echoed by others on the department's faculty email group after he and others shared their results. "This is the first time I have ever given an exam to 200+ people in 20+ years, online or in-person, and not really worried about cheating. And so much more pleasant for them and us than stadium seating."

Added Omland: "I spoke with about 10 students afterward - 8 said the format was fine, but 2 said they preferred paper exams so they could cross out answers, write on the side. We really appreciated the ability to have students spread out from each other . . . and walk to any part of the room to answer student questions.  Also, two students reported that they were home with covid. Since it is an online Bb exam with open book, open web etc, we just had them take the exam at home at the same time as the other students.  Can't do that with a paper exam!"

Lu also spoke briefly with about 8 students after the 302 exam. "Only one expressed that she was not used to clicking on the screen for an exam but she did not seem to be bothered by it. All students seemed to appreciate the benefit of the format - reducing the use of paper, reducing grading labor (their own words). [We] were also very happy to see that the cheating problems could be largely eliminated/reduced with this format. As I walked around the room, I rarely saw that students next to each other had the same screen. Of course, this will depend on the pool size of a given exam."

DoIT briefly presented the Biology faculty's "experiment" at the February 28 Faculty Senate Computer Policy Committee (CPC), which had been exploring the idea of a UMBC testing center before the pandemic. The biology faculty have also been invited to share their experience at an upcoming Classroom Committee meeting this semester.

By John Fritz

After participating in rigorous peer reviews of their online and hybrid courses, five UMBC faculty have earned Quality Matters certification as part of the campus’ Quality Matters Impact (QMI) pilot. In partnership with the Provost’s Office, the QMI pilot launched in FA2021 as an extension of the award-winningPIVOT program to recognize exemplary course design. 

Faculty participants received feedback from certified Quality Matters peer reviewers at UMBC and other online learning experts in the QM community. The five courses include:

These courses are eligible to display the QM Certification Mark, an internationally recognized symbol of online and blended course design quality. The QM Certification Mark represents these instructors’ ongoing commitment to creating learning environments that provide students with a clear pathway to success. QM Certification is the result of the faculty’s dedication to continuous improvement. 

Promoting quality and lessons learned 

In the first GO Chat of 2022, scheduled for February 25 at 1 PM, Louise Murray and Mary Tabaa will share their experiences with the formal review process and showcase their QM-certified courses. 

The call for participation in the Quality Matters Impact program is open twice a year with reviews scheduled in fall/winter and spring/summer. Using the 6th edition of the QM Higher Education Rubric, a formal QM course review is designed to be a collegial process to provide helpful recommendations for continuous improvement and reinforce effective, research-based practices for online and hybrid course design. 

A QM Official review provides independent validation of quality, learner-focused course design. As of February 2022, only nine courses in the University System of Maryland are formally certified by Quality Matters, which is valid for five years. To maintain QM certification, and be listed on QM’s website, courses may be recertified for an additional three years or officially reviewed again. A course achieves QM Certification when it meets QM Rubric Standards at the 85% threshold or better and meets all Essential Standards.

DoIT’s Instructional Technology team manages the formal review process and facilitates the fully online, asynchronous Applying the Quality Matters Rubric workshop designed to help instructors prepare for a formal course review. When a course is deemed having met QM standards, the instructor is eligible to display the QM Certification Mark on the syllabus or within the course itself. QMI program participants receive a $1,000 professional development award for completing required activities and their course meeting QM standards.  

For more information about the Quality Matters Impact program, please contact the QM coordinators for UMBC:

Dr. Mariann Hawken

Director, Instructional Technology

mariannhawken@umbc.edu

Dr. Susan Biro

Manager, Online Learning

sbiro@umbc.edu