Setting Exams with Proctorio and Webwork

A few months ago, I wrote an article on the strategies I used to respond to four challenges I faced while preparing a second-year engineering course to be delivered online. My main role was to manage the delivery of tutorial sessions and test invigilations. Upon revising the course plan, we ended up with one 30 minutes quiz every other week and one homework per week.

Webwork is an online homework system, which supports student learning by providing immediate feedback to the students and removing unnecessary marking duties from the instructor’s and TAs’ shoulders. Instructors can link the questions to Canvas and use the Webwork tools to randomize mathematical questions or get insights on the performance of students on each assignment. By randomizing the questions, students cannot just tell a peer the final answer, but they have to explain their method. The system supports multiple attempts and with unlimited attempts, the students can make as many attempts as they like before the deadline, which makes Webwork part of the learning process. Webwork has a structure called Open Problem Library (OPL) for sharing problems and we used an extension of OPL to develop the questions for the assignments. The list of questions is available for free on Github in one of the projects related to the MECH 2 course at UBC (UBC Engineering Webwork). You can find more information on this project pursued by three UBC Engineering faculties here.

Assessment is an important part of the learning process and preparing proper assessment is much more difficult in a remote environment. Short quizzes can give feedback to both students and teachers on their learning and teaching effectiveness, respectively. My role as a senior teaching assistant was to use the available TA resources to create an inclusive environment for students and remove the elements that influence the likelihood of academic misconduct.

The academic misconduct model argues that removing the academic pressure, misconduct opportunity, and rationalization path can reduce the likelihood of academic misconduct. while online assessment provides additional flexibility to use new types of assessments, it also creates new challenges for students. Different timezones, access to a reliable internet connection, and the requirements of online proctoring tools can increase the anxiety of students during the assessment.

In order to reduce the anxiety of students as well as the elements of misconduct, I booked consultation sessions with UBC CTLT and MECH 2 coordinator at UBC. Since all choices had some trade-offs between the above-mentioned aspects, I drew the line at inclusive access of students to the assessments. We ended up with three sessions for each assessment. The first session was monitored by Proctorio, eliminating the need for an active TA to monitor the session. We planned a simultaneous session, which was monitored on Zoom since Proctorio was not always stable and some students had difficulties accessing the service. Also, a third session was available for students in special time zones. This session was also monitored on Zoom.

In order to eliminate the chance of duplicate trials for quizzes and have all information in one place, I came up with the idea to use Webwork for publishing quizzes. But this idea introduced a new difficulty. Webwork cannot be linked to exams on Canvas and only available in the assignment section. Also, you cannot use Proctorio to monitor assignments on Canvas. Thus, we created each exam as an assignment and protected the Webwork link behind a Proctorio screen. While this increased the steps to create a new assessment, it did not complicate the assessment procedure for the students significantly.

Proctorio and Webwork were new tools to our students. So, we did not expect them to be able to use them properly in their first trial. Thus, we made two practice tests for them, that had no effect on their final mark, to get used to the new technologies. Also, we made a video that described all steps to install and enable Proctorio on their system and take the exams. Personally, I am not a fan of Proctorio, but it was our only option based on the form of the quizzes used in this course. Since our quizzes were in the form of traditional Pen and Paper problems, students had to submit their written answers on Canvas after the end of te quiz. We also asked them to enter their final answer on Webwork and do not change it after closing the Webwork session. We also used Webwork for keeping track of time.

Later, we developed the following exam procedures, which you can use as templates for your own assessments:

• You have 30 minutes for Quiz 05.
• You do not need to join the ZOOM session for a tutorial before your quizzes.
• Use the Chrome browser. Make sure Proctorio is installed on your browser and permissions are given to Chrome for sharing video, audio, and screen.
• All quizzes will open on CANVAS 10 minutes before {Time of Exam}. You have 10 minutes to run pre-exam checks. The questions on Webwork will be available EXACTLY at {Time of Exam}. You will see an error if you open Webwork before {Time of Exam}.
• You will have 10 minutes after the end of the exam to upload your work on CANVAS. Use “{Assessment Name}” for uploading the scan of your exam. It is available in Modules.
• You are expected to keep Proctorio open during your exam until you click “Grade Test” on Webwork. When you clicked “grade test”, stop working on your exam and submit the scan of your exam on CANVAS. Do not leave any Webwork field blank at the end of your exam.
• If Proctorio asks for an access code, Click on Secure Exam Proctor on the right menu on Canvas. If that does not work, log out of Canvas and log in.
• If you face problems with Proctorio or lost access to Webwork, join the ZOOM session for this tutorial. We will help you to continue your exam.
• Webwork closes if you hit “Grade Test” anytime during the exam and you will lose access to the questions. Just hit “Grade Test” after you are done with the whole exam.
• Do not leave any Webwork field blank at the end of your exam. Do not change the final answers after you click “Grade Test”. Enter “0” in the blank fields if you could not find the final answers to receive marks for the rest of your solution.
• When you are uploading the scan of your quiz, please use the following name convention for the file name: Firstname_lastname_studentnumber_quiz05.pdf

Students will definitely face problems in different steps of this process, especially during the first few quizzes. It is necessary to keep a troubleshooting Zoom session during the exam to help students with those issues. Also, I found it really helpful to make a Google Spreadsheet and ask students to report their problems in that instead of writing them on Zoom. This way, we could review the problems and improve the process for the next sessions.

For the final exam, We also used Webwork to randomize the order of questions. Send me an email if you need more information about this process or if you have used efficient ways of assessment. I am interested to know about your methods.