Students’ Experience of Online University Education During the COVID-19 Pandemic: Relationships to Psychological Health

The COVID-19 pandemic resulted in the complete closure of many university campuses and a rapid shift to complete online delivery of university teaching.  Understanding the student experience of online learning under these conditions is important to inform improvements and adaptations to continued online delivery of university services. The aim of this study was to examine students’ experience of online learning, studying, and assessment during the pandemic and investigate the association between these perceptions and measures of psychological health. A cohort of 391 undergraduate students completed measures of stress, anxiety and psychological wellbeing at the beginning and end of a semester during which a shift to complete remote delivery of university teaching occurred due to COVID-19 restrictions.  Students also rated how stressful or difficult they found five aspects of online learning, study and assessment which was used to calculate a composite score. T-tests were used to compare stress, anxiety and psychological wellbeing scores from the beginning and end of the semester.  Regression models were used to examine the relationship between online learning composite score and psychological health measures. Unexpectedly, stress and psychological wellbeing improved over the semester during which the university campus closed, and all teaching and assessments were online. Students with higher stress scores and lower psychological wellbeing scores at the beginning of the semester were more likely to experience difficulty and stress with online teaching. 
 


Introduction
Tertiary students are at a pivotal stage in their lives as they seek to direct their future by educating themselves, often at a time of life transition and movement toward increased independence. Understanding the student experience of the rapid shift to online education delivery resulting from measures to limit the spread of COVID-19 and its relationship with psychological health is important because this has implications for student wellbeing and academic engagement.
The term online learning before the COVID-19 pandemic referred to a spectrum of possible education delivery models that ranged from all course material being in electronic format for students to watch, read or download to exchange of emails. Government restrictions to limit the spread of COVID-19 resulted in the closure of university campuses requiring teaching and assessment to be remotely delivered. A period of rapid adjustment for both teaching staff and students ensued in which delivery of written coursework, lectures, tutorials, laboratory teaching, student support services, and examinations were moved off campus for students to access remotely.
Online delivery of tertiary education is not new. Previous research has investigated the components of successful online teaching, focusing on the social aspects of learning and student participation in the learning process. Facilitating active student The COVID-19 pandemic resulted in the complete closure of many university campuses and a rapid shift to complete online delivery of university teaching. Understanding the student experience of online learning under these conditions is important to inform improvements and adaptations to continued online delivery of university services. The aim of this study was to examine students' experience of online learning, studying, and assessment during the pandemic and investigate the association between these perceptions and measures of psychological health. A cohort of 391 undergraduate students completed measures of stress, anxiety and psychological wellbeing at the beginning and end of a semester during which a shift to complete remote delivery of university teaching occurred due to COVID-19 restrictions. Students also rated how stressful or difficult they found five aspects of online learning, study and assessment which was used to calculate a composite score. T-tests were used to compare stress, anxiety and psychological wellbeing scores from the beginning and end of the semester. Regression models were used to examine the relationship between online learning composite score and psychological health measures. Unexpectedly, stress and psychological wellbeing improved over the semester during which the university campus closed, and all teaching and assessments were online. Students with higher stress scores and lower psychological wellbeing scores at the beginning of the semester were more likely to experience difficulty and stress with online teaching. participation through course design and delivery, creating opportunities for dialogue and interaction between teachers and students, and providing context and stimulating thought have long been identified as central to a successful online course (Harasim, 1995;Rainbow & Sadler-Smith, 2003). In a recent qualitative study, students and faculty staff reflected a desire for a personalised learning approach while maintaining the social aspects of distance learning (Shearer et al., 2020).
Closure of university campuses due to stay-at-home orders presented several novel contextual factors, including the shift occurring at a time of heightened stress and anxiety about the pandemic. While familiar with online resources as an adjunctive to in-person, many students and staff were not anticipating a complete and rapid shift from on-campus learning to exclusively remote delivery of university services. Students could no longer access the internet, computers, or study spaces on campus and were required to adjust to potentially different study methods with a greater need for learning progress to be independently managed in the absence of the external structure offered by on-campus teaching. Understanding the student experience of this shift to online education delivery and identifying students who may find this transition more problematic is of interest due to the mental health implications of university students' stress. Pre-pandemic studies reported an association between poor psychological health in university students and lowered academic performance and later mental health problems (Baker, 2003;Kötter et al., 2017;Sohail, 2013).
Traditionally, university examinations and assessments have been periods associated with a transient increase in stress and decreased psychological wellbeing in students (Pitt et al., 2018;Surtees et al., 2002;Takada et al., 2016). Rapid curriculum redevelopment to shift to a completely online service impacted the delivery of teaching and assessment of students. Closed book, invigilated examinations, and tests were no longer able to proceed on campus, and a move to open book, open web uninvigilated assessment was required. These exams tend to focus more on applying knowledge to problems rather than testing stored knowledge. They recognise that students have access to various resources during the examination (Herrington et al., 2004;Williams & Wong, 2009). A Finnish study found that students spent equivalent time preparing for online and inperson exams but reported taking longer to complete online examinations (Myyry & Joutsenvirta, 2015). While studies show students feel less anxious and stressed about online examinations (Greenberg et al., 2009;Schmidt et al., 2009), other studies indicate that not all students experience online assessment as less stressful. Yang and Taylor (2013) found that high performance avoidance increased anxiety about online examinations indicating that students do not experience online examinations in the same way (Yang & Taylor, 2013).
Studies in the general population have reported that rates of depression, anxiety, and stress increased during the initial lockdown period in response to the COVID-19 pandemic. Three weeks into the strict lockdown imposed in Italy in March 2020, high rates were found of severe depression symptoms (17.5%), severe anxiety (20.8%), and perceived stress (17.5%) (Rossi et al., 2020). Research on the mental health of university students during the first half of 2020 found mixed results. In a sample of Chinese medical college students, 0.9% reported severe anxiety symptoms, and 2.7% reported moderate symptoms. Disruption to study was significantly associated with increased severity of anxiety in this sample (Cao et al., 2020). A study of 195 students from a public university in the United States found that 71% reported increased stress and anxiety due to the COVID-19 pandemic (Son et al., 2020). Concern about personal health and the health of family members correlated with increased anxiety in both the Chinese and American studies. By contrast, Capone et al. (2020) found that levels of academic stress in 1,120 Italian university students were not significantly different from pre-pandemic levels. They reported that 22% of their sample were flourishing and that levels of psychological distress were not significantly different from the normative sample of young adults assessed pre-pandemic. Self-reported help-seeking was associated with better psychological health (Capone et al., 2020).
The current study aimed to: • Examine the psychological health of university students at the beginning and end of a semester impacted by COVID-19 • Examine students' experience of the shift to online learning and assessment • Determine the relationship between psychological health and experience of online learning and assessment.

Participants
Participants in this study were 483 undergraduate students enrolled at the University of Auckland, New Zealand.

Data Collection
The New Zealand university year begins in March and ends in November, with two semesters each comprising of 12 teaching weeks in each semester. A full-time course of study would typically involve taking four papers or courses e.g. Introduction to Psychological Theories, in each semester of a calendar year. Students participating in this study examining psychological health were undergraduates enrolled in the first semester of 2020..
In New Zealand, the first semester of 2020 teaching at the University of Auckland began on March 2 2020. At that time, the COVID-19 global pandemic was present in international news but had not emerged at a local New Zealand level. As concern began to rise about the potential impact of the pandemic, three weeks into the semester, the University of Auckland instituted a one week teaching-free period (23-27 March 2020). This period allowed university staff time to make curriculum adjustments to provide remote delivery of teaching, given the increasing likelihood that COVID-19 management strategies could interrupt on-campus activities. On 23 March 2020, at the beginning of the teaching free week, the New Zealand Government announced an alert Level 4 lockdown would come into force on Wednesday, 25 March. The lockdown required people to stay at their place of residence with three exceptions: Essential service workers, e.g., those in healthcare practices could travel to and from their workplace; leaving home to access medical care or buy food, and; leaving home for personal exercise while maintaining physical distance from those not in their household.
Following the Level 4 lockdown announcement, many students returned to their family homes. The University of Auckland subsequently decided to keep all delivery of teaching and assessment of students in remote mode for the entirety of the first semester to provide students with some certainty and to avoid a situation where, after reopening, the university would close again at short notice.
Students who participated in this study received on-campus teaching three weeks before the University of Auckland closed the campuses and moved to a complete remote delivery model. A secure online database managed all consent and data collection, and students could give consent and answer questionnaires using their phone, tablet, or computer. Brief study information was given to students at the beginning of a lecture in the first three weeks of the semester and also via placement of the participant information sheet on the online resource page for the individual courses. Students answered questions about stress, anxiety, and psychological wellbeing at the beginning of the semester, and the end of the semester in the period after teaching had finished and before the commencement of the end of semester examinations. The majority of participants (89.9%) completed the baseline psychological health questions at the beginning of the semester prior to the lockdown and forced closure of the university. Recruitment for the study continued online for three weeks following closure of the university and 10.1% of participants registered after the lockdown.

Demographic Data
Students provided demographic information at the time of registration for participation in the study, including sex, age, and ethnicity. In New Zealand, it is common for people to identify with more than one ethnic group. A system of prioritised ethnicity classifies individuals who identify with more than one ethnicity according to the following priority: Māori, Pacific, Asian, European, MELAA (Middle Eastern, Latin American or African) and other.
All participating students were undergraduates. Students were divided into two groups: those enrolled in a stage 1 paper or those enrolled in either Stage 2 or Stage 3 papers.
The primary course of study for participants was categorised as Medicine, Medical Science, Population Health, Psychology, or Other.

Stress.
The Perceived Stress Scale is a 10 item questionnaire that asks about stress and coping in the previous month (Cohen et al., 1983). Scores range from 0 to 40, with higher scores being indicative of higher levels of stress. Scores from 0-13 represent low stress, scores from 14-26 equate to moderate stress, and scores from 27-40 equate to high stress.

Anxiety.
The State Trait Anxiety Inventory 6 item version (STAI6) is a short 6 item scale validated as an anxiety screening questionnaire based on the more extended State Trait Anxiety Inventory, which is 20 items (Marteau & Bekker, 1992).

Psychological Wellbeing.
The World Health Organisation wellbeing index, the WHO-5, is a five-item, positively worded measure of psychological wellbeing, which gives scores ranging from 0 to 25. Higher scores represent better wellbeing. Scores of 13 or lower indicate low levels of psychological wellbeing. A systematic review of the WHO-5 concluded that it was a widely used and sensitive measure of depression (Topp et al., 2015).

Questions About Online Learning, Studying, and Assessment
In addition to completing the psychological measures again at the end of the study, students answered five questions about online university education.
• How stressed are you feeling about your university exams being online?
• How stressful have you found online learning (lectures/tutorials) this semester?
• How stressful have you found studying at home this semester?
For these three questions, students ranked their answers on a scale from 0 to 3 (0=not very stressed, 1=a bit stressed, 2=stressed, 3=very stressed) using the same 0-3 point scale (0=not very difficult, 1=a bit difficult, 2=difficult, 3=very difficult). Students were then asked: • How difficult have you found it to motivate yourself to learn at home?
• How difficult have you found not being able to talk to other students on campus about your studies?
A composite score was calculated for each student from their answers to the five questions, ranging from 0 to 15.

Ethics
The University of Auckland Human Participants Ethics Committee approved the research (Reference ID: 023964).

Statistical Analysis
Statistical analyses were performed in SAS 9.4. Two sample t-tests assessed whether respondents and non-respondents differed in baseline stress, anxiety, or psychological wellbeing score. Chi-square tests tested whether respondents and nonrespondents differed in sex, ethnicity, year of undergraduate study, or study paper.
Changes in stress, anxiety, and psychological wellbeing between baseline and the end of the study (T2) were calculated by subtracting the score at baseline from the T2 score and analysed using paired sample t-tests.
Linear regression models were used to analyse the association between the online learning composite score and baseline stress, anxiety, and wellbeing scores with adjusted analysis considering sex and ethnicity.

Results
Of the 483 participants who answered questions at baseline, 391 (81.0%) completed the psychological questionnaires at the end of the university semester before examinations. Respondents did not differ from non-respondents in initial stress scores (p=0.997), anxiety scores (p=0.81) or wellbeing scores (p=0.87). Similarly, there was no significant difference between respondents and non-respondents in sex (p=0.91), ethnicity (p=0.51), study course (p=0.65) or undergraduate stage of study (Stage 1 or later) (p=0.42).
Stress, and psychological wellbeing scores significantly improved from the beginning of the semester to just before examinations, while anxiety scores increased (Table 1).  Table 2 shows the number and percentage of students who endorsed each response for the five questions about online learning, study, and assessment. Forty eight percent of the students reported finding the motivation to study very difficult. For each of the other four questions about online learning and assessment, approximately half the group endorsed the answers not stressful/difficult or a bit stressful/difficult. The composite score created from the five individual questions about online learning, study, and assessment (range 0-15) had a mean of 8.0 (SD=3.7). The composite score was significantly associated with baseline stress, anxiety, and psychological wellbeing (Table 3). For every 1 point increase in baseline stress score, there was a 0.23 (95%CI: 0.17, 0.28) point increase in online learning composite score in adjusted analysis. For every 1 point increase (indicating improvement) in psychological wellbeing score, there was a 0.30 (95%CI: 0.38, 0.22) point decrease in how stressful students found online learning using the composite learning score.   Table 4 shows the mean baseline stress score according to the student ratings of online learning. In each case where students rated an aspect of online learning as very stressful or very difficult, they had higher baseline stress scores than students who rated aspects of online learning as not stressful or difficult.

Discussion
In this cohort of university students, we found psychological health was better at the end of a semester during which the COVID-19 pandemic forced the closure of the university campus and a complete shift to online teaching and assessment. The pandemic is often associated with increases in stress and anxiety. Our findings are consistent with previous research demonstrating that academic stress in university students may not be significantly different from pre-pandemic levels (Capone et al., 2020). In addition, the move to online examinations may have reduced stress. Previous studies have reported that, in general, students experience open-book, uninvigilated examinations as less stressful (Greenberg et al., 2009;Schmidt et al., 2009).
Students may have had higher than typical stress, anxiety, and lower psychological wellbeing scores at the beginning of the semester due to the uncertainty about the global emergence of COVID-19. However, two lines of evidence suggest that this is less likely. Firstly, a pilot study conducted at the same university during a semester in 2019 found similarly high mean stress, anxiety, and poor wellbeing scores at the beginning of the semester. In the pilot study, the mean scores were: stress (21.2), anxiety (55.0), and psychological wellbeing (13.1). In the current study, baseline scores were: stress (20.9), anxiety (54.3), and psychological wellbeing (12.9). Further, when we compared the stress, anxiety, and psychological wellbeing scores at baseline of the 434 (89.9%) of students who completed these measures before the New Zealand Government announcement of a Level 4 lockdown and the closure of the university to those who completed baseline questions after this time (n=49, 10.1%), there was no significant difference between those who registered before and after the lockdown.
Previous studies of student perception of online learning during the COVID-19 pandemic have found that university students prefer on-campus conventional teaching and learning. A survey of medical and dentistry students from a private Pakistani university found that 77% did not enjoy online learning (Abbasi et al., 2000). Beltekin and Kuyulu (2020) surveyed student perceptions of distance learning compared with in-person teaching in a sample of 455 Turkish university students. They found that participants preferred in-person teaching and that technological difficulties with online teaching reduced student motivation. Similarly, a Pakistani study found that many students reported difficulty accessing material due to hardware limitations and internet access problems (Adnan, 2020). Few studies describe the student experience of the complete shift to online learning during the COVID-19 pandemic, and to our knowledge, none have examined the relationship between student perceptions of online learning and psychological health. Our results suggest that despite an overall improvement in stress, and psychological wellbeing, the experience of online learning is not the same for all students. Those who reported finding online learning, studying, and assessment more stressful or difficult had higher baseline stress and lower baseline psychological wellbeing scores at the beginning of the semester. Routinely assessing psychological wellbeing in university students could assist in tracking overall student wellbeing throughout continued pandemic restrictions. Identifying those who begin a semester with higher levels of stress would allow future intervention trials to target students who may go on to find online learning more difficult during COVID-19 interrupted semesters. More generally, future research should further examine student characteristics that are associated with success in online learning to assist tertiary institutions in developing a tailored approach to student support.
In our cohort, 48% of students reported finding the motivation to study very difficult during the semester when all university teaching and assessment was online. This finding is consistent with the limited number of studies investigating motivation in university students during COVID-19. In the Pakistani study, only 10.3 % of their 126 students reported that online learning was more motivating than conventional learning (Abbasi et al., 2020). Baber (2020) found in a small study of 100 South Korean and Indian university students that motivation was a determinant of perceived learning outcome and satisfaction with learning. The compulsory closure of university campuses forces students to engage in distance learning. Forced distance learning in contrast to online learning by choice, may affect student motivation to learn differently.
There are some potential limitations of our study that need to be acknowledged. The results from our cohort may not reflect the experience of university students in all countries. New Zealand moved from Alert Level 4 (complete lockdown) to Alert Level 1 (no restrictions other than international travel) during the first semester of 2020 and has been one of the most successful countries in containing and eliminating COVID-19. Therefore, the psychological health of students in our study may not be representative of all tertiary students internationally. However, the study by Capone et al. (2020) also reported that Italian university students had similar levels of stress and anxiety during the COVID-19 pandemic to pre-pandemic levels.
The global environment is rapidly changing, and many countries are extending COVID-19 restrictions. Therefore, the psychological health of students and the association with online learning may change over time. Future research should examine the relationship between student wellbeing and engagement with online university education.

Conclusions
We found that student psychological health improved over a semester during which university education delivery shifted entirely to an online format. Difficulties with motivation to study were common and students who found online learning and assessment more stressful had higher baseline stress and lower baseline psychological wellbeing scores. Further research examining the association between psychological health and experience of forced online university education is essential to identify those students who may struggle more during pandemic-imposed distance learning. Our study was conducted during the early months of the COVID-19 pandemic and future research will also be needed to understand student experience of online learning more generally as pandemic restrictions ease and countries emerge into a new normal.