Improving Student Success with Online Embedded Tutor Support in First-Year Subjects. A Practice Report

Institutional commitment to the student experience in the early stages of university has the greatest potential to exceed student expectations. The cross-institutional Embedded Tutors Program provides undergraduate students with access to subject content experts from 12 first-year subjects across a regional university. Tutors provide one-on-one draft assessment feedback in subjects with a large written assessment task. A total of 428 students attended 615 tutor sessions held online via Zoom. Students who met with a tutor had higher average assessment marks and cumulative subject marks than those students who did not attend a session. Feedback from students was positive, with 78% of students rating the tutor session “extremely helpful.” This study provides evidence of the value of subject-specific draft assessment feedback for students in first-year subjects.


Introduction
More than 2000 years ago, Alexander the Great had a personal tutor; his name was Aristotle.Though one-on-one support is not a new concept, a lot has changed since 300 BC.The pedagogical value of tutoring is now well established, and technology has evolved, allowing the ancient practice of one-on-one tuition to become more widely available.However, as university budgets around the world tighten (Hurley, 2020;Kelly & Columbus, 2020), it is often the resource-heavy teaching experiences that are the first to be cancelled.This practice report explores the value of a cross-institutional Embedded Tutors Program at a large regional Australian university.The Program supports first-year students and leverages access to technologies, from online scheduling systems to online delivery of the tuition.
Universities worldwide are looking for ways to improve student performance outcomes including student experience, student success and graduate employment outcomes.There is a wide body of literature describing best practice transition pedagogy, which promotes an educational framework to increase student engagement through acknowledgement of the multi-faceted influences on the first-year student experience (Kift, 2015;Nelson et al., 2012).Determinants of student engagement and success include access to resources, a sense of community and belonging, and preparedness to study.Therefore, supporting students requires a coordinated, whole of institution support.Lizzio (2006) proposed the Five-Senses of Success framework which proposes that developing a sense of capability, purpose, identity, resourcefulness and connectedness are key factors in predicting student satisfaction and progression.More recently, Kahu and Nelson (2018) outline four constructs within the student engagement framework "self-efficacy, emotions, belonging and well-being" as being "critical" for student engagement and success (p.58).The themes the Embedded Tutors Program speaks most to are a sense of connection and belonging that is nurtured through meaningful interactions with students.Providing tutor support in the transition to university will likely translate to self-efficacy where students have developed a sense of both capability and resourcefulness towards studying at the university level.
Institutional commitment to the student experience in the early stages of university has the greatest potential to exceed student expectations.The cross-institutional Embedded Tutors Program provides undergraduate students with access to subject content experts from 12 first-year subjects across a regional university.Tutors provide one-on-one draft assessment feedback in subjects with a large written assessment task.A total of 428 students attended 615 tutor sessions held online via Zoom.Students who met with a tutor had higher average assessment marks and cumulative subject marks than those students who did not attend a session.Feedback from students was positive, with 78% of students rating the tutor session "extremely helpful."This study provides evidence of the value of subject-specific draft assessment feedback for students in first-year subjects.
Since 2020, the COVID-19 pandemic has impacted student experiences at universities worldwide and the rapid move to online learning undertaken by most institutions fast-tracked the use of technology in ways not seen before (Dhawan, 2020;Smith, 2021).Online meeting tools such as Zoom, Webex and Microsoft Teams have been used at scale for all manner of learning and teaching activities from lectures and tutorials (Dhawan, 2020) to practical classes and exam invigilation (Diaz et al., 2021).This does not come without some risk; a lack of student engagement has been a major concern with moving online (Martin, 2020).Attrition is higher in completely online courses than in comparable courses in which at least part of the course is undertaken on campus (Cupitt, 2015;HESP, 2018).Students often find it harder to connect with peers where there is an online or blended delivery of subject content.The divide is greater for students from non-traditional backgrounds who are less likely to have access to key study resources and an appropriate workspace at home (Sanderson et al., 2021).This is compounded by the individual experiences of students throughout the COVID-19 pandemic such as changes to work and caring responsibilities.However, it is not all bad news; online meeting tools can provide flexibility and facilitate connection across the university.
Universities have the potential to increase student retention and success by providing opportunities for increased engagement for students from diverse backgrounds.Programs to increase student engagement and first-year experience model different approaches, this includes learning hubs, mentor programs, and peer-assisted study programs in which experienced students provide support to commencing or undergraduate students.One-on-one tutoring has shown success in supporting Australian First Nations students at university (Lydster & Murray, 2019;Nakata et al., 2019).Academic tutors and increased online engagement have been shown to play a key role in the retention of students, particularly those from non-traditional backgrounds who are often the most affected by the shift to online study (Douglas, 2020;Kelly & Columbus, 2020).However, one-on-one tutor support is not often embedded in subjects across an institution at scale.To ensure that programs, such as those that offer tutorial support are not impacted by university budget cuts, it is important to establish the value that they have on student success.
The cross-faculty Embedded Tutors Program was a new initiative in 2021 and started as a successful pilot in three subjects in semester 1 in which the embedded tutors provided online one-on-one feedback on a draft of a large, written assessment (Linden, 2022).The aim of the expanded program reported here was to provide timely, subject-specific support for first-year students in 12 undergraduate subjects to improve student success.We hypothesise that subject-specific tutor support will increase assessment marks and subject grades.

Subject Selection and Tutor Support
At Charles Sturt University, Australia, the three main teaching sessions run from March to June (semester 1), July to October (semester 2) and November to February (semester 3).An applied, mixed-method research study was performed across three faculties at this large regional university.A total of 2386 students (an average of 199 students per subject) were enrolled in the 12 first-year undergraduate subjects that were selected to participate in this study in semester 2 of 2021.All 12 subjects included an online offering, and 10 subjects with on-campus offerings were taught across all six main regional campuses.All subjects had a high proportion of commencing students and participated in a larger, university-wide student retention project (Linden, 2022).Early in semester 2, the on-campus subject offerings were moved online due to an outbreak of COVID-19.
One-on-one tutor support was offered to students in preparing at least one assessment item.These assessments had a value between 15% and 60%.Online meeting software (Zoom) was used to facilitate the online tutor sessions.The bookings were made using Calendly (Calendly LL, Atlanta, Georgia), an online scheduling tool that was embedded within the subject learning management system (LMS) site (Blackboard).Students were required to book a meeting and email a draft assessment before the booked time as previously described (Linden, 2021).The Embedded Tutors Program was promoted through announcements made by Subject Coordinators on the subject site and/or in class time.
Tutors were generally recruited (n=36) by the Subject Coordinator.Tutors were required to have a minimum level of understanding of the subject content and/or familiarity with assessment items as appropriate for each subject.All tutors were required to attend introductory meetings with the Retention Team in which they participated in formal training around the use of Zoom, Calendly and LMS subject sites.Tutors also met with the Subject Coordinator to discuss specific information about the assessment or subject where appropriate.

Subject Mark Analysis
Marks for individual assessment items and cumulative subject marks (out of 100) that had been entered in the LMS grading platform (Grade Centre) were downloaded and analysed.Students who were enrolled but did not submit an assessment item valued at 20% or higher were excluded from the analysis (n=361).These students often did not submit any assessment items, and could therefore be described as non-participating enrolments, or ghost students (Stephenson, 2021).To include these enrolments would have reduced the average mark of the group of students that did not meet with a tutor.

Evaluation of Draft Assessment Feedback
Three subjects had tutor support available for assessment 1.In the remaining nine subjects, tutors were available for assessment 2 and/or assessment 3.In these subjects, marks in assessment 1 were used to compare the students' grades before and after meeting with a tutor.Assessment performance was analysed in seven subjects that included tutor support in an assessment item and that did not have tutor support available for the final assessment item to determine if meeting with a tutor for an earlier assessment item could help in the performance of later assessments.Analysis of individual assessment items is expressed as a percentage.

Students from Non-Traditional Backgrounds
Charles Sturt University is one of the largest providers of online and blended learning in Australia and in 2021 had approximately 31,000 students enrolled online and 16,000 on campus students, with a high proportion of students from nontraditional backgrounds.Of the 2386 students, 5% were Australian First Nations students, 25% were from postcodes with a low socioeconomic status (SES), 58% were from a rural, regional and remote (RRR) background and 54% of students were recorded as the First-in-Family to attend university.While students were not targeted for support through the program, there were a significant proportion of students from non-traditional backgrounds that meet the Australian Government Indigenous, Regional and Low-SES Attainment Fund (IRLSAF).These students had access to additional support as a result of this initiative.

Surveys and Qualitative Analysis
Ethics approval was received from the Charles Sturt University Human Research Ethics Committee (HREC Protocol No H21170).Student and tutor feedback regarding the program was collected using online surveys distributed via email.Participation in surveys was voluntary.The anonymous feedback on tutor sessions was collated to evaluate the efficacy of the Embedded Tutors Program.Students were asked to "Rate your experience from a scale of 1 (unhelpful) to 5 (extremely helpful)."This was followed by the open-ended question, "Anything else you would like to tell us about the consultation."Any information that could be used to identify the tutor or the student was removed from the data prior to analysis.
Responses to the open-ended question were thematically analysed using NVivo 12 qualitative analysis tool to identify possible recurring themes.Qualitative analysis of feedback was used to generate themes and to inform future improvements to the program.

Quantitative Data Analysis
Assessment mark and cumulative mark descriptive statistics were calculated, and group means were compared (Tutor, No Tutor) using a paired Student t-test.Significance was set at p<0.05 and data reported ± standard deviation (SD).All analyses were performed using GraphPad Prism version 9.3.1 (GraphPad Software, La Jolla, California, USA).

Subject Mark Analysis
In total, 428 students attended 615 tutor sessions with an embedded tutor across 12 first-year subjects; 116 students attended two or more tutor sessions.The students who met with a tutor had an average cumulative subject mark 6.2% higher than the students who did not meet with a tutor (Figure 1, 67.4% vs 61.2%, P<0.0001).

Figure 1
Overall Subject Marks Notes: Average cumulative mark for students who did not attend a tutor session (No Tutor) and those who did attend a tutor session (Tutor) for (n=12) Data are expressed as mean ± standard deviation (* p<0.0001).
Assessment marks were compared from the two groups, i.e., students who attended a tutor session for a later assessment item and those who did not, before tutor support was available.There was no significant difference in the median assessment mark, or the distribution of marks, for the first assessment task in which no students were able to meet with a tutor (Figure 2A).Meeting with a tutor for draft assessment feedback was associated with an 6.3% increase in the average assessment mark (Figure 2B).Students who had previously met with a tutor had an average assessment mark 6.6% higher in a subsequent assessment item, despite not meeting with a tutor regarding that assessment item (Figure 2C).

Figure 2
Assessment Marks Notes: Distribution of assessment item marks for students who did not attend a tutor session (No Tutor) and those who did attend a tutor session (Tutor).A) Subjects with an assessment item before tutor support was available (n=9).B) Subjects with an tutor support available for the assessment item (n=12).C) Subjects with a subsequent assessment item in which no tutor support was available (n=7, *, p<0.05).
A major finding of this study is that students who met with a tutor regarding specific feedback on a draft of an assessment had on average higher assessment marks and cumulative marks compared to those that did not meet with a tutor.Students who met with a tutor for an earlier assessment item, on average received a higher mark for a subsequent assessment item when no tutor was available.As there was no difference between groups in the marks of the first assessment item, it is therefore likely that improvements in marks in subsequent assessments were at least in part a result of the intervention (Figure 2C).
Meeting with an embedded tutor resulted in a significantly higher assessment mark in seven individual subjects (p<0.05).Five subjects showed no significant difference in assessment mark, however, in two subjects there was a trend for students who met with a tutor to have an increased mark.The non-significance can be attributed to a lack of student engagement with the tutoring support that was offered.For example, in one subject in which only seven students met with a tutor the average assessment mark was 60% for those students that did not meet with a tutor vs 83% for students who met with a tutor.One reason that students did not engage with tutor support in these subjects was a lack of preparedness to study.In the remaining three subjects, the tutor sessions were not established until several weeks into the semester and may not have been promoted well enough, missing the key period at the start of the semester where study expectations, timetables, and advice on how to succeed in this subject are often communicated to students.

Students from Non-Traditional Backgrounds
The cumulative subject mark of students from low SES backgrounds who met with a tutor was 5.9% higher than those who did not (n=131).The 22 Australian First Nations students achieved a mark 9.5% higher following a consultation with a tutor than First Nations students who did not meet with a tutor (n=175).Students who are from RRR backgrounds or are First-in-Family to study at university had an increase in cumulative grades of 5.6% and 6.1%, respectively.In total, 66% of students who met with a tutor met the IRLSAF criteria.A major benefit of the program is to provide support to students from nontraditional backgrounds without targeting IRLSAF students specifically.

Surveys and Qualitative Analysis
Following booking a tutor session, 219 students (51%) provided feedback (Figure 3).In total, 78% (n=170) scored the embedded tutor session 5 "extremely helpful."Sixteen students (7%) rated the embedded tutor session 3 or lower, with the most common reason for negative feedback due to issues with the booking system (n=7).4).Thematic analysis identified several key themes indicating that students found the embedded tutors helpful (49 comments) and that they were grateful (25 comments).In addition, 11 students made a positive comment regarding confidence, seven comments mentioned feedback and three comments regarding mental health.Four comments were identified as being distinctly negative and included difficulty in understanding feedback and narrow focus of assistance, as well as the tutor not attending the meeting or responding to communication.

Positive Student Feedback
Following the conclusion of the tutor sessions, feedback was discussed with the Subject Coordinators and tutors from several subjects via Zoom meetings.Eighteen tutors provided feedback on their experience via the online survey.Tutors commented on the increased confidence seen in students, how appreciative and grateful the students were for the support and indicated that it was a rewarding experience for the tutors themselves.Subject Coordinators provided feedback on the impact of the embedded tutors in their subject and several comments were around the program being a great initiative.A Subject Coordinator from a subject with a large and diverse student cohort provided feedback that the support in their subject was invaluable to students and has helped students enormously.

Discussion
The Embedded Tutors Program successfully supported first-year undergraduate students to achieve higher assessment and cumulative subject marks.Nominated tutors were faculty-specific, often sessional, academic staff with the capacity to enhance student learning and increase understanding of subject-specific content.The Embedded Tutors Program has impacted the student experience positively by supporting students from across the University to feel connected while studying remotely.The incredibly positive feedback from students, tutors and Subject Coordinators has affirmed the impact of embedded tutors on student satisfaction.Previously established literature highlights the importance of a sense of connection (Lizzio, 2006) and a sense of belonging (Kahu & Nelson, 2018).It is likely that the increased assessment marks after meeting with a tutor can be attributed to meaningful interactions with students and "just in time" feedback resulting in an increased sense of both connection and belonging.This has a flow-on effect to increased ongoing engagement in the subject (Kift, 2015).We predict that the students' increased confidence and sense of capability to study, when established at an early point in their study journey, will translate to increased self-efficacy and success in future subjects and throughout the university course.
The online nature of the Embedded Tutors Program had many benefits.Charles Sturt University has six main teaching campuses in New South Wales, and the Embedded Tutors Program was able to offer equivalent support to students, and targeted support for at-risk students, irrespective of their location or mode of study.Students previously restricted by regional or remote study had equitable access to the additional support throughout the entire semester.In subjects where multiple tutors were available across campuses, students had flexible access to more tutors including outside of business hours.The approach also allowed tutors with appropriate expertise to be sourced from wider afield.In addition, there were less administrative costs associated with preparing individual contracts and training tutors, making the program more financially efficient and sustainable.
An important factor in the success of the program is the timely, content-specific feedback as outlined in the literature linking transition pedagogy to best practice (Kift, 2015).A concerted effort has been made to monitor student engagement with the tutors and evaluate the program as a whole.Targeted tutorial support exclusively for students from non-traditional backgrounds has been successful in improving confidence and academic achievement in Australian universities (Lydster & Murray, 2019;Nakata et al., 2019).The future hopes for the project are to model an approach to supporting students that is impactful and sustainable.
The Embedded Tutors Program has received funding for 2022 and will be expanded to include more first-year subjects with high numbers of commencing students.In 2022, as the University resumes face-to-face teaching more students will have the opportunity to meet face-to-face with an embedded tutor.However, due to the flexibility and accessibility of online tutor sessions, many subjects will continue to offer online tutor sessions only.The next step in student retention will be to contact students who receive a fail grade for a submitted assessment and encourage these students to meet with a tutor.The Embedded Tutors Program is targeted at first-year subjects; this is not a limitation of the program but rather has the highest potential for impact according to transition pedagogy, as most attrition occurs in the first year of university (Kift, 2015).

Conclusion
The Embedded Tutors Program has auspiciously emerged during a time when students are feeling isolated from the learning community due to the COVID-19 pandemic (Martin, 2020).A key outcome achieved was more personal and meaningful interactions and feedback for students.The embedded tutor support has introduced variety and flexibility in approaches to supporting students.The Charles Sturt University Retention Team will continue to encourage at-risk students, who may not seek it otherwise, to access tutor support and academic support services, and to increase student retention beyond the pandemic.
The Embedded Tutors Program provides a model for increasing student experience and success.The program provides evidence as to the value of subject-specific, one-on-one tutor support for all first-year students, including those from nontraditional backgrounds.As we continue to bridge the gaps during the transition to university, the long-term impact on student retention and graduation will be monitored and evaluated.

Figure 3 Student
Figure 3Student Responses to Answer the Question 'Rate Your Experience'