You’ve Got Mail: A Technology-Mediated Feedback Strategy to Support Self-Regulated Learning in First-Year University Students

Abstract


Introduction
Though the transition to independent and novel tertiary learning environments has improved in recent years, this process may still be challenging for some first-year university students (Baik et al., 2015).New learning environments such as flipped classrooms-where traditional classroom activities (e.g., content presentation) are to be completed at home, whilst homework activities (e.g., problem solving) are in-class activities (Sohrabi & Iraj, 2016)-often require students to self-regulate and use effective study strategies to prepare for in-class activities.Higher education literature indicates that there is often limited student pre-class preparation, and clear guidelines regarding pre-class time and learning material usage are required to overcome this concern (Akçayır & Akçayır, 2018).Further, Baik et al. (2015) found that first-year students across eight Self-regulated learning (SRL) is associated with university student academic success outcomes, however students often need support to develop these skills.Technology-mediated feedback is one strategy that may aid educators in supporting students' SRL development.This study aims to explore whether a tec hnology-mediated feedback strategy targeting tutorial preparation for flipped classrooms enhances first-year students' self-report SRL and observed implementation of the strategy.Self-report SRL was measured using the Motivated Strategies for Learning Questionnaire (MSLQ); strategy implementation profiles were based on lecture video access patterns.First -year psychology students (n = 99) were sent technology-mediated feedback emails aimed at developing their SRL.Pairedsamples t-tests revealed significant increases in post-intervention self-reported motivational SRL subscales; selfreported and observed learning strategies implementation did not improve.Future research could build upon this exploratory work to form a multi-pronged strategy to increase understanding of the role of technology-mediated feedback in first-year students' SRL development for flipped classroom learning.
Australian universities struggled to work consistently and maintain motivation.However, where students more consistently apply self-regulated learning (SRL) strategies, their performance and motivation toward learning has been found to increase (Kizilcec et al., 2017).University students must develop effective strategies to complete pre-class activities in flipped classrooms, as these support engagement with in-class activities, and students' development of conceptual understanding (Matcha et al., 2019).University teaching staff are well positioned to support students' SRL development through targeted feedback (Kramarski, 2017;Winne & Hadwin, 1998), however, increasing student class sizes, diverse student populations and growth in blended and online learning have presented challenges to timely delivery of high -quality personalised feedback to large cohorts (Iraj et al., 2020;Lim et al., 2021;Pardo et al., 2019).Emerging hi gher education research presents learning analytics (LA) driven technology-mediated feedback as one potential strategy to enhance university students' SRL, particularly for larger cohorts (Chou & Zou, 2020;Lim et al., 2021;Matcha et al., 2019;Viberg et al., 2020).This study explores a technology-mediated feedback strategy targeting tutorial preparation in a first-year university course, and its impact on first-year students' SRL development.

Self-Regulated Learning (SRL)
Defined as a comprehensive and holistic approach to understanding the behavioural, cognitive, emotional/affective, and motivational factors influencing learning (Panadero, 2017), SRL is noted as one critical factor contributing to student success in academic and learning outcomes (Mega et al., 2014).As Panadero (2017) notes, SRL theories differentially emphasise (meta)cognition, motivation, and emotion, due to the theoretical orientation of the model's author(s), however a cross theories, SRL is conceptualised as a feedback loop involving student-context interactions influencing students' learning processes.Winne and Hadwin's COPES model (1998) provides a meta-cognitive view of the SRL process.Studying is posited as a goaldirected activity occurring through four interconnected phases: task definition, goal(s) setting/planning, study tactics/strategies, and metacognitive adaptation.The final phase involves an evaluation where students maintain or change learning strategies for future learning tasks.These phases are recursive, and weakly sequenced in form; that is, stages may not occur in order, and products of posterior stages may update products and processes of earlier stages encountered in future studying events.The COPES acronym represents internal and external task factors influencing the four phases.The factors are (a) Conditions: accessible resources, and inherent task or environment constraints (e.g., student's motivation); (b) Operations: the student's cognitive processes (i.e., learning strategies); (c) Products: information generated by operations; (d) Evaluations: feedback concerning the fit between products and standards, established internally (i.e., by the student) or externally (e.g., via teacher feedback); (e) Standards: criteria for monitoring products (e.g., assessment criteria).A key component of this model is that external feedback can assist with SRL skill development as it loops between the different phases, identifying which SRL skill(s) require further development.The present study selected this model due to i ts positioning of external feedback (i.e., evaluations) to assist students in assessing their learning strategies (i.e., operations) and mot ivation (i.e., conditions) as one method to support student SRL development.

Feedback
Feedback is central for empowering students to become self-regulated learners as it encourages self-monitoring, provides direction, and guides action (Hattie & Timperley, 2007;Nicol & McFarlane-Dick, 2006).As per the COPES model, external feedback facilitates SRL by providing information for students to evaluate strategy effectiveness, study performance and monitor learning goal(s) progress (Winne & Hadwin, 1998).As a result, students can consider adjusting strategies to allow them to better achieve learning goals and improve performance (Lim et al., 2021).Furthermore, effective feedback including explicit standards supports SRL development by allowing students to make informed judgements about their performance through comparison to standards.Without appropriate feedback, misconceptions can lead to selection of ineffective learning strategies (Malmberg et al., 2014).As per the COPES model, students need clear standards against which to judge their learning strategies (i.e., operations) (Winne & Hadwin, 1998).Especially when it c omes to flipped classrooms, provision of clear guidelines and feedback on students' learning strategies is important so that they understand how to prepare for tutorials (Akçayır & Akçayır, 2018).However, providing effective feedback has become more difficult in recent years (Iraj et al., 2020;Lim et al., 2021;Pardo et al., 2019).Changes in higher education have led to educators seeking novel ways to provide timely, personalised feedback to students to promote their SRL (Pardo et al., 2019).

Utilising Technology-Mediated Feedback to Support Students' SRL Development
As blended online learning becomes more common, collection of student engagement data via Learning Management Systems (LMS) presents educators with additional opportunities to support stude nt learning.Learning analytics (LA) tools allow for collection and analysis of student data to explore students' learning processes (Pardo et al., 2019).Specifically, the OnTask system (Pardo et al., 2018) enables teaching staff to generate personalised feedback based on LA data drawn from student interactions with course content in the LMS.This insight into student progress can allow for feedback to be formulated to promote SRL (Duval, 2011).Following feedback provision, students can adjust their learning strategies for future study events (Winne & Hadwin, 1998).When students are provided with timely, personalised feedback aided by automated technology, strategy adjustments may occur more efficiently.
Although there is interest in exploring LA tools to promote SRL, the LA field is relatively new given its emergence in 2011 (Banihashem et al., 2022) and research on its use for SRL development is limited (Viberg et al., 2020).Viberg and colleagues' meta-analysis investigating SRL and LA applications in online learning environments found only 9% of examined studies used feedback to promote SRL.Despite this limitation, the findings identify technology -mediated feedback as a promising tool for supporting SRL.Previous research demonstrates LA based feedback is effective in improving students' engagement, satisfaction, and performance (Banihashem et al., 2022, Iraj et al., 2020;Lewis et al., 2021;Pardo et al., 2019).LA based feedback may aid students by way of internal and external task factors (Winne & Hadwin, 1998), whereby students receive automated, yet personalised, feedback on their engagement or performance, to inform SRL strategy monitoring (i.e., via the evaluations process).Lim et al. (2021) demonstrated positive changes in SRL when technology-mediated feedback was introduced into a large first-year course.In the study, an experimental cohort received feedback based on their learning activity data and interactions.Compared to a control cohort, the experimental cohort demonstrated more sustaine d course e-book interactions, suggesting the feedback 'nudged' them towards more regular study.Furthermore, the experimental cohort achieved higher grades compared to the control, indicating that adjusted learning operations as a result of feedback, were effective.The technology-mediated feedback provided students with explicit standards to evaluate and adapt their learning operations (i.e., extent of their e-book engagement).This finding is promising as it suggests that technology-mediated feedback presents a potential strategy for enhancing students' SRL, by enabling instructors' provision of personalised feedback to large class sizes in an efficient manner.
Flipped classrooms require first-year students to develop self-regulated learning skills and feedback can play a key role in promoting SRL, however providing timely feedback can be challenging.This study aims to explore a technology -mediated feedback strategy targeting tutorial preparation in a flipped classroom to develop first-year university students' SRL as guided by the following research questions (RQ):

RQ1: Do self-reports of students' motivational orientations of self-regulated learning increase following technology-mediated feedback?
RQ2: Do self-reports of students' use of (meta)cognitive learning strategies increase following technology-mediated feedback?
RQ3: Do students implement a learning strategy designed to assist with tutorial preparation outlined in technology-mediated feedback?

Participants
Psychology students enrolled in a core first-year course were invited to participate in the study (N = 244).At Time 1 (T1), 204 students completed data collection, while 144 students completed data collection at Time 2 (T2).At T1, 103 students were excluded, and 43 students were excluded at T2 due to: missing data; aged 17; or electing to not participate.Thus, data from 101 students was screened for analysis.After screening, the final sample (n = 99) was comprised of students aged 18-46 years old (median = 19), of which 81.8% were female.

Context of the Study
Technology-mediated feedback system: OnTask OnTask (Pardo et al., 2018) was used for the technology-mediated feedback strategy.OnTask is a LA tool that enables educators to transform students' data to create 'if-then' rules to generate personalised emails based upon students' own activity (e.g., lecture viewing) sourced from the LMS.Students received 13 weekly OnTask emails from their course coordinator, however only three of these directly related to this study; all three emails related to the study focused upon the suggested learning strategy.This strategy aimed to reinforce weekly study habits for tutorial preparation (e.g., watching lecture video [s] prior to class).The subsequent two feedback emails followed two forms, dependent on the student's implementation of the suggested learning strategy: (a) praise, or (b) prompt (Figure 1).The first feedback form was coded as: if the stud ent had completed the strategy prior to the tutorial class, then they were sent (a) direct 'praise' feedback email as positive reinforcement.The second feedback form was coded: if the strategy was partially or not completed at all, then the feedback email contained (b) a reminder and link to the lecture video[s] as a prompt to complete the preparation task.In alignment with the COPES model (Winne & Hadwin, 1998) and select feedback principles for self-regulation facilitation (Nicol & Macfarlane-Dick, 2006), the feedback emails clarified good performance (i.e., standards), presented an opportunity to selfassess and monitor learning strategies, and were timely via OnTask automation.

Figure 1
Praise or Prompt Feedback Forms of Technology-Mediated Feedback Self-reported SRL: Motivated Strategies for Learning Questionnaire (MSLQ) Self-reported SRL levels were determined by a modified version of the MSLQ (Pintrich et al., 1991).MSLQ subscales were used to explore students' levels of SRL characteristics in alignment with the COPES model.Two of the six motivation subscales were selected.Control of learning beliefs (CoLB; 4 items) explored students' perception of the relationship between their academic outcomes and their learning efforts.Self-efficacy for learning and performance (SELP; 8 items) determined students' perceptions about their task-performance and task-mastery abilities.The two motivation subscales were selected to capture the conditions component of the COPES model.
Five of the nine learning strategies subscales were selected.Elaboration (6 items) determined students' perceived use of notetaking strategies to build meaningful connections between course materials.Organization (4 items) assessed students' perceived use of identifying main ideas from course materials to actively contrast connections among information.Metacognitive self-regulation (MSR; 12 items) determined students' perceptions of their adjustment of the learning process; this subscale is considered to directly reflect the fourth phase (i.e., metacognitive adaptations) of the COPES model.Time/study environment (8 items) assessed students' self-reported planning and monitoring of their study time and environment.Effortregulation (4 items) measured students' perceptions of their abilities to manage their effort and attention against distractions, therefore regulating continued use of learning strategies.The five selected learning strategies subscales were selected to reflect the operations element of the COPES model.
Responses to the items were scored on a 7-point Likert scale, where 1 is coded as "not at all true of me", and 7 is coded as "very true of me" (Pintrich et al., 1991, p. 11).Estimates of internal consistency for the MSLQ range from 0.52 -0.93 (Duncan & McKeachie, 2005) and satisfactory convergent-and factor-validity in tertiary students has been demonstrated (Pintrich et al., 1993).The MSLQ was administered to students in-class using SurveyMonkey.Following the collection of this data at T1 and T2, the mean MSLQ subscale scores were calculated to determine students' self -reported levels on selected SRL characteristics (Pintrich et al., 1991).

Implementation of the learning strategy: LMS log data
Profiles were determined through analysis of students' lecture video access patterns.This data was used as watching video materials prior to attending their class was one of the preparation tasks students were required to complete for optimal learning in the course.The observed behaviours were re-coded into four strategy implementation profile categories: (a) improvement, (b) partial improvement, (c) maintained strategy, and (d) no improvement.The (a) improvement profile denotes the student shifted their lecture video access pattern from not following the suggested strategy at baseline (i.e., pre -class viewing), to accessing the lecture videos pre-tutorial as suggested post-intervention.The (b) partial improvement profile represents students that did not access the videos at all during the baseline phase, who shifted to accessing the videos albeit after their tutor ial.The (c) maintained strategy profile reflects students who were already following the suggested learning strategy at baseline and continued to do so post-intervention.Finally, the (d) no improvement profile captures students who either did not access videos pre-class at all during the study or were originally following the strategy at baseline but then did not continue postintervention.

Study Design and Data Collection Procedure
The present study utilised a quantitative repeated measures research design as outlined in Figure 2. Ethics approval was obtained (Ethics Protocol 203672).The participation information sheet was uploaded onto the LMS and students were requested via email to read the information sheet prior to attending the Week 2 tutorial.In the Week 2 tutorial (T1), students were briefed on the study and given time to complete the MSLQ in-class.Participation in the study was voluntary; informed consent was obtained via the questionnaire.Students were sent personalised emails every week, regarding aspects of the course (e.g., assessment submitted) though these emails did not relate to the study.Students were sent three emails in this study specifically relating to the suggested learning strategy which outlined the importance of accessing specific resources prior to attending tutorial classes in weeks 4, 5 and 9 of the course.In Week 4, the first email outlined the suggested learning stra tegy.This strategy encouraged students to develop a weekly study habit that involved preparing for in-class activities by watching lecture materials and completing relevant readings.In line with the COPES model, the subsequent two feedback emails provided students with guidelines (i.e., standards) against which their learning operations could be evaluated and prompted them towards adapting their study tactics and strategies (i.e., operations) to align with the suggested study strategy.In Week 5, the second email reminded students of the strategy and gave feedback based on whether they had implemented this strategy (e.g., accessed the resources).The third and final email contained similar feedback relating to resources accessed by students in Week 9. Students completed the MSLQ in Week 10 (T2) to determine post-intervention self-report SRL levels.Students' LMS log data were obtained at course completion to determine if students had employed the learning strategy (i.e., the strategy implementation profiles).

Statistical Analyses
Analyses were conducted using the Statistical Package for the Social Sciences (SPSS, Version 27).Participant data were screened for normality, linearity, outliers, Mauchly's Sphericity, and Levene's Homogeneity of Variance.Initial data screening revealed two outliers in the MSLQ subscale scores, resulting in the final sample (n = 99).To address RQ1 and RQ2, pairedsamples t-tests were conducted to determine differences in MSLQ subscale scores at T1 to T2.Due to small sample sizes in three of the four profiles determining students' implementation of the learning strategy, no further inferential analyses were conducted for RQ3.

Results
Students' mean motivation subscales scores were higher at T2 than at T1, while the mean MSLQ and learning strategies subscales scores decreased from T1 to T2 (Table 1).The assumptions screened for were met, and the planned inferential testing for RQ1 and RQ2 was conducted.

RQ1: Do Self-Reports of Students' Motivational Orientations of Self-Regulated Learning Increase Following Technology-Mediated Feedback?
As seen in Table 1, paired-samples t-tests revealed that students scored significantly higher on the two motivation subscales (i.e., CoLB, and SELP) following the intervention with a medium effect size, as determined by Cohen's d (Cohen, 1988).

RQ2: Do Self-Reports of Students' Use of (Meta)Cognitive Learning Strategies Increase Following Technology-Mediated Feedback?
Students' scores on three of the learning strategies subscales (i.e., organisation, time/study environment, and effort-regulation) were significantly lower following the intervention (Table 1); effect sizes were medium to large.While not statistically significant, students' mean scores on two of the learning strategies subscales (i.e., elaboration, and MSR) decreased from T1 to T2.

RQ3: Do Students Implement a Learning Strategy Designed to Assist with Tutorial Preparation Outlined in Technology-Mediated Feedback?
Across the strategy implementation profiles, four students were categorised into improvement, two into partial improvement, 10 into maintained strategy, and 83 into no improvement.This hypothesis was unexplored by statistical analysis given the pattern of behaviour in the strategy implementation profiles and available data.Profiles showed 16% of students either improved or maintained the suggested learning strategy of watching lecture videos prior to tutorials, while remaining students showed no improvement following technology-mediated feedback.(Cohen, 1988).

Discussion
This study examined a technology-mediated feedback strategy to support self-regulated learning (SRL) in first-year university students.The strategy focused upon preparing students for tutorials in a flipped classroom setting.Students' perceived motivational orientations and learning strategies were investigated following technology -mediated feedback.Results showed that students reported an increase in motivation following technology -mediated feedback; however, students did not report that use of learning strategies increased following the feedback.Strategy implementation profiles indicated that most students were not observed to employ the tutorial preparation learning strategy suggested in the technology -mediated feedback.
Researchers interested in the use of technology-mediated feedback to help students develop SRL skills for flipped classrooms can learn from these findings for future studies.

Interpretation of the Findings
Student's motivational orientations measured by the MSLQ were higher following technology-mediated feedback.Specifically, students reported they felt more in control of their learning efforts, and higher self-efficacy for learning and performance in the course after the feedback.Previous research by Lewis and colleagues (2021) found students felt more motivated and were more likely to engage with course materials following technology -mediated feedback; students also reported a flow on effect of motivation toward their other courses following feedback.Similar to other studies investigating technology-mediated feedback emails (Lewis et al., 2021;Lim et al., 2021), students in the current study r eceived not only emails about the learning strategy, but also general course related emails (e.g., assessment information, due date reminders).These general emails, combined with the tutorial preparation strategy emails, may have been useful more broadly resulting in higher self-reported motivation.However, it is important to note students may have felt more in control of their learning efforts and report higher self-efficacy due to feeling more confident as the semester progressed; the present study was not able to control for this time effect.Furthermore, the motivational orientations examined refer to efforts to learn and expectations around success and confidence to master course work more generally (Pintrich et al., 1991), rather than the suggested l earning strategy specifically.It is possible students found the emails motivated them towards their studies more generally and this could be why an increase in motivation was reported.Further research should consider a mixed methods approach, asking students about perceived benefits of technology-mediated feedback to determine what aspects are motivating and in what situation.
Although previous research has demonstrated that students have implemented SRL strategies following technology -mediated feedback (e.g., Lewis et al., 2021;Lim et al., 2021), students in the current study did not report increased use of (meta)cognitive learning strategies after technology-mediated feedback.Furthermore, LMS data showed that most students did not implement the suggested strategy outlined in the three emails.In previous studies (e.g., Lewis et al., 2021;Lim et al., 2021) students received feedback about their performance in graded multiple-choice quizzes within a first-year course.Depending on performance, students were provided with feedback suggesting a strategy to improve if they did not do well or praise if they performed well in the quizzes.Results from previous studies suggested that students were learning to self -regulate as LMS data showed they were implementing the suggested strategies outlined in the feedback.The current study trialled a different strategy focusing on tutorial preparation in a flipped classroom setting.Future research could build on the strategy employed in this study to further understand how technology-mediated feedback may be useful for developing SRL in flipped classrooms.Though the feedback emails in this study align with select feedback principles for self -regulation facilitation outlined by Nicol and Macfarlane-Dick (2006), participants may not have been adequately 'nudged' towards developing the selected SRL characteristics.Though LA based feedback can be used to nudge students' learning processes with suggestions of useful course materials, educators must be mindful that feedback can have a mixed impact depending on its quality and timeliness (Banihashem et al., 2022).Similar to Lim and colleagues' ( 2021) study, the current study only provided strategy implementation feedback at select time points during the course.Given students' ability to access feedback whilst completing pre-class activities is restricted in flipped classrooms (Akçayır & Akçayır, 2018), future studies investigating SRL and preclass preparation in flipped classrooms could try increasing feedback timepoints to determine if further emails improve perception and use of learning strategies for tutorial preparation.This may be particularly important for first -year students new to flipped learning environments.
Whilst SRL tools and external feedback work for some students, other students require further support, potentially from multiple avenues (Chou & Zou, 2020).Future studies could trial a multi-pronged approach to help develop students' SRL for tutorial preparation.A study by Matcha et al. (2019) examining learning strategies used when technology-mediated feedback was provided found students who utilised a complex learning strategy (i.e., high engagement with multiple learning tactics) were observed to obtain higher final course grades compared to their peers who utilised a simple learning strategy (i.e., low engagement with a single or few learning tactic[s]).Notably, although complex learning strategies were associated with higher performance, most students in Matcha et al.'s study utilised strategy tactics that concentrated on assessment activities.These findings indicate students were focussed on learning activities impacting their grades.Technology -mediated feedback that focusses more on graded performance may be more impactful to students.This may expla in the positive results in previous research that provided feedback about graded performance (Lewis et al. 2021;Lim et al., 2021) as opposed to the contrary findings from this study where the strategy was aimed at tutorial preparation.In the present stud y, strategy implementation was determined by students' video lecture access patterns alone, whereas Matcha et al. paired pre-class video watching with formative online quizzes as part of their class preparation strategy.Formative online quizzes are common in flipped classrooms as they provide students with an incentive to complete preparatory activities and provide educators with learning material usage data (Akçayır & Akçayır, 2018).This pairing of both video and learning activity is worthy of further ex ploration and may provide students with a level of accountability not present in the current study.

Strengths and Limitations
The current study is one of few to investigate technology-mediated feedback as a strategy to support SRL in relation to tutorial preparation.The feedback emails students received were guided by the COPES model (Winne & Hadwin, 1998) such that students were provided with evaluations (i.e., strategy implementation feedback) so that they could assess their operations (i.e., learning strategies).This model has been used to guide technology-mediated feedback in previous research (e.g., Lim et al., 2021) with promising results.Furthermore, the feedback was also aligned with Nicol and Macfarlane-Dick's (2006) principles to develop SRL.A further strength was the use of self-report and observational measures.Self-report data helps capture students' perceptions, whereas the use of trace data from the LMS allowed for an examination of the students' authentic learning behaviour without any intervention (Matcha et al., 2019).A limitation of this study was not including a qualitative component.Future research should explore the features of technology-mediated feedback that are motivating for students and the type of courses that are most suited to this type of feedback (e.g., courses utilising on-going quizzes).Furthermore, as this study is not experimental, it is not known if students' self-reported motivation increased due to technology-mediated feedback received or if this was a time-effect.Future research could explore this by including a control condition.

Conclusion
Higher education research indicates that first-year students, especially those in flipped classrooms, benefit from developing their self-regulated learning, though they may need guidance on this process.Students' SRL skills can be developed with support from educators through personalised feedback, however significant provisional issues have been noted.Technologymediated feedback has been identified as one way to help educators support students' SRL through timely, personalised feedback.The present study found students' motivation increased following technology-mediated feedback, however selfreported and observed implementation of learning strategies did not improve.Several possibilities exist as to why larger changes were not observed, including insufficient feedback timepoints, or lack of incentive to action the suggested strategy (e.g., graded performance via formative quizzes).Future research would benefit from including more feedback timepoints, a qualitative component and/or graded learning activities to address the challenges noted by this study.Summarily, though a mixed impact upon students' SRL development was observed, this study highlights that future research should consider technology-mediated feedback as one part of a multi-pronged approach to holistically support SRL development of students enrolled in flipped classroom courses where various study tactics are required for optimal learning.

Table 1
Descriptive Data and Paired-samples t-test Results of Students' MSLQ Subscales Scores at Time 1 and Time 2. Data are Expressed as t-values, Cohen's d, and Significance (p).