Tapping Librarians for a Successful Technology Training Pilot. A Practice Report

This text reports on the early success of a technology training program designed to introduce incoming first-year students to the basics of academic technologies they will use during their undergraduate career on a campus in a U.S institution. The training was built upon existing but independent programs. Their content was consolidated and redesigned for the remote learning environment of Summer and Fall 2020, leveraging the experience of librarians and library staff to facilitate its quick development. The content of this training, its implementation using the campus learning management system (LMS), and the initial assessment results are reviewed, along with thoughts on the next steps to extend the pilot to the Spring of 2021 and beyond.


Introduction
The COVID-19 pandemic forced higher education into a remote learning environment beginning in the second half of the Spring 2020 1 . This situation highlighted the lack of experience most incoming students have with academic technology, and the need to rectify this deficit before the start of the Summer and Fall semesters, which were projected to stay at least partially remote. This need led a major university to develop a systematic instruction program to, at the very least, provide some basic knowledge of these tools to first year-students.
The folly of referring to students who grew up with technology as "Digital Natives" has done nothing to support the needs of Millennial students or the Gen Z's that followed them. Studies like the one by Waycott et al. (2010) have begun to push back against the assumption that these students need little or no instruction in technology use because they have grown up in a technology rich environment. However, familiarity with smartphones and various applications (apps) still leaves students with incomplete knowledge when saving to cloud storage, uploading an assignment to a learning management system (LMS), or navigating the ribbons of Microsoft Word. Academic technology is a different beast, and often is more sophisticated than what undergraduate students use daily. Howard et al. (2016) provide a summary of research studies that report students lack confidence and familiarity with academic technology. The social-economic status of students can affect their exposure to technology and, even if they are familiar with basic technology, they often do not have confidence to tackle more "sophisticated tasks, such as those relating to processing and creating" (p. 31).
This text reports on the early success of a technology training program designed to introduce incoming first-year students to the basics of academic technologies they will use during their undergraduate career on a campus in a U.S institution. The training was built upon existing but independent programs. Their content was consolidated and redesigned for the remote learning environment of Summer and Fall 2020, leveraging the experience of librarians and library staff to facilitate its quick development. The content of this training, its implementation using the campus learning management system (LMS), and the initial assessment results are reviewed, along with thoughts on the next steps to extend the pilot to the Spring of 2021 and beyond.
This knowledge gap was exacerbated with the shift to remote learning necessitated by COVID-19. With the continuance of remote learning, incoming first-year students would need to rely more on academic technology than ever before. Administrators at Penn State University 2 recognized the need to develop a more systematic strategy to introduce students to academic technology. However, this training needed to be developed quickly, before the start of the Summer 2020 session, to be beneficial. Fortunately, there were groups at the university who had independently identified these knowledge gaps previously and had individually created content to expose first-year students to the technology tools and skills they would need in higher education. Included in this group were librarians, student success center staff, and instructional designers. Administration tasked representatives from these diverse groups, in the timeframe of just a few months, to transition and integrate existing content into a standardized training that could be administered to students across the 19 geographically dispersed university campuses before the start of the Summer session.
This practice report details the genesis of this initial pilot program and the significant role librarians had in its development. Final evaluations of the pilot and plans for a larger implementation are still underway at the time of writing. The initial conception of this training should be useful for institutions beginning to develop their own academic technology training and identifying potential institutional collaborators. Librarians are an important partner in student successone that is often overlooked. This project highlights how librarians can be essential assets when designing a student training program, technology or otherwise.

Penn State's Tech Academy
Academic technology training already existed, in some version, at a handful of Penn State's campuses. But, the content of the training varied, and was not adapted to the remote learning environment. Some instances were self-graded modules integrated into the learning management system (Canvas), while others were intricate in-person sessions facilitated by librarians. These in-person sessions became a considerable influence on the development of the new academic technology training program, largely because of their focus on student engagement and practical application.
The structure of these in-person trainings grew out of research done by librarians Eichelberger and Imler in 2015 and 2016. They recognized the presence of technology related questions at their reference desks and through their research concluded that these questions represented a systemic problem. Students did not have a strong knowledge of academic technology and were not receiving this instruction through any existing course. So the libraries, and their reference desk, were a common way that students were filling the gap. After identifying basic technology tasks that challenged students, Imler developed an in-person Tech Academy for PASSS (Pathway to Success: Summer Start) students at Penn State Altoona starting in 2016.
PASSS students were an important group to involve in this type of training. Though the selection criteria can vary between campuses, generally they are first-year students who would benefit from additional support during their first semester. Because of this, they are offered the opportunity to start their college career during the summer before their traditional start date. The number of PASSS students can vary depending on the campus, but typically it is 50 or less. In addition to two for-credit courses, PASSS students are asked to complete non-credit programs like Math Academy or English Academy to help them prepare for their first-year courses. Academic technology training, consisting of six hands-on weekly sessions, was also part of the cocurricular support given to PASSS students. This in-person Tech Academy had been offered since 2016, so its success and impact on students had already been observed through student and faculty feedback. Both groups responded favorably to the training's goal to cover only the basics of the technology and its focus on providing practical experience with the tools used on campus. Content of the in-person sessions covered basic skills by having students complete a task, like adding an attachment to an email, that most faculty assume students know but may be rarely used outside an academic setting. It also gave students the permission to explore the technology available, so took the time to introduce more sophisticated tools like Adobe Cloud that most students do not have access to in high school but may need to use heavily depending on their major. This structure was unique among the other technology trainings used on campuses because it required the students to actively engage with the technology and to provide deliverables using that technology. The favorable response received by the training led a handful of other campuses to implement their own versions of the in-person Tech Academy for their PASSS students before the shift to remote in Spring 2020.
For the new online Tech Academy, the designers compared the content of the existing academic technology trainings to find common themes to include in the new version. The content of the online training was also compared to new student orientation to ensure that information was not repeated. In the end, the content stayed mostly the same as the in-person sessions highlighted above. The online training modules included: Campus Communication (including Zoom and Outlook), Document Creation and Storage (Office 365 and G Suite), Libraries and Research Support, and Digital Projects (including Adobe Creative Cloud and the university's WordPress site). Assignments in each module kept the practical nature of the in-person training. For example, students must create and save a Word document, upload a version of that document to the LMS, and in addition send that document as an email attachment. They also are required to take screenshots of various material and upload those files to the LMS, along with creating a blog and uploading that URL to the LMS. These assignments also have the added benefit of giving students hands-on experience with the LMS, something that was not part of the in-person training but an important addition for the remote learning environment.
The main difference between the in-person and online Tech Academy is that the new version is designed to be full asynchronous, with the assignments and instructional materials hosted in the LMS (Canvas). However, the in-person training succeeded because of the practical interaction students had with the technology and the face-to-face experience they had with the librarian instructor. Since those two characteristics were so important, designers of the online version tried to keep as much of the personal engagement as they could. Since these practical assignments cannot be graded by computer, human facilitators were needed. In order to find a group willing to facilitate the Summer 2020 pilot, the designers turned to librarians, knowing that their technology skills and instruction experience made them perfect partners for the first iteration of Tech Academy.
Librarians were a critical support network for the pilot online training. They were already skilled in providing informal instruction to students through their library instruction programs and were familiar with providing support for academic technology at their reference desk. Plus, on a practical level, as 12-month employees most were available during the summer months, and their workload was somewhat lighter as most faculty and students were not in classes. As facilitators, their task was to check that students completed the assignments in the LMS, provide any support that students needed to complete the assignments, and to offer the designers feedback on the program content and the training they received. The assignments do not require any rubrics or extensive grading, only a "check" grade to be sure that students completed the assignment.
Sixteen campuses opted in for the initial Summer 2020 pilot. The students in the pilot were all participants in the PASSS program. The number of students varied between the campuses, with the most being 53 students per facilitator. Some campuses chose to utilize two facilitators to help spread out the work. PASSS students had been the original recipients of the in-person Tech Academy and, as they were a group already identified as benefiting from additional support, they were an ideal cohort for the first pilot of the online program. Plus, as the number of PASSS students is significantly smaller than a traditional in-coming first-year class, they allowed for easier troubleshooting and adaptions. A total of 398 students were enrolled and 20 librarians served as facilitators.
The online Tech Academy pilot was offered after new student orientation in May or June, but before the start of summer classes on July 1, 2020. The goal of this placement was to present Tech Academy before the first day of students' online classes, while distinguishing the information it presented from the glut of new student orientation content. It also allowed the designers to retain autonomy over the Academy since it was not part of orientation programming or an add-on to another faculty's course. This autonomy allowed the designers to quickly gather data and to assess the success of the pilot in time for a potential larger second pilot during Fall 2020. The Summer pilot lasted two weeks: starting one week before classes started and overlapping with the first week of classes. In total 66% of the PASSS students completed Tech Academy during the pilot, which was favorable since the training was not tied to a particular course, nor did the students receive any credit for completing the training.
For assessment, the designers utilized a "3,2,1" survey built into the Tech Academy blog assignment that asked students to share what they learned, what they still had questions about, and provided a space for any additional comments. By integrating the survey into the assignment, the designers hoped they would receive feedback from as many students as possible. Use of the survey resulted in 74% response rate from those students who completed the program. This provided the designers with feedback from 139 students. Over 78% of their comments were positive which showed that the students felt that Tech Academy was valuable experience. Analyzing the responses let the designers see which modules the students found most useful and helped identify areas where instructions or assessments needed to be revised. Designers also reviewed the length of time it took students to complete the training, which average about two and a half hours. This seemed a reasonable amount of time for the designers, but they hoped that the refinements made based on the student feedback would lead to a more efficient time scale. One surprise finding from the surveys was that students also seemed to recognize and respond to the fact that a live facilitator ran the Tech Academy. About 30% of the respondents used the space to ask questions beyond technology, such as how to register for Fall courses, how to drop/add courses for the Summer session, or even solicited advice on how to be successful in college.
The librarians were also offered an opportunity to de-brief and offer feedback on their experience as facilitators. They shared information on areas where students seemed to need more assistance (which were corroborated by the student survey results) and shared the length of time it took them to check the assignments. Their report of 20 minutes per student was concerning, considering the goal to scale up the training. Their feedback however identified areas where the designers could refine some of the instructions and other content to reduce student confusion and the time facilitators spent managing the course. Based on the librarians' feedback the designers also developed a timeline document to help facilitators pace the two weeks implantation of the program, and various communication templates to help with reminders and other announcements.
The Summer 2020 pilot was a successful proof of concept for the content and structure of the online Tech Academy. Both the students and librarians felt that it was a valuable and worthwhile experience and responded positively to having a facilitator grade the material as opposed to a self-grading module. The students appreciated that someone was taking the time to introduce and provide basic instruction in academic technology that very few had been exposed to before. Tying the training to a particular class or offering an outside incentive may have led to a larger completion rate, but the flexibility found in this implementation, both related to assessment and revision, was a larger payoff then having more students complete this initial pilot.

Role of Librarian Collaboration with Summer 2020 Pilot
One important way that the Summer pilot was both unique and successful is the way that it utilized librarians. The content of the online training was based on librarian-led research related to students experience with academic technology, and librarians' daily experience at the reference desk. Students inexperience with academic technology, and librarians need to provide support, is not unique to the Penn State campuses. Saunders (2015;2020) recognized that one of the important competencies for reference librarians is technology, with knowledge related to software and hardware troubleshooting being vital to supporting patrons. Some libraries, like the one at the University of La Verne (Jiang et al., 2015) have even included technology support as part of their organizational mission statement.
Beyond their work troubleshooting technology at the reference desk, librarians also have extensive experience with developing and implementing instruction programs. Ducas et al. (2020) found that a majority of librarians surveyed participated in some vein of "Teach and Learning" services. The core of this work is often with first-year students, providing library and research orientations during many of the 100 level 3 courses or first-year seminars. This background was uniquely valuable for the Summer pilot of the online Tech Academy since it allowed the librarian facilitators to quickly troubleshoot and scaffold instructions or content that confused students.
In a more practical mode, librarians are one of the few groups on campus with instruction experience that are considered 12month employees, so they were available when most 10-month contract faculty employees were not. Moreover, with less instruction work occurring in the summer, librarians use summers to evaluate and develop instruction programs for the coming academic year, so initiatives like this pilot fit well within their summer work objectives.
The daily technology support work undertaken by academic librarians is often unnoticed by administrators and the larger university community. However, with the inclusion of librarians as part of the design team for the Tech Academy and their willingness to serve as its initial facilitators, librarians were vital collaborators to the success of the Summer 2020 pilot. Beyond this first pilot they have become champions of the Tech Academy at their campuses and have helped pave the way for the Fall 2020 pilots and potential Spring 2021 pilots, which will reach even more students.

Fall 2020 Pilot and Beyond
The success the Summer pilot led to a larger pilot of the online Tech Academy in Fall 2020. However, some scaling issues needed to be solved for the second pilot to be as successful as the first. Eight campuses participated, which was lower than the Summer pilot, but the number of students enrolled grew to over 1,100. This meant that librarians alone could not serve as facilitators and other campus staff and faculty would need to be involved. Varying strategies for implementation also needed to be utilized, beyond the standalone program that was used for the PASSS students. Since the Summer pilot served its purpose as a "proof of concept", it made sense to lose some of the autonomy and flexibility and focus instead on the modes of implementation and integration.
Each of the campuses involved in the pilot chose a slightly different way to release the online Tech Academy to their first-year students. Only one campus chose to retain the library led facilitation, while many others chose to embed Tech Academy into a first-year seminar course or another 100 level course. In most cases the instructor of record served as facilitator for these iterations of Tech Academy, though in one case a librarian served as guest facilitator. Two campuses chose to pair the program with a peer mentor initiative which used second-or third-year students as facilitators. Varying implementation strategies allowed for a unique opportunity for the designers to compare student success and satisfaction across the modes of implementation. They could then develop more concrete recommendations and best practices for future semesters.
The content of Tech Academy largely stayed constant from the Summer to Fall pilot, though some small tweaks were made to streamline the experience for both facilitators and students. One of the biggest changes was the addition of pre/post survey, so that the designers could gauge students' comfort and confidence with academic technology before and after completing the Tech Academy. The "3,2,1" survey was also maintained to gather qualitative feedback, though with 1,100 students it was not as feasible to gather and review all the responses. Facilitators were asked to analyze their students' responses and summarize them for the designers during a de-briefing session, where they were also asked to share their overall experience to help with pilot assessment. The facilitator de-briefing indicated that many of the issues identified by the librarians were rectified, leading to a positive experience for both the facilitators and the students.
The data and feedback from the Fall pilot are still being analyzed but based on an initial scan of the pre/post test results, the students' overall comfort and confidence with academic technology appeared to increase. The time spent both by students and the facilitators was also reduced, which was important with the larger number of students enrolled. The data are also being reviewed to see if any method of implementation is more successful than others, so that the designers can make recommendations going forward and encourage more campuses to implement the online Tech Academy in the coming semesters.

Conclusion
The success of the initial Summer and Fall pilots of the Tech Academy lie with the ability to quickly modify existing content for a systematic distribution and the identification of unique ways to gather feedback to evaluate the pilots in real time. One of those unique approaches is the utilization of librarians as collaborators. Their existing work related to student technology support, their initiatives to build their own independent training programs, and their expertise in teaching to first-year students, all built a strong foundation for the Summer 2020 pilot. Starting in such a place of strength, the pilot online Tech Academy is on track to become an established training program that can provide all first-year students at Penn State campuses an opportunity to learn the academic technology skills that they need to succeed.