Can you teach an old dog new tricks? Institution-wide pedagogical reform at an elite university in Ireland

Institution-wide pedagogical reforms are common across higher education institutions. Such reforms may be driven by rationalisation as well as recognition of the need to provide students with interdisciplinary learning experiences that equip them with the “social and analytic competencies needed in contemporary careers outside the academy” (British Academy, 2016, p.5). This paper reports on an institution-wide pedagogical reform initiative, the Trinity Education Project (TEP), at Trinity College Dublin, an elite and ancient Irish university. We describe the development of the TEP and the implementation of its Assessment Framework, which aims to bring diversity into teaching, learning and assessment through the assessment of graduate attributes in a system strongly focused on assessment of learning, examinations and lectures. Reflections on challenges are presented. Discussion centres on the extent to which it is possible, and the best approach, to achieve consensus in an educational system where autonomous disciplinary structures and traditions prevail. *This ‘Emerging Initiative’ was first presented at the 2017 STARS Conference in Adelaide, Australia in July 2017 and was selected by the Conference Committee as one of the top-rated reports. The authors have kindly given their permission to have this report published in this special issue of the Journal and it has undergone a further review by the editors to confirm it aligns with the Journal format. Please cite this article as: Share, M. & O’Farrell, C. (2017). Can you teach an old dog new tricks? Institution-wide pedagogical reform at an elite university in Ireland. Student Success, 8(2), 87-93. doi: 10.5204/ssj.v8i2.384 This practice report has been accepted for publication in Student Success. Please see the Editorial Policies under the ‘About’ section of the Journal website for further information. Student Success: A journal exploring the experiences of students in tertiary education This work is licensed under a Creative Commons Attribution 4.0 International Licence. As an open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings. ISSN: 2205-0795 Student Success, 8(2) July, 2017 | 87 Can you teach an old dog new tricks? Institution-wide pedagogical reform at an elite university in Ireland


Introduction
Irish higher education institutions [HEIs] have been encouraged at national and international (OECD) levels to enhance student learning and achievement through pedagogical reform. Institutional responses have been shaped by the economic downturn, a decline in state funding, and controls on academic appointments (Clancy, 2015). Though belated, government response to HEI reform aligns with others that favour liberal market-led educational reforms; it has also called for HEIs to rationalise program offerings and reduce duplication across Ireland's seven universities and 14 Institutes of Technology (Government of Ireland, 2011).
Such wide-scale reform also aims to impact on teaching and learning in the secondary school system. Some HEIs have highly restricted program intakes that disproportionately raise entrance requirements. For many students, the final two years of secondary school are characterised by rote-learning to succeed in the 'points race'. As a result, school-leavers tend to be ill-prepared for transition to learning at tertiary level. Thus, tertiary program restructuring has the potential for a ripple effect on teaching and learning in the secondary school system. Some HEIs have addressed organisational restructuring in advance of pedagogical reforms while others, such as Trinity College Dublin [TCD], have focused on renewal of undergraduate programs without prior organisational reform. There is an interesting challenge in the implementation and sustainability of institution-wide curricular reform in an institution where academic organisational structures have changed little over the last 50 years.
In this paper, we describe the development and implementation of an institutional pedagogical reform initiative in an elite and 'ancient' university, Trinity College Dublin. We focus on one element of the reform: the Assessment Framework. This is a core element of reform that aims to enhance student learning, transitions, achievement and employability. We reflect on the challenges and lessons of initiating such changes in an institution where academic practices are deeply rooted in school and departmental traditions and structures, within a wider institutional context that privileges heritage and status. In May 2013, TCD embarked on an ambitious renewal of its undergraduate programs -the Trinity Education Project (TEP). The 'official' impetus, as expressed by the project Steering Group, was a recognition that the curriculum must continue to evolve in the light of advances in disciplinary knowledge, pedagogy, and that the context in which graduates participate is ever changing. There was an acknowledgement that, despite positive results from programmatic reviews, many practices and policies were unsupportive of student engagement in learning and subsequent employability. These included: unnecessary complexity in program structures; outmoded regulations and diversity in their application; absence of an institutional study abroad model; limited flexibility (subject combinations, pathways and part-time study options); overlap in material taught by different Schools and lack of integration; and an over-reliance on traditional delivery and assessment methods, particularly large lecture format and summative examinations (Trinity College Dublin, 2016).

The Trinity Education Project
The overall purpose of the Trinity Education Project (TEP) is to ensure that the undergraduate curriculum enhances TCD's reputation as a 'university of global consequence' (Trinity College Dublin, 2014, p.2) delivering a distinctive student experience and the Trinity graduate attributes. The TEP objectives are framed around the provision of an undergraduate curriculum designed to enable more choice and less complexity in transitions to and through programs and to enhance student achievement through innovative teaching, learning, and assessment methods. A central aim is to shift from dominant and traditional modes of assessment and delivery. The TEP has been designed as a four-phase process: Phase 1 included a benchmarking exercise of universities comparable in size, goals/culture; and/or with existing agreements with TCD, that had undergone curriculum reform: Brown University (US); University of Maastricht (NL); University of Western Australia, University of Aberdeen (UK), University of Warwick (UK). A parallel best practice review used data from the UK Higher Education Academy (HEA) (Pegg, 2013) to highlight key issues in curriculum reform processes.
At the end of Phase 2, the University Council developed and agreed on: • Trinity graduate attributes; • curriculum principles; • new program architecture; • assessment framework; and • a revised academic year structure.
Each strand is overseen by a working group of senior and non-senior academic and administrative staff that reports to the TEP Steering Committee, chaired by the Provost, the University's CEO.

The Assessment Framework
Those involved in Strand 1, Education, had responsibility for the development of a set of Trinity Graduate Attributes and an institutional Assessment Framework.

Rationale and development
Assessment is a fundamental function of higher education with 'a vital impact on student behaviour, staff time, university reputations, league tables and, most of all, students' future lives' (HEA, 2012, p. 7). It serves to motivate and challenge students and stimulate learning, whilst it protects the validity and reliability of Student Success, 8(2) July, 2017 | 89 academic standards. Despite this, assessment may be an afterthought in program and module design, with assessment of learning as measurement and accreditation taking priority over assessment as a vehicle for learning.
A history of TCD notes, pejoratively, that it 'possesses the questionable distinction of being the cradle of the public examinations system' (McDowell & Webb, 1982, p.123). As in many universities, much of TCD academics' time is spent on examinations and marking, an approach that has been robustly challenged as it can incentivise study 'to the test' and support shallow or strategic learning (Race, 2015). Traditional HE assessment practices do not adequately equip students for future challenges as graduates or as life1ong learners (Boud & Falchikov, 2006). They are insufficient to ensure that graduates possess a balance of knowledge, skills and dispositions to face the cognitive and operational challenges of the 'supercomplex' (Barnett, 2000) uncertain world that demands constant re-learning in rapidly changing contexts (Lombardi, 2008).
The TEP Implementation phase required the development of an explicit and standardised Assessment Framework in the context of undergraduate program renewal. The TEP Framework centres on three areas: assessment of graduate attributes; program-focused assessment; and range of assessment.

The Trinity Graduate Attributes
The TEP has put forward a set of Trinity graduate attributes and, using these attributes, agreed on a set curriculum principles and a program architecture to support their development. A Trinity education should produce graduates who think independently, communicate effectively, develop continuously, and act responsibly. It should ensure that Trinity graduates are prepared for further academic study or for a rewarding career, or both, and that they possess the 'meta' skills needed for a lifelong learning.

Program-focused and innovative assessment
The Assessment Framework recommends that programs be designed to include appropriate assessment strategies/tasks that: • support effective learning; • provide students with opportunities to practice new forms of assessment; and • enable achievement of the full spectrum of graduate attributes-the qualities, skills and understandings a university community agrees its students should develop during their time with the institution (Bowden, Hart, King, Trigwell, & Watts, 2000).
There was recognition that to provide evidence of the achievement of graduate attributes, they should be explicitly articulated, aligned and embedded in the curriculum, and contextualised and assessed within the discipline (Barrie, 2004). The Framework represents an institutional approach that encourages ownership and development of an effective and innovative assessment strategy, with three interrelated layers: tenets, enablers and areas of focus.

Implementation of the Assessment Framework
Constructive change in assessment practices and policy in TCD is enabled by an infrastructure to support change. Essential enablers were identified as: professional development in academic practice; fractional secondments/sabbaticals to facilitate change to academic practice; quality (including curriculum review and development); use of technology to enhance assessment practices and processes; reward and recognition.

Fractional secondments: Trinity Education Fellows
Professional development in academic practice, use of technology for the enhancement of assessment practices, and rewards such as the Provost's annual teaching awards have been key strategies of the Trinity Centre for Academic Practice for more than a decade. Whereas Teaching Fellow schemes operate internationally, nationally (e.g. Australian Learning and Teaching Fellows), and at institution level (e.g. University College London), the introduction of Trinity Education Fellows to support implementation of the Assessment Framework is novel in an Irish context. In October 2016, implementation of the Assessment Framework commenced with the appointment of six Teaching Fellows, appointed for one academic year, to work with Schools/Departments to support the implementation of the TEP, with a particular focus on a program-based approach to the assessment of the graduate attributes.

Reflections on the journey so far
While some academic staff consider that graduate attributes are rooted in employability, and so the responsibility of the Careers Advisory Service, placing the Trinity Graduate Attributes at the centre of its curriculum reform has enabled robust discussion of the place and assessment of graduate attributes in academic programs, in the context of a new program architecture that supports breadth and depth in the curriculum, and expansion of student choice. While some of the Trinity attributes are regarded as relatively open to assessment, others-, typically, the more complex attributes related to 'wicked competences' (Knight & Page, 2007, p.11) have attracted concern about whether they can and/or should be assessed. It has also opened up conversation about the need to embrace a broader range of assessment methods, reduce the number of examinations, and increase interdisciplinary teaching and learning opportunities to enable students to develop qualities, skills and understandings that can shape their employment and societal contributions through academic and co-and extra-curricular activities.
There are concerns in Departments and Schools that changes to assessment practices, such as reducing the number of examinations, generates more work for academics in marking and student feedback. Staff point to the need for more resources to deal with increased variation in teaching and learning, particularly in the context of interdisciplinary teaching and learning. Schools are autonomous entities and ownership and resource issues dampen debates about interdisciplinarity. For some, there are fears of 'dumbing down', becoming like 'other universities' in Ireland that have modernised, and of losing Trinity traditions and even the distinctiveness of language that is used to describe students ('Freshers' and 'Sophisters') and terms ('Michaelmas', 'Hilary' Term).

Outcomes from discussion at 2017 STARS Conference presentation
Presented as an Emerging Initiative, we were keen to learn from session attendees about their experiences of institutional pedagogical reform. We were cognisant of the fact that many Australian HEIs had undertaken such reforms decades ago. The response to our first question: 'What is your experience of institutional pedagogical reform (in general) or change in assessment culture?' confirmed this when one participant pointed out that there was a wealth of information available in Australia on how to undertake institutional pedagogical reform and referred to the Australian Office for Learning and Teaching (OLT) and the Australian Learning and Teaching Council (ALTC). We responded that Irish HEIs often refer to the UK Higher Education Academy Subject Centres. Despite Student Success, 8(2) July, 2017 | 91 being defunded by the UK government, they remain an important resource for curriculum design and pedagogical supports. Interestingly, it seems that the OLT has also undergone recent structural reorganisation. 1 There was some depth of discussion to the question: 'Can pedagogical program reform be sustained in the absence of structural organisational reforms?' Some participants reflected that there was little point in waiting for structural reforms prior to commencement of pedagogical reforms. Structural organisational reforms tend to be slow moving and politically contentious and therefore can halt progress at the learning and teaching interface.
Some participants commented that institutional pedagogical reform should be considered as, and responded to, as a significant change management process and required strong leadership at the highest level of the HEI. Such sentiments were reaffirmed in contemporaneous Twitter conversations and that the issue had an international agenda (Kift, 2017).
However, another participant commented on the need to recognise that HEI staff have had to experience considerable amounts of organisational change and disruption in recent years and that issues of power and the lack of resources could not be ignored. One participant further stressed the importance of the involvement of HEI stakeholders at all levels and of engaging students in the process.