First year student conceptions of success: What really matters?

Success at university is a complex idea, with evidence that what “counts” as success is conceived differently by students and academics. This study contrasts two methodologies (“Likert-type” ordered response and quadratic voting, which does not appear to have been applied to education research previously) to identify which factors are important in university success to first year health science students. Completion (passing subjects and obtaining qualifications) and achievement (getting good grades) were the most important factors in both methodologies, but important differences were found between the two in the relative importance of four factors, particularly in the importance of a sense of belonging and personalisation of study options. Contrasting data from the two methods potentially separates factors students think are vital from those that are important but not essential—a distinction which is concealed using Likert-type


Introduction
Success is a widely discussed but arguably under-theorised aspect of higher education ). Several models of success exist, most of which focus in some way on students achieving their academic potential (Wood & Breyer, 2017). Particularly in older literature, "success" was often used in terms of academic performance or persistence (Harackiewicz, Barron, Tauer, & Elliot, 2002;Willingham, 1974;Yorke & Longden, 2004), with other factors that moderated retention, such as sense of belonging, placed in a secondary role.
More recent literature has offered fuller and more nuanced conceptualisations of what it is to have a successful experience at university. Kuh, for example, has argued for the centrality of student engagement in success, suggesting that a student is successful if they are intellectually and socially engaged with their academic lives (Kuh, 2007;Kuh, Kinzie, Schuh, & Whitt, 2011). Other conceptualisations have focused on the transformative nature of higher education, such as the development of graduate attributes and skills (e.g. Zepke & Leach, 2010), or sustaining and nurturing personal and professional transitions (Wood & Breyer, 2017, building on Gale and Parker's 2014 theory of "transition as becoming"). Hannon, Smith and Lã (2017) identified that students themselves focus on affective factors, such as happiness, personal growth and good relationships with peers and family, and achievements such as obtaining a qualification, having new experiences or academic achievement, with relatively few reporting the importance of intellectual engagement. These findings indicate the gap that may exist between student-derived and academic-derived conceptions of what constitutes success at university.
Relatively few of these papers describe a formal typology for conceptions of success.  provide an exception, developing a nine factor framework based on the literature, student interviews, and input from university leaders. This framework was used as the core of this study, although it shows significant overlap with the more recent literature discussed above, such as the 16 principles proposed by Wood and Breyer (2017).
The typology used in this paper was based on a modification of the framework established in Coates et al. (2016). This modification was made (replacing "value" with "completion") in part to focus on a specific aspect of value already known to be relevant to students-the value of completing a qualification and having access to an improved job market (Hannon, Smith, & Lã, 2017). This was hypothesised to be particularly relevant and easily understood by first year students in professionally-focused health science degrees. However, it was also intended to be able to separate two aspects of academic achievement: passing subjects and good grades, which were otherwise conflated but play very different roles in student behaviour (the so-called "Ps [passes] get degrees" attitude).
The factors investigated (with a short title in square brackets after the item stem) were: The primary aim of this research was to investigate students' attitudes about the factors that contribute to a successful student experience, using a modification of the theoretical framework developed by . A further aim was to compare the effect of two different rating methodologies (Likert-style and quadratic voting) on student responses. Quadratic voting (Quarfoot, Von Kohorn, Slavin, Sutherland, & Konar, 2016) is an emerging methodology in political science that potentially separates factors respondents think are vital from those that are simply "good to have", but to my knowledge has not yet been applied to educational research. This study therefore has conceptual outcomes in investigating new methodologies and frameworks for the field as well as empirical outcomes in our understanding of student expectations.

Likert-type surveys and quadratic voting
Likert-type (LT) surveys are widely used in education research, and in many other fields where attitude measurement is important. LT questions form the backbone of many wellstudied national surveys such as the National Survey of Student Engagement (NSSE) or the Australasian Survey of Student Engagement (AUSSE), as well as smaller institutional surveys, including the near-ubiquitous student feedback surveys. While debates around the reliability and validity of specific items, scales or instruments (e.g. see Campbell & Cabrera's 2011 study of the NSSE) or how to best interpret the data (e.g. Allen & Seaman, 2007) sometimes occur, the methodology itself has long been widely accepted without question.
LT approaches are, as Quarfoot et al. (2016) explain, methodologies of abundance. Respondents are able to cast votes however they like, with no incentives to consider tradeoffs or relationships between their responses. In a study examining factors contributing to success at university for example, there is nothing to prevent participants from rating every factor as maximally important. In many studies, this is not problematic, and may indeed be an asset, if there are good theoretical reasons to believe that responses should not require trade-offs or limitations in the available choices.
In contrast, quadratic voting (QV) is a methodology of scarcity. Participants are given a limited budget of "points" (50 in this study) with which to indicate the strength of their attitudes for all nine factors. The weighting process used is to square (hence quadratic) the initial scale scores so that 1, 2, 3, 4, 5, 6, and 7 become 1, 4, 9, 16, 25, 36 and 49. This leads to two main differences to LT approaches.
Because participants are only given 50 points with which to cover nine factors, scarcity inherently forces trade-offs between proposals, and leads to different response behaviour (Lalley & Weyl, 2015;Quarfoot et al., 2016). Participants who choose to spend 49 points on one item only have 1 point left to spend across the 8 remaining factors. This is therefore a much stronger and clearer signal than a high response on a LT survey. Participants who want to rate all nine factors as equally important only have enough points to allocate four points to each item (or six items with four points plus 3 items with nine points). Scarcity thus leads overall to moderation of viewpoints, as consequences exist for holding extreme viewpoints, and strength of response is more strongly related to willingness to take action (Quarfoot et al., 2016).
Additionally, positions on a Likert scale such as "important" and "very important" are quantitatively (if potentially erroneously) similar in intensity. The quadratic increase from 1 to 4, or 36 to 49 provides a clearer signal of response weighting for participants and researchers.
QV is an emerging methodology in political studies (Lalley & Weyl, 2015) and has great potential utility in higher education research. Although the finding requires further investigation, the stronger relationship between response strength and willingness to take action in this methodology suggests QV may be particularly useful in studies that examine factors affecting behaviour such as satisfaction, effective learning outcomes and attrition.

Participants
All students enrolled in the first year of a health science degree at the university of interest at the time of the study were invited to participate (following ethics approval for the study to proceed). These health science degrees include entry-to-practice qualifications in vocational fields such as nursing, physiotherapy and dietetics, as well as more generalist degrees, such as a bachelor of health sciences or of exercise science. Overall, 2,226 students were invited to participate in the study.
Participants were recruited via emails sent to their student addresses during a two-week period early in semester two, with three reminder emails sent as prompts during that time. Early semester two was chosen in order to capture students' conceptions of success when they had some experience with university life. As has previously been observed, expectations of higher education based on school and other previous study may not be suitable for university (Appleton-Knapp & Krentler, 2006), and Lowe and Cook (2003) observed that pre-existing study habits persisted until the end of first semester. Thus, a point early in students' university study, but after exposure to the university's community of practice had potentially begun to moderate their conceptions of success, was selected.

Procedure
Participants were asked to complete a short online questionnaire. Relevant demographic information was collected. Students were then asked to "Think about what having a successful experience at university would mean to you," then asked to rate the importance of the factors (detailed in the Introduction above) in having a successful experience.
Participants sequentially used two different rating schemes to indicate importance. In the first condition, respondents rated each factor on a five-point ordered-response LT scale: "Not at all important;" "Slightly important;" "Important;" "Very important;" and "Extremely important." In the second condition, respondents were given a short introduction to QV, and asked to rate the factors again according to the costs shown ("1 -slightly important;" 4; 9; 16; 25; 36; "49 -extremely important."), spending no more than 50 points across all the options. A buffer of 6 points was permitted to allow for minor addition errors. Respondents were advised that they did not have to spend all 50 points, and did not have to rate every item. In both conditions, respondents were able to navigate between questions and change their answers, but a response was required for all demographic questions and the LT scale.
Following submission, these quadratic votes were rescored for comparison with the LT condition. Items that were not allocated any points were scored as "1 -Not at all important." Items allocated one point were scored as "2 -Slightly important." Items allocated four or nine points were scored as "3 -Important." Items allocated 16 or 25 points were scored as "4 -Very important" and those allocated 36 or 49 points (more than half the available budget) as "5 -Extremely important."

Data analysis
Results are typically reported as means with 95% confidence intervals (based on standard error of the mean- Altman & Gardener, 2000). Where relevant, proportions are reported instead of means, in which case confidence intervals are calculated using the method for proportions by Newcomb and Altman (2000). In providing 95% confidence intervals, there is a 5% chance that the population parameter lies outside the interval.
Estimation-based approaches encourages a more sophisticated interpretation of data than significance testing by drawing attention to the size of effects and the precision of data, rather than encouraging accept/reject dichotomies (Newcombe & Altman, 2000). Statistical significance can, however, be read directly from the 95% confidence intervals (at p < 0.05), and is noted where appropriate. Where appropriate, chi-squared tests with Bonferroni correction were used to assess significance for multiple comparisons between proportions.

Completion rate and sample demographics
Two-hundred and one students (out of the population of 2,226) completed at least the first survey condition, providing a 9.0% response rate. Although participants were not forced to complete the QV section, all but nine students did so, creating a pool of 192 responses to both conditions. Of the students who completed both sections, 134 spent 50 points or less as instructed, and 58 spent between 51 and 56 points, presumably due to minor arithmetic mistakes in calculating the points budget. Although participants were able to allocate different factors the same number of points, relatively few chose to. The mean completion time was 10 minutes, with most participants completing the survey in 5-10 minutes, although some respondents took much longer.
The majority (n = 171; 85.1%) of the sample were female, which was expected due to the higher proportion of women in the health sciences across the sector (72.0% across the sector; 76.8% at the university of interest-Department of Education, 2015) and the observed gender bias in response rates.
Approximately two-thirds of the sample were aged 18-20 (n = 129; 64.2%, compared to 52% of commencing first year students across the sector) and the remaining 72 students (35.8%) were aged 21 years or over. The mean age was 22.8 years (SD 7.16 years). Thirty-four students (16.9%) spoke English as an additional language, and 71 (35.3%) reported being the first in their family to attend higher education. The statistical power of the survey was sufficient to detect effect sizes of approximately 0.4-0.5 standard deviations in these categories (β = 0.9).

Factors contributing to success
Using the LT method, every factor was rated as very or extremely important by at least half the cohort (Table 1). Consistent with the work noted in the literature review above, completion and achievement (getting good grades) were seen as fundamental to a successful university experience for the vast majority of students, providing further evidence for the strong association between performance and success in academia. Flexibility in studying (being able to accommodate other commitments) and discovery (or intellectual engagement) were jointly ranked third, with more than threequarters of students rating them very or extremely important. This may be due to the relatively large number of mature-aged and first-in-family students in this cohort, which have previously been shown to be more intellectually engaged and have higher needs to accommodate work and family responsibilities than school leavers (Baik, Naylor, & Arkoudis, 2015). Alternatively, it may represent broader trends in student expectations of their course offerings.
A surprising finding was the relative lack of importance given to having a sense of belonging and making friends at university. This was particularly surprising given the importance of belonging to making as successful transition to higher education in the literature (Elkins, Braxton, & James, 2000;Friedlander, Reid, Shupak, & Cribbie, 2007;Kift, 2009). In contrast, the ability to personalise a course of study was considered slightly more important to success. Both belonging and personalisation formed a cluster with connection and opportunity, which also overlapped with identity.
Using the QV method (Table 2), completion and achievement were again ranked as the most important factors in success at university. The absolute difference between the proportion of people who rated the factor as important and the mean scores remained the same (approximately 15% and 0.4 respectively). However, because of the difference in rating systems, this equated to a much higher relative difference: about half as many students again (150%) voted for the importance of completion to success compared to achievement using the second method, as opposed to 117% under the first method. This underlines the importance of passing subjects to students' conceptions of success, even compared to a relatively similar measure such as achieving good grades (Hart, 2012).
Also of note is the increased relative importance of a sense of belonging. This factor moved from sixth in importance to third, which was a statistically significant change in ranking, and making it cluster with discovery and flexibility in terms of importance rather than any of the factors it had previously clustered with. In contrast, personalisation moved from fifth most important to least important factor, with only 3.1% of respondents suggesting it was very or extremely important, and a mean response of between "slightly important" and "important." These changes are discussed in detail below.
The rankings of other factors were largely unchanged. Please note that the ranking of discovery and flexibility in Table 2 could be reversed, if they were ordered by mean score rather than by proportion of responses. As with the Likert-style method, discovery and flexibility appear to essentially be of the same importance to these students. Finally, a higher standard deviation was found for most items using the QV methodology compared to the LT methodology. This indicates that there was more variation between respondents and a wider range of options chosen. This is consistent with Quarfoot et al.'s (2016) finding that the QV methodology moderated responses and suggests that a more diverse data set was collected using this methodology.

Discussion
The aims of this study were to provide empirical data collected from student responses to add to our understanding of what constitutes success in higher education according, and to explore the effect of changing the survey methodology (from one of abundance to one of scarcity) on the responses collected. Although students completed both parts of the survey sequentially (typically, within a total of 10 min), several interesting differences were found. This suggests that QV offers a promising alternative (or addition) to LT instruments in higher education research. Comparing the two methodologies, it may be possible to separate factors that students believe are vital from those that are "good to have" or "important but not essential" It is clear that the traditional academic conceptions of success retain their currency with this group of students. In both methodologies, passing subjects and completing degrees was considered the most important factor in a successful university experience by a significant (both statistically and pragmatically) degree, followed by academic achievement or getting good grades. Based on this data, it appears that students consider completion and achievement to be essential hallmarks of university success. This is consistent with the conceptions of success used widely (and possibly unreflectively) by many researchers, who identify academic performance with success (Harackiewicz et al., 2002;Willingham, 1974;Yorke & Longden, 2004).
A sense of belonging and social connection was identified as third-most important in the QV methodology, but only sixth-most important using the LT methodology (although 61.5% of students still judged it to be very or extremely important using this methodology). This change in ranking was statistically significant and affected the factors with which belonging clustered in the analysis. Belonging has been the focus of enduring theorising in the literature, particularly in terms of its importance to the transition experience of first year students (Kift, 2009). Sense of belonging and social integration into university has been identified as vital to success at university and strongly predictive of retention (Bean & Eaton, 2001;Elkins et al., 2000;Friedlander et al., 2007). In Tinto's model of transition, separation from a student's previous communities is the first step in transition; this must then be followed by a new sense of belonging to the university community for transition to be successful (Tinto, 2006-7).
One might conclude from the LT data reported here that first-year students may not be fully aware of the importance of creating new social ties to their university communities. However, the QV data indicates that many students are indeed aware, and are willing to spend a significant proportion of their 50 points to signal this importance.
How then can one interpret the statistically significant change in ranking between the two methodologies? It may be that other factors (discovery, flexibility and particularly personalisation) are considered "ideally good but not essential" (whereas belonging might arguably be of middling importance but indispensable). These factors show the largest difference between the two methodologies, with a difference in proportions of approximately 60% and a difference in means of approximately 2.3 (compared to a difference of 40-50% or 1.8-2.0 respectively for the other factors). While these factors are clearly important to students, and, all other things being equal, students consider them valuable aspects of success, they are willing to dispense with them if they have to. This may have parallels in the discussion of students as strategic learners or adopting surface approaches to learning (Chin & Brown, 2000;Ramsden, 2003;Scouller, 1998): ideally, students want an intellectually engaging, flexible education that appeals to them at a personal level. However, they are prepared to sacrifice those aspects to achieve other outcomes, such as obtaining a qualification with a strategic investment of time or effort. In this cohort, many degrees offer no choice of elective subjects, suggesting students may have already made similar sacrifices for personalisation in service to the greater motivation of obtaining a professional qualification. Qualitative research into students' motivation for study and course choice is required to support this conjecture, however.
Finally, there is a cluster of three factorsdeveloping professional connections, identity formation, and opportunity for new experiences-that are relatively consistently low-ranked. This is not to say that these factors are not considered important by the students in this study; no factor was found to be truly unimportant, but on average this cohort appeared to judge these factors to be "important" or "slightly important" rather than vital. It is interesting that identity formation and new experiences are frequently brought up as aspects of the transformative nature and liberal arts heritage of higher education (Mezirow, 2003;Naylor et al., 2016;Wood & Breyer, 2017), whereas developing professional connections are key to both the value argument for on-campus education and the current trend for work-integrated learning (Orrell, 2011;Yuan & Powell, 2013). The relatively low importance of these factors may be due to specific characteristics of this cohort. Many health science students are required to do work placements as part of their degree, which may lead them to downplay the relative importance of forming connections since it is structured into their course; the relatively high proportion of mature-aged students (who may be less interested in identity formation or new experiences than someone who has just left school) may lead to lower importance given to these factors, or the professional nature of the courses may lead them to consider the liberal arts tradition to be less important. Further research will be conducted in another, more generic discipline, to investigate these theories, and analysis of the data for particular subgroups (such as mature-aged or first-infamily students) is being conducted. The low number of responses from male students may also have implications for generalising these findings to first year students as a whole (Diniz et al., 2016).

Conclusions
The aims of this research were to investigate students' attitudes about the successful student experiences, and to compare the effect of two different rating methodologies on student responses. This study therefore has conceptual outcomes in investigating new methodologies and theoretical frameworks for success, as well as empirical outcomes in our understanding of student expectations.
It is clear from this study that quadratic voting produces a different pattern of responses to the more widely used Likert-type surveys. Not only were some factors ranked differently, all factors were ranked more moderately, and a greater diversity of individual responses were provided. Quadratic voting therefore offers a promising alterative to Likert-type instruments in order to more fully investigate major attitudinal, affective or motivational constructs in education. Unlike some ranked choice methodologies, quadratic voting allows the gaps between rankings to be more precisely quantitated (and therefore provide a better estimate of attitude strength), and factors are able to be ranked equally. The methodology may be particularly appropriate where interventions based on professed attitudes have not led to substantial changes in behaviour, where resources for interventions are limited, or in cases where it is important to identify what is vital to stakeholders. The difference in ranked importance of a sense of belonging clearly demonstrates that potential insights may be offered by this method. Further research to more clearly establish and test research protocols for this methodology in education is required.
While this study provides further empirical support for the theoretical framework of success at university used (a relatively undertheorised area until recently), comparison of the two methods allowed the separation of the nine factors into four broad clusters: "essentials" (completion and achievement), "always important" (belonging), "important but not essential" (discovery, flexibility and personalisation) and "moderately important" (connection, identity and opportunity). This "hierarchy of success" potentially provides an insight into student motivation and behaviour in the classroom (why students may adopt surface approaches to learning, for example), and has implications for how academics and university leaders manage a satisfactory university experience (by providing opportunities for flexibility and personalisation where none are currently offered, for example). At an individual level, this research may allow university services to target interested parties more easily and avoid "junk mail." Once again, the importance of a sense of belonging in creating a successful university experience is underlined.
Further research will be performed based on this study to investigate conceptions of success in other disciplines (including those with less of a clear professional pathway than the health sciences, or with a stronger liberal arts heritage) and in subgroups within the cohort (such as mature-aged and first-in-family students). Final year students will also be examined to examine how conceptions of success mature during higher education.