Competency-Driven Admission Policy 1 Competency-Driven Admission Policy Conclusions from the Surrey Portfolio Pathway Partnership David P. Burns, Ph.D. Department of Educational Studies, Kwantlen Polytechnic University Burns, D. P. (2018). Competency-driven admission policy: Conclusions from the Surrey Portfolio Pathway Partnership. Surrey, British Columbia, Canada: Kwantlen Open Resource Access. October 18, 2018 CC-BY Competency-Driven Admission Policy 2 Between 2016 and 2018, the Kwantlen Educational Policy Incubator conducted a study on competency-driven admission to post-secondary institutions. The study, because it relied on our partnership with the Surrey Schools, was called the Surrey Portfolio Pathway Partnership [S3P]. The S3P was guided by a single question: To what extent can competencies be used to augment conventional achievement indicators, such as letter grades,1 in post-secondary admission? Prospective answers to this question have substantial implications both for education scholarship and education policy. It is, however, only the latter domain with which this document is concerned. This paper argues that competency-driven admission to postsecondary education is possible. It begins by identifying the need for post-secondary admission reform in British Columbia, and then addresses the issues intrinsic to that reform. Once these issues have been outlined, I propose a way forward – a new model of post-secondary admission that tracks competencies and not only grades. British Columbia’s K-12 system The Canadian Province of British Columbia began the “full transition” of the K-9 system in 2016 (see Ministry of Education, 2018) and is expected to complete the transition at the grade 12 level in 2020. The changes, broadly construed, are intended to increase flexibility and individualization, and to increase the extent to which competencies drive the educational process. Increased emphasis will also be placed on student-centered learning, 21st century skills, Aboriginal perspectives and knowledge, cross-disciplinary learning, and social, personal and environmental responsibility. Most notable for our present purposes is the emphasis placed on competency. At the centre of the entire transition of the current curricular system are the Core Competencies: 1. Communication 2. Positive Personal and Cultural Identity 3. Critical Thinking 4. Personal Awareness and Responsibility 5. Creative Thinking 6. Social Responsibility Tied to these Core Competencies are the Curricular Competencies, which are discipline-specific and housed within individual courses in much the same way as are traditional learning outcomes. While the assessment structure in the graduation years will include two standardized tests (literacy and numeracy), the new curriculum is often thought to encourage a wider range of 1 I will not discuss here the question of replacing letter or number grades. This raises the question of how such a reporting system could be interoperable with systems in other provinces and countries and is, as such, outside of the scope of this analysis. Competency-Driven Admission Policy 3 demonstrations of achievement. Surrey Schools, for instance, is encouraging teachers to experiment, in the ways they see fit, with ePortfolios. 2 It is the idea of competency-driven curricula, communicated through student portfolios, that motivated the S3P. The Need for reform If one is to accept that an educational future that relies on competencies and portfolios more, and on grades and standardized tests less, is desirable, one policy risk stands out. It is a concern I have heard spoken by teachers, principals, superintendents, university professors, higher education administrators and government officials. While it is articulated variably, I will refer to it here as the risk of incentivized stagnation. The story of this risk begins with the data on postsecondary admission. 3 These data point to two relevant phenomena. First, they show that the number of persons in BC, between the ages of 25 and 64, with tertiary education is large (56%). Second, they show that this number is rising over time – up from 48% in 2010. Given that the OECD definition of tertiary education excludes many fields Canadians might consider post-secondary education (such as trades training at a college), it is possible that this study actually understates the extent of this change. To simplify, then, it is fair to say that most British Columbians will attend post-secondary institutions, and that this is likely to become more common over time. British Columbia has a finite number of seats in its public post-secondary system, and some institutions receive thousands of applications each year from applicants that they cannot admit. This means that most British Columbians will grow up engaged in a competition for a scarce, and valuable, post-secondary seat. Many of these students will attend open-access institutions or might not attend an institution at all. They are, however, going through their K12 experience with the assumption that this is going on in the background. Something similar is occurring with respect to post-secondary entrance scholarships. Add to this the fact that students have a limited amount of time and attention. Each day they must make decisions about which school work is the most important, which subjects are the best investments of their time, and so forth. If they value post-secondary education, and the given data suggest they do, they will surely make these decisions in light of post-secondary admission policies. If the post-secondary system does not build competency-driven admission options into their systems, and instead rely only on conventional grades, it seems fair to conclude that secondary students will prioritize activities leading to conventional grades. Competencies and portfolios, to the extent that they are non-traditional, would be a luxury. From the perspective of post-secondary institutions [PSIs], this situation is very much acceptable. Consider that: 1. Comprehensive reform of admission systems is resource intensive, complex, and time consuming. For the purposes of this analysis “ePortfolio” is being used to denote collections of student work (summative and formative) that can be collected digitally and stored over time. 3 “Educational attainment of the population aged 25 to 64, by age group and sex, Organisation for Economic Cooperation and Development (OECD), Canada, provinces and territories” (Statistics Canada, 2018) 2 Competency-Driven Admission Policy 4 2. PSIs typically control their admission policies (and not, for instance, the provincial government). 3. Admission policies dictate who has access to a scarce resource (seats in post-secondary programs). Given that the current admission policies are stable (if not ideal), it seems unreasonable to expect PSIs to engage in radical reforms that are expensive and within their control. The PSIs didn’t write the new curriculum, after all, so why should they invest time, money, and human resources accommodating it? In sum, we have arrived at a system in which students are incentivized to follow the lead of the post-secondary system, which is itself incentivized to hold fast to the status quo. These incentives encourage the education system to temper reforms not because they are pedagogically unsound, but because the structure of incentives in the sector fails to give the key actors a reason to change. What we did This is the context in which the S3P project began. A more fulsome scholarly analysis, as well as other related writing on the study, are in progress and this document will be updated to refer to them as they are published. For the purposes of this analysis, though, I will briefly summarize the work we have conducted. The first stage of the study was to collect a sampling of portfolios from students in Surrey Schools – specifically from work in a grade 12 career course, Career Life Connections. Having seen about 20 portfolios submitted anonymously by students, we decided that a test admission would only be possible if students received the kind of guidance they would receive in a competency-driven system, which was not yet in place. My team thus recruited six students and paired them with three of my undergraduate researchers (two high school students for each undergraduate student). For the final six months of high school, these six students would build capstone portfolios with the guidance of undergraduate mentors and myself (collectively taking the place of a hypothetical grade 12 capstone teacher). I received permission, by a vote of my university senate, to admit those six students on the basis of their ePortfolios – without assessing their grades. The only requirement would be that the students graduate grade 12. We then said to the students, “pretend that you did not receive this pre-acceptance, what would you show us to prove you are ready for undergraduate study?” The result was months of conversation about different forms of achievement, possible future paths, and about 40 assignments. We took those assignments and used them to simulate a system in which competencies took centre stage. We made no effort to randomize the sample of students or assignments, and consequently make no claims about the student population in general. What we did attempt was to ascertain the ways in which competency-based admission policies might fail. These conditions of failure will be described in the following section and will inform the model I propose would avoid them. Competency-Driven Admission Policy 5 Failure Condition 1: Labour Duplication Competency-driven admission policies will fail if they involve or imply a substantial duplication of effort and resources between secondary and post-secondary institutions. From a systemic perspective one could say there are three stages to the transition from secondary to post-secondary education. 1. Students take part in hundreds of assessments (formal and informal) during their graduation years. Taken together, these represent our best guess as to the students’ actual achievement of given outcomes. 2. Secondary educators (and in some educational systems, standardized tests) summarize student achievement in a transcript – distilling the broad range of assessments into a much smaller set. 3. The transcript is sent to a PSI, which then uses the data to make admission decisions. Under the traditional division of labour, (1) and (2) are undertaken by secondary educators. They assess the countless quizzes, projects, essays and other performances that students take part in during their schooling experience. Those same educators, sometimes in conjunction with a standardized test (typically administered by the part of a government responsible for K12 education), create a small set of grades that represent the broad achievement of the student across subjects. These are incredibly labour-intensive activities. Taking measurements about a student’s knowledge, skills and abilities, and then distilling those into a single representation, is one of the most important and complex things for which the K12 system is responsible. By the time a transcript is finally handed off to a PSI, this task is complete. At the post-secondary level, the enterprise is complex but less labour intensive. A university, for instance, might stipulate that it will admit students only with a grade of C or better in a high school English course. Admission decisions, then, become a straightforward process – does the transcript indicate the student received a C or higher? The amount of labour required at each step steadily decreases. Summarizing achievement is less labour intensive than is the actual observation of that achievement and using a summary to make admission decisions is easier still. This is one of the reasons large PSIs are so weary to include items such as personal letters in applications. In one formulation of competency-driven admission, PSIs would review application packages that include student ePortfolios. This scenario understandably gives university administrators (particularly admission officials) pause. Such a system would, in practice, result in the creation of a second transcripting stage in which PSIs would duplicate some of the work currently done by secondary educators. The students in this system would have their work summarized for their high school transcript, and then again for their university admission. Since admission departments are usually staffed by administrative professionals, rather than professional academics, this duplication would require substantial restructuring of the PSI workforce. A system like this one, in which PSIs evaluate student performance directly, is therefore likely to fail because its success would require a costly new step that duplicates work already being done. Competency-Driven Admission Policy 6 Failure Condition 2: Platform Reliance Competency-driven admission policies will fail if they are reliant on changing technology. Digital technologies are fluid and are iterated upon at a speed that cannot be matched by public policy. The creative and educational tools and platforms in use today will either be replaced or substantively different in, at most, a few years. This reality poses serious questions for any competency-driven system of education. One of the central ways in which competencies are demonstrated and showcased is through the collection of student work in ePortfolios. The digital tools that students use to create this work vary dramatically. Taking our small team as an example, we received videos, paper diagrams, word processing documents, and so forth. Ideally, students could demonstrate their competencies through all of these media. It would also be preferable that the assignments – insofar as they represent evidence of their competencies – follow the students so that they can be used in future applications to PSIs or jobs. The ways that students store data and create content are, however, constantly changing. Given that a student spends 13 years in K12 education, and that most students spend additional years in PSIs, the time horizon for the education system is necessarily far longer. What is needed, then, is a system that allows long term storage of evaluative information and student work that is not vulnerable to fast-paced changes in the information technology sector. Building a competency-driven admission system through any particular software, or any particular web platform, is simply far too risky for a large public education system. Failure Condition 3: Imbalance in Prescription and Discrimination Competency-driven admission policies will fail if they over-prescribe or under-discriminate. Two of the most common questions I have received in interviews, after speeches and presentations, and during workshops, are: 1. How can you ensure people don’t just put random things into portfolios? This concern is sometimes called the “shoebox” problem. 2. Don’t you need structure or rules about what the portfolio will look like? These questions address the same underlying issue: what should be in a portfolio? To get to a compelling answer, though, one needs to step back to the notion of student portfolios. In the context of BC’s K12 reforms, portfolios are a means rather than an end. They are useful to the extent that they help demonstrate competencies. They are, in this context, an antonym for traditional, narrowly constructed assignments. Portfolios can be used to collect many different forms of student work, as well as feedback, and so are often used as a shorthand for assessment methods that focus on what students are able to demonstrate through their work. Crucially, portfolios can be platform agnostic. What matters is not where a student learned how to create a particular model, how to write in a particular way, or how to build a particular thing – what matters is that the student demonstrates competency. This is one of the core benefits of a curriculum based on competency rather than content – competency can come from anywhere. This means that a competency-based curriculum can accept a far wider range of student work Competency-Driven Admission Policy 7 than a conventional assessment (such as a paper and pencil test). In this sense, the feared shoebox of work is a feature and not a bug. If students cannot place a unique assortment of work into their portfolios the system starts to be content-driven rather than competency-driven. At the same time, the wider the range of work accepted the harder it becomes to discriminate between poor, satisfactory, and exemplary work. While there are sound pedagogical reasons to say that we should do less of this sort of relative evaluation, it is, nonetheless, a clear operational imperative that PSIs must sometimes refuse applicants. A successful policy would therefore need to both permit diversity and allow for discrimination between students on a transparent and just basis. A model If one accepts the premise that post-secondary admission drives much student behaviour in high school, one is led to the conclusion that a true transformation in BC K12 education requires meaningful change in post-secondary admission policy. One such change is competency-driven admission to post-secondary education. Through my work attempting to design such a policy, I have encountered three failure conditions that any viable proposal must address. 1. Labour Duplication 2. Platform Reliance 3. Imbalance in Prescription and Discrimination Avoiding these three conditions does not make a proposal the right proposal. It is entirely possible that a range of options would satisfy these requirements. It is also true, on the other hand, that failing to address any of these three conditions is disqualifying. So what might fit the bill? The Province of British Columbia has organized the curriculum under three headings: Core Competencies; Curricular Competencies; and Content. The Core Competencies, listed earlier in this document, are too general to serve as admission tools. Whether a person can communicate or not is important, but not specific enough to take one student over another at the local university. The Curricular Competencies, on the other hand, are specific. Indeed, they are far more granular than high school transcripts. They are, importantly, already assessed by the teachers offering those courses. In the traditional system, a teacher would assess the range of outcomes required by government and would then aggregate them into a letter or number grade. It would be minimally disruptive for teachers to report that outcome-specific feedback before they aggregate it. If I already know that a student has met certain objectives, why not pause to note which of those objectives they have met? This would allow a teacher to accept any form of demonstration a student offers, subject only to the pedagogical judgement that teacher already exercises when designing assessments and assigning grades. Even if one were to use a proficiency scale (not meeting, meeting, exceeding standards) the transcript this would result in entirely avoids the problem of duplicating costly work. PSIs could, under such a system, receive transcripts much like the ones they receive today. The single difference is that they would receive specific feedback by Curricular Competency. This form of transcripting could be maintained without the kind of restructuring that would be Competency-Driven Admission Policy 8 required by the direct reading of student portfolios. The same systems that process letter and number grades could be adapted to process numbers used to represent achievement of outcomes. These tasks, in information technology terms, are comparable. Such a system would also have the advantage of avoiding the content creation and storage issues intrinsic to ePortfolio use. A competency-driven transcript could directly include written qualitative feedback, much as contemporary report cards do. This feedback could include links to the evidence of achievement the students have produced. This evidence would be made obsolete over time, as technology changes and links break, but would be isolated from the record of achievement that the education system must hold indefinitely. Perhaps the most compelling benefits of this kind of system can be found when we test for the third failure condition. Since the determination of achievement is being made by the teachers, they could in principle accept any form of demonstration that met their pedagogical standards. Since high school teachers know students better than do post-secondary admission departments, they are clearly the best positioned to make that judgement. Shoeboxes are, in other words, welcome. At the same time the need to maintain structure and discrimination is addressed. The structure, in this case the Curricular Competencies, sits atop the shoebox. A motley collection of assignments might appear chaotic, but a teacher expert in her subject area can tell you if these things demonstrate competence in her field of endeavour. Using a proficiency scale to replace or augment letter grades would result in less discrimination, but dividing a given course according to its competencies would even more dramatically increase our ability to discriminate between students at varying stages of their learning. Knowing that a student received an A in an English course is potentially valuable, but knowing that she demonstrated every competency and excelled in two of them is far more actionable. As someone who has both helped to set admission standards, and who teaches first year undergraduate students, I find competencies far more informative. A competency-driven admission policy that uses the report card and transcripting systems to track achievement of competencies, and that links to evidence of achievement (ePortfolios), avoids all three failure conditions I have discussed. What is more, this approach would provide a more specific basis on which to discriminate between levels of achievement. To return to the central policy risk I articulated earlier in this analysis, it is useful to imagine the incentives this model would create. A grade 12 student could be told not that their university of choice requires an A+ in a biology course, but that it requires that she exceed the standard in each of twelve discrete competencies. It seems plausible to suggest that this student would both be motivated to exceed across the curriculum (not merely in areas likely to be on the test) and that she would be encouraged to think about ways in which she could bring different competencies together in different ways. It seems to me that this curricular reform process offers educators, at all levels, the chance both to give students more space to show what they can do and to give other educators much better information about those students. Thank you to Surrey Schools and Kwantlen Polytechnic University for their generous support of this work.