Questions & answers about criterion-referenced standards-based assessment
Understanding the need for change
Why have these new policies been brought in?
What’s happening for students?
Why have these new policies been brought in?
The moderation policy was developed in response to the 2009 AUQA report which suggested the need for a University-wide moderation policy. It is acknowledged that most Schools have moderation and quality assurance processes already in place and this policy will complement those processes.
The assessment policy was informed by “Assessment 2020: Seven propositions for assessment reform in higher education” (Australian Learning and Teaching Council, 2010). It is widely thought that assessment is the single most important determinant of learning behaviour; it’s an integral part of the teaching and learning process that contributes significantly to learning outcomes. What is assessed and how it is assessed gives clear messages to students about what the University considers to be important, and the CSU Assessment Policy needed to be updated to reflect this.
Together, the two policies close the loop in ensuring our assessment practices are aligned to our course and subject learning outcomes and that we have a process of continuous improvement, all focused on enhancing the student experience.
So what is different?
Assessment and grading are now based on a criterion-referenced standards-based approach where assessment is aligned to pre-determined and defined criteria and related standards of skills, knowledge and competencies. Criterion-referenced standards-based assessment takes some of the ‘smoke and mirrors’ out of assessment for students. It ensures they understand what they are being assessed on, what the performance criteria are to pass, as well as to achieve higher grades.
Criteria and standards-based assessment practices require specific criteria and performance standards to be identified and stated so that students understand clearly the level of performance required for each assessment task. In this context, criteria are defined performance attributes to which the assessor refers when forming a judgment about the student’s response to the different aspects of the assessment task. Standards describe the quality of student performance in relation to the criteria in any assessment task. The use of assessment criteria enhances transparency and consistency because expectations about student performance are made clear to students and staff. This enables students to develop better judgments about their own, and others', performances.
There are no pre-determined failure rates for subjects, and there are no given guidelines specifying what percentage of students in a specific cohort will achieve each passing grade (that is, there is no scaling). In addition, every subject needs to be moderated pre-, during- and post-delivery at least once per year. How this is handled and recorded will be decided by your School, although in many instances the pre-delivery moderation can be easily included as part of the existing QA processes.
What do I have to do?
For every subject you coordinate, you need to ensure marking criteria and the associated performance standards are included in your Subject Outline, starting with 201430.
In addition, each subject needs to be moderated pre-, during- and post-delivery at least once per year, to ensure the quality of our assessment processes, including the need for assessment tasks to align with subject and course learning outcomes, assessment requirements to be clearly communicated and ensuring marking is fair and consistent across different delivery modes and campuses. This means you may be asked to review the subject materials, assessments and outcomes of your colleague’s subjects and your colleagues may be asked to review your subject. The formal moderation process needs to be completed by academic staff, whether they are internal or external to the University, but should not be completed by CSU general staff.
What support is available?
Professional development workshops will begin in Schools in the next couple of months. There will be a range of workshops available focusing on developing criteria and standards; using professional judgement in marking; refining criteria and standards; and moderating assessments. Within each School, there are a number of people you will be able to ask for assistance, including your Course Director, Educational Designer and other assessment and moderation leaders.
In addition, the assessment and moderation website will house a comprehensive suite of resources including how-to guides, checklists, guidelines and annotated examples of marking rubrics.
What happens next?
Talk to your Head of School, Course Director, Educational Designer and designated assessment and moderation leaders in your School to find out when professional development workshops have been planned. If you want to get started now, have a look at the materials on this website and work your way through the self-paced resources.
For project updates, check back with this website, keep an eye on What’s New and join the Assessment & Moderation Yammer group.
What’s happening for students?
A communication campaign will begin in February 2014 to notify students of the change in Assessment policy and what to expect in their subject assessments. Academic Support staff will also participate in professional development to ensure they have the knowledge to assist and advise students about criterion-referenced standards-based assessment. However, you have a role to play too. Assessment can’t stand alone and is best supported by scaffolded learning, explicit discussion about the expectations for each assessment and even providing exemplars for students as to what is expected.
Close
Understanding criterion-referenced standards-based Assessment
So this means we don’t have any grading now, we just pass or fail students?
Aren’t we spoon-feeding students too much information about what to do in their assignments?
If we give students too much feedback won’t they just share it with other students in lower years?
Doesn’t criterion-referenced standards-based assessment just dumb things down for students?
Why can’t I normalise grade distribution in criterion-referenced standards-based assessment?
How do you interpret the distribution of grades in criterion-referenced standards-based assessment?
So this means we don’t have any grading now, we just pass or fail students?
No, we will still have the grading we currently use: Fail, Pass, Credit, Distinction, High Distinction. However rather than comparing and ranking students against their peers, they will now be graded against pre-defined and clearly communicated criteria, with standards of performance defined for each grade level.
If a student fails on one criterion within an assessment, does that mean they fail the whole assessment?
No, when writing your criteria and standards, you make a judgement about what the minimum standards for a pass are. While a student may fail a criterion in an assessment, you need to make a holistic judgement about the student’s overall performance in the assessment and assign a grade accordingly. Obviously, your overall score should make sense when students look at sub-scores on the rubric and this will also depend on the weighting you assign to each criterion.
What happens if students fail one assessment, does that mean they automatically fail the whole subject?
No, each discipline (course and subject) will define its own minimum standard to decide if a student has met the requirements of the qualification. With carefully designed assessments and criteria, you can ensure these minimum standards are achieved.
Good assessment design suggests that students are given at least two opportunities to demonstrate competency for each learning outcome. This means your assessments should be designed to allow students multiple chances to demonstrate their knowledge and skills for each learning outcome.
I’m an experienced marker and grade assessments holistically, why do I need to use criterion-referenced standards-based assessment?
Criterion-referenced standards-based assessment ensures transparency, consistency and fairness for students by clearly communicating the standards of performance we expect from them, before they start the subject. Marking holistically without explaining your rationale for grading doesn’t assist students to achieve performance standards or learn how to self-assess their own work. If you think about it, your holistic judgement contains a number of elements or assumptions about the students’ performance; criterion-referenced standards-based assessment is asking you to make these explicit so students know what those expectations are and can work towards meeting them.
Isn’t criterion-referenced standards-based assessment too atomistic, you can’t grade holistically using marking rubrics?
While holistic judgements play a role, they don’t benefit students, both in terms of setting expectations prior to assessment or assisting with feedback after assessment. Holistic marking is often processed within an expert’s mind, which can’t be replicated by other markers. Using criteria and standards asks you to break down that holistic judgment into its components to make the process more transparent for everyone involved. The key benefit is making expectations clear and allowing students to understand the task. It also assists markers to ensure their judgements are consistent both within a cohort and across different offerings. For assessment to be fair, it must be transparent and consistent.
Aren’t we spoon-feeding students too much information about what to do in their assignments?
Good assessment design means we clearly explain our expectations to students. Building ambiguity into assessment is not a valid way to differentiate student performance as this is assessing a different skill that may or may not be relevant to the subject and course being studied, or to the level of study. Scaffolding assessment is a valid approach that can have a powerful influence on the content and structure of the student response and the quality of feedback that is shared. Scaffolding, advance organisation and feedback have been shown to exert a powerful and visible effect on learning (Hattie, 2009). The Principles of Assessment contained within the Assessment Policy clearly require that in order to progressively build student capabilities and skills, assessment must be sequenced and scaffolded.
If we give students too much feedback won’t they just share it with other students in lower years?
Feedback is an important part in the learning process, it helps students understand what they have done well and gaps where they need to improve their knowledge and skills. Students have been telling us, particularly in the OES, that they are not getting enough feedback or guidance about assessment, and criterion-referenced standards-based assessment is one way that helps us address this. Feedback closes the learning loop and cannot be held back for fear that it will be used by other students; if sharing assessment feedback amongst students is a concern for you, you may like to consider developing a bank of assessments that can be rotated each time the subject is offered, therefore minimising the usefulness of sharing past assessments.
Doesn’t criterion-referenced standards-based assessment just dumb things down for students?
Criterion-referenced standards-based assessment gives students explicit descriptions of the performance expected of them to achieve each grade. It doesn’t tell them how to complete an assessment, but what is expected in terms of performance and the factors their work will be judged on. Assessment shouldn’t be a guessing competition for students, where those who can correctly guess the elements get the best marks. It is much better that students focus upon and are assessed according to clear expectations about what they need to do. This also ensures they are achieving the subject learning outcomes which have been set as the most important aspect of knowledge and skills for students to learn from that subject.
Why can’t I normalise grade distribution in criterion-referenced standards-based assessment?
Criterion-referenced standards-based assessment is all about students performing to certain pre-defined standards which have a grade attached. This means students are assessed on their performance alone, regardless of the performance of other students. To “normalise” the distribution of grades is to disregard the students’ achievement, comparing (or ranking) them in relation to their peers. Doing this invalidates their performance against the criteria and is therefore no longer criteria-based assessment.
How do you interpret the distribution of grades in criterion-referenced standards-based assessment?
While grade distributions are no longer monitored to ensure a “normalised” distribution, they can be useful to moderate assessment post-delivery. As in the past, where there is an unusual distribution, this flags to you that there may be an issue with the design of the assessment, the standards and criteria being used in marking, or perhaps the teaching strategies. Remember, using criteria and standards is a challenging new skill to learn that takes time to develop; you won’t necessarily get it exactly right the first time. As with all teaching and learning, this is a process of ongoing improvement.
Close
Developing criteria and standards
Why can’t I make up standards and criteria?
How do you make levels in standards descriptors clear?
Why do I need marking criteria, I’ve already told the students what I expect in the assessment task description?
While a well-worded task description is beneficial, more guidance needs to be given to students to clearly outline the criteria and performance expectations they will be judged on. Without clearly outlining expectations, assessment requirements are open to interpretation from different markers, meaning there is no transparency or consistency, thus making it unfair for students. Clearly defining the criteria we use to mark assessments, as well as the standards of performance required for each grade level, makes the assumptions explicit both for us as markers and for the students. It also helps us better understand what we need to teach, and ensures we are judging against only those criteria that are relevant to the assessment task, subject and course outcomes.
Why can’t I make up standards and criteria?
The Principles of Assessment within the CSU Assessment Policy require that students are notified of the assessment tasks, expectations, criteria and standards of performance prior to the subject commencing. These same Principles state that assessment practices need to be equivalent across all delivery modes and courses for that subject. The only way to do this effectively and fairly is through the use of pre-defined criteria and standards. Standards for assessment tasks should reflect the literature in the field, contemporary practice and professional accreditation requirements. It is essential that they are rigorous and appropriate for the level of the subject and the needs of students, so can’t just be made up as you go along.
Can I use a taxonomy like Bloom’s to develop my standards’ descriptors? Do lower grades only relate to lower levels on taxonomies like Bloom's Taxonomy?
While taxonomies can be useful, you need to be careful that you aren’t just matching lower grades to lower levels on the taxonomy; it’s more complicated than that. For example, you may decide that a task or criterion requires evaluation (high level on Bloom’s taxonomy) that is described at pass level and up. Imagine if all you ever did was describe (lower level on Bloom’s taxonomy) things to get a pass; this wouldn’t help develop the CSU Graduate Attributes, where we expect our graduates to hold strong analytical and critical thinking skills. You need to think carefully about how you expect your students to demonstrate and interact with the subject matter, as well as what is appropriate for their year level of study.
Can a pass level be assigned for performance that is less than or lower than that required by an accreditation standard?
No, to ensure the integrity of our courses and professional standards we must uphold accrediting standards, therefore pass level requirements should line up with threshold competency on accreditation standards. If a student can’t perform to that standard, they have not met the criteria for the accreditation and should not pass.
How do you make levels in standards descriptors clear?
The careful development of your criteria and standards can result in clear descriptions of different levels of performance for each criterion. We need to avoid unhelpful, general descriptions that provide little guidance to students (or markers), and ensure there are informative descriptions to distinguish between performance at each level.
| Criterion | Fail | Pass | Credit | Distinction | High Distinction |
| Understanding of the topic | Poor understanding | Evidence of basic understanding | Evidence of a good understanding | Evidence of a thorough understanding | Evidence of an exceptionally high understanding |
Can you see the problems with this rubric?
- The language is vague, making the rubric mysterious and disempowering for the student, eg how does the student differentiate between ‘good’, ‘thorough’ or ‘exceptionally high’?
- The standards are too general, it is hard to know exactly what the marker is looking for, and in terms of feedback, the student cannot see what needs to be fixed.
- The criteria is too generic and may or may not relate the subject’s learning outcomes.
- The validity of standards is also questionable; should a student pass (ie ultimately a graduate) who has only demonstrated ‘basic understanding’?
This is what Dr Alan Bain, Director of the CSU Smart Learning Project, describes as the “very, very problem”: the use of verbs and adverbs, nouns and adjectives to differentiate standards of performance, but that add little to the student understanding of assessment tasks and their criteria.
To assist you with developing clear criteria and standards, resources will be available on the CSU Assessment and Moderation website, and workshops will be run within your course/discipline teams. Remember, developing good criteria and standards is a skill that takes time to learn and you will continue to refine them each time you use them.
Close
Using Rubrics
Why can’t I use a generic rubric?
I’m already using rubrics, will this be any different?
Doesn’t it take a long time to develop rubrics?
Don’t marking rubrics stifle creativity as they prescribe the approach and don’t allow the student to think about or explore other approaches?
So all I need to do is give the students a rubric?
Do I have to use a rubric?
The CSU Assessment Policy stipulates that you must provide marking criteria and performance standards for each assessment task, but it does not state this needs to be in the form of a rubric or matrix. Many people find a rubric to be a useful format for communicating this information, however it’s up to you how to present it.
Why can’t I use a generic rubric?
Good assessment design links a subject’s learning outcomes to the assessment task and the assessment criteria, therefore a rubric should be tailored to reflect these key elements. Each discipline and style of assessment task will have its own unique needs that a generic rubric won’t cover. Too often, generic rubrics only assess a student’s intelligence, rather than task or discipline specific knowledge and skills. Also, generic rubrics don’t give students enough guidance as to your expectations of their work.
I’m already using rubrics, will this be any different?
Developing robust, useful criteria and standards descriptors is a complex skill that takes time to develop and refine. You may find there is no difference at all to what you are currently using (other than not “normalising” the grade distribution), or that you need to do some more thinking in terms of developing useful standards descriptors and linking your assessment tasks and criteria with the subject and course learning outcomes.
Doesn’t it take a long time to develop rubrics?
As with any new skill, it can take time when you first attempt to write your assessment criteria and performance standards; however this will get easier as you refine your skills. Keep in mind, a well-designed rubric can save time in the long-term as it can make the process of marking and providing feedback to students much quicker.
Don’t marking rubrics stifle creativity as they prescribe the approach and don’t allow the student to think about or explore other approaches?
A marking rubric should define the standards of performance, and what you as the marker expect to see included in the assessment (the criteria the students’ work will be marked upon). The ‘performance’ can relate to skills, knowledge and competencies required in the student work. Unless it is relevant to your discipline and task, the rubric doesn’t need to prescribe the approach.
So all I need to do is give the students a rubric?
Rubrics should be clearly written and communicate the scope and essence of the assessment task. However, they are part of the overall learning experience and there is no better way to clarify a rubric than by sharing examples of what needs to be done. The approach advocated at CSU requires that learning and assessment are scaffolded and clearly linked to learning outcomes at both a subject and course level. This means that assessment tasks and teaching strategies are designed with learning outcomes in mind, and that the criteria for each task reflect these outcomes.
Close
Ascribing Grades
To receive an overall HD grade for an assessment task does the student need to get an HD for all criteria in the rubric?
No, markers have always added scores for components of assignments. In criterion-referenced standards-based assessment the criteria comprise those components. While a student need not attain an HD for everything to receive an HD, overall, the score and grade should line up with the levels of performance on each of the criteria.
Can you ascribe numerical marks to an assessment?
You can assign ‘marks’ or weighting to each criteria and then add these together to assign a score for an assessment. CSU’s grading scales (FL, PS, CR, DI, HD) remain unchanged under the new policy.
Do rubrics weight all criteria equally?
No a rubric does not weight all criteria equally; you can choose to assign different weightings to your criteria. For example, you may decide that achievement against one criterion is worth 50%, while the remaining two are only worth 25% each. The performance of a student against each criteria within an assessment can then be added together to give a final grade.