There are a number of important elements to criterion-referenced standards based assessment, and information about each are presented in the Assessment and Moderation pages. In practice you often complete these concurrently rather than sequentially. Whatever order you complete them in, you need to consider:
Criteria and standards are not something you develop once and use forever; you'll want to tweak them over time, based on student feedback, questions from markers and from your own experience as you see how others interpret them. For example, if you find most of your feedback to students relates to the need to use the appropriate style for the assessment type, however you haven't explicitly included this as a criterion, you would update that for the next session you use the assessment.
A good place to start when developing criteria and standards, is to review the subject learning outcomes in relation to the assessment task.
CRSBA stands for Criterion-referenced standards-based assessment.
No, when writing your criteria and standards, you make a judgement about what the minimum standards for a pass are. While a student may fail a criterion in an assessment, you need to make a holistic judgement about the student's overall performance in the assessment and assign a grade accordingly. Obviously, your overall score should make sense when students look at sub-scores on the rubric and this will also depend on the weighting you assign to each criterion.
No, each discipline (course and subject) will define its own minimum standard to decide if a student has met the requirements of the qualification. With carefully designed assessments and criteria, you can ensure these minimum standards are achieved.
Good assessment design suggests that students are given at least two opportunities to demonstrate competency for each learning outcome. This means your assessments should be designed to allow students multiple chances to demonstrate their knowledge and skills for each learning outcome.
Criterion-referenced standards-based assessment ensures transparency, consistency and fairness for students by clearly communicating the standards of performance we expect from them, before they start the subject. Marking holistically without explaining your rationale for grading doesn't assist students to achieve performance standards or learn how to self-assess their own work. If you think about it, your holistic judgement contains a number of elements or assumptions about the students' performance; criterion-referenced standards-based assessment is asking you to make these explicit so students know what those expectations are and can work towards meeting them.
While holistic judgements play a role, they don't benefit students, both in terms of setting expectations prior to assessment or assisting with feedback after assessment. Holistic marking is often processed within an expert's mind, which can't be replicated by other markers. Using criteria and standards asks you to break down that holistic judgment into its components to make the process more transparent for everyone involved. The key benefit is making expectations clear and allowing students to understand the task. It also assists markers to ensure their judgements are consistent both within a cohort and across different offerings. For assessment to be fair, it must be transparent and consistent.
Good assessment design means we clearly explain our expectations to students. Building ambiguity into assessment is not a valid way to differentiate student performance as this is assessing a different skill that may or may not be relevant to the subject and course being studied, or to the level of study. Scaffolding assessment is a valid approach that can have a powerful influence on the content and structure of the student response and the quality of feedback that is shared. Scaffolding, advance organisation and feedback have been shown to exert a powerful and visible effect on learning (Hattie, 2009). The Principles of Assessment contained within the Assessment Policy clearly require that in order to progressively build student capabilities and skills, assessment must be sequenced and scaffolded.
Feedback is an important part in the learning process, it helps students understand what they have done well and gaps where they need to improve their knowledge and skills. Students have been telling us, particularly in the OES, that they are not getting enough feedback or guidance about assessment, and criterion-referenced standards-based assessment is one way that helps us address this. Feedback closes the learning loop and cannot be held back for fear that it will be used by other students; if sharing assessment feedback amongst students is a concern for you, you may like to consider developing a bank of assessments that can be rotated each time the subject is offered, therefore minimising the usefulness of sharing past assessments.
Criterion-referenced standards-based assessment gives students explicit descriptions of the performance expected of them to achieve each grade. It doesn't tell them how to complete an assessment, but what is expected in terms of performance and the factors their work will be judged on. Assessment shouldn't be a guessing competition for students, where those who can correctly guess the elements get the best marks. It is much better that students focus upon and are assessed according to clear expectations about what they need to do. This also ensures they are achieving the subject learning outcomes which have been set as the most important aspect of knowledge and skills for students to learn from that subject.
Criterion-referenced standards-based assessment is all about students performing to certain pre-defined standards which have a grade attached. This means students are assessed on their performance alone, regardless of the performance of other students. To "normalise" the distribution of grades is to disregard the students' achievement, comparing (or ranking) them in relation to their peers. Doing this invalidates their performance against the criteria and is therefore no longer criteria-based assessment.
While grade distributions are no longer monitored to ensure a "normalised" distribution, they can be useful to moderate assessment post-delivery. As in the past, where there is an unusual distribution, this flags to you that there may be an issue with the design of the assessment, the standards and criteria being used in marking, or perhaps the teaching strategies. Remember, using criteria and standards is a challenging new skill to learn that takes time to develop; you won't necessarily get it exactly right the first time. As with all teaching and learning, this is a process of ongoing improvement.
Yes, all undergraduate and postgraduate coursework subjects must include criteria and standards for each assessment task within the Subject Outline (MSI). This includes marking rubrics or guides. Specifically, the CSU Assessment Principles Policy states:
Yes, provided criteria and descriptors are developed for each choice/task and criteria and descriptors for each are included in the Subject Outline, students can be given a choice of assessment tasks for the same % weighting.
Criteria and standards-based assessment practices require specific criteria and performance standards to be identified and stated so that students understand clearly the level of performance required for each assessment task. The use of assessment criteria enhances transparency and consistency because expectations about student performance are made clear to students and staff. This enables students to develop better judgments about their own, and others', performances.
To ensure that all students receive information about assessment tasks and the criteria against which tasks are assessed and marked in a timely manner, assessment criteria and performance standards need to be communicated to students before the subject offering commences via the Subject Outline.
At the moment the CSU Assessment Principles Policy states:
Yes, each criteria needs to have clear performance standards that align with CSU passing grades, i.e High Distinction, Distinction, Credit, Pass, or Satisfactory/Unsatisfactory.
No you do not have to include a fail in your standard descriptors, but you might like to have this discussion with colleagues who teach into your course and see if you can come up with an agreed approach so that students have a consistent experience.
Yes. Clarifying assessment tasks, criteria and standards during session is good teaching, learning and assessment practice. It is also a good way to get student feedback on your assessment task description and on your standards descriptors so that improvements can be made for the following session/offering of the subject.
Involve partners in development and review of rubrics where possible. Perhaps there is an opportunity to do this in pre-delivery moderation activities.
While a well-worded task description is beneficial, more guidance needs to be given to students to clearly outline the criteria and performance expectations they will be judged on. Without clearly outlining expectations, assessment requirements are open to interpretation from different markers, meaning there is no transparency or consistency, thus making it unfair for students. Clearly defining the criteria we use to mark assessments, as well as the standards of performance required for each grade level, makes the assumptions explicit both for us as markers and for the students. It also helps us better understand what we need to teach, and ensures we are judging against only those criteria that are relevant to the assessment task, subject and course outcomes.
The Principles of Assessment within the CSU Assessment Policy require that students are notified of the assessment tasks, expectations, criteria and standards of performance prior to the subject commencing. These same Principles state that assessment practices need to be equivalent across all delivery modes and courses for that subject. The only way to do this effectively and fairly is through the use of pre-defined criteria and standards. Standards for assessment tasks should reflect the literature in the field, contemporary practice and professional accreditation requirements. It is essential that they are rigorous and appropriate for the level of the subject and the needs of students, so can't just be made up as you go along.
While taxonomies can be useful, you need to be careful that you aren't just matching lower grades to lower levels on the taxonomy; it's more complicated than that. For example, you may decide that a task or criterion requires evaluation (high level on Bloom's taxonomy) that is described at pass level and up. Imagine if all you ever did was describe (lower level on Bloom's taxonomy) things to get a pass; this wouldn't help develop the CSU Graduate Attributes, where we expect our graduates to hold strong analytical and critical thinking skills. You need to think carefully about how you expect your students to demonstrate and interact with the subject matter, as well as what is appropriate for their year level of study.
No, to ensure the integrity of our courses and professional standards we must uphold accrediting standards, therefore pass level requirements should line up with threshold competency on accreditation standards. If a student can't perform to that standard, they have not met the criteria for the accreditation and should not pass.
The careful development of your criteria and standards can result in clear descriptions of different levels of performance for each criterion. We need to avoid unhelpful, general descriptions that provide little guidance to students (or markers), and ensure there are informative descriptions to distinguish between performance at each level.
For example, consider these descriptors for one criterion within a marking rubric:
|Understanding of the topic||Poor understanding||Evidence of basic understanding||Evidence of a good understanding||Evidence of a thorough understanding||Evidence of an exceptionally high understanding|
Can you see the problems with this rubric?
This is what Dr Alan Bain describes as the "very, very problem": the use of verbs and adverbs, nouns and adjectives to differentiate standards of performance, but that add little to the student understanding of assessment tasks and their criteria.
External standards inform your course and subject learning outcomes and can be incorporated in your criterion-referenced standards-based assessment as part of your criteria.
The CSU Assessment Policy stipulates that you must provide marking criteria and performance standards for each assessment task, but it does not state this needs to be in the form of a rubric or matrix. Many people find a rubric to be a useful format for communicating this information, however it's up to you how to present it.
Good assessment design links a subject's learning outcomes to the assessment task and the assessment criteria, therefore a rubric should be tailored to reflect these key elements. Each discipline and style of assessment task will have its own unique needs that a generic rubric won't cover. Too often, generic rubrics only assess a student's intelligence, rather than task or discipline specific knowledge and skills. Also, generic rubrics don't give students enough guidance as to your expectations of their work.
A marking rubric should define the standards of performance, and what you as the marker expect to see included in the assessment (the criteria the students' work will be marked upon). The 'performance' can relate to skills, knowledge and competencies required in the student work. Unless it is relevant to your discipline and task, the rubric doesn't need to prescribe the approach.
Rubrics should be clearly written and communicate the scope and essence of the assessment task. However, they are part of the overall learning experience and there is no better way to clarify a rubric than by sharing examples of what needs to be done. The approach advocated at CSU requires that learning and assessment are scaffolded and clearly linked to learning outcomes at both a subject and course level. This means that assessment tasks and teaching strategies are designed with learning outcomes in mind, and that the criteria for each task reflect these outcomes.
No, markers have always added scores for components of assignments. In criterion-referenced standards-based assessment the criteria comprise those components. While a student need not attain an HD for everything to receive an HD, overall, the score and grade should line up with the levels of performance on each of the criteria.
You can assign 'marks' or weighting to each criteria and then add these together to assign a score for an assessment. CSU's grading scales (FL, PS, CR, DI, HD) remain unchanged.
No a rubric does not weight all criteria equally; you can choose to assign different weightings to your criteria. For example, you may decide that achievement against one criterion is worth 50%, while the remaining two are only worth 25% each. The performance of a student against each criteria within an assessment can then be added together to give a final grade.
No, you don't need to write a separate criterion referenced standards based rubric for each individual question in the exam. You must provide (within the subject outline) marking criteria and standards of performance for each assessment task including exams; it's up to you whether this is best represented in a rubric.
If you develop a rubric, it needs to makes clear:
CRSBA can communicate more clearly to the student what they will be assessed on in the exam.
Criteria and standards can help to focus students' work when preparing for the exam – what they need to be aware of. Do they need a hard hat?
Criteria and standards can be used as a guide for markers of extended responses especially when there are multiple cohorts and/or multiple markers. Everyone knows the benchmark.
Criteria and standards allow us to check that the exam questions align with the intended learning outcomes in the subject. Intentions are clear.
Discussion and sharing with your course team and possibly external standard bodies allows a shared understanding of the criteria and standards to be developed. Another way of checking whether the standards in exams are valid is by mapping and aligning subject outcomes and exam criteria using a mapping tool and trialling and evaluating the standards.
An exam map helps you to:
You can download the template for the extended assessment map.
Q: How do I reverse a signed off subject?
A: Only the HOS can reverse a signed off subject in the Online Moderation system. Instructions are linked here.
Q: How do I get assigned to a subject in the Online Moderation System?
A: The Subjects Team enter the subject convenors and moderators in the OMS based on information from the Head of School.
A: Guidelines and timelines for OMS use have been created to guide you through the process. A general timeline for getting started is to do the Pre-delivery moderation at the same time as the subject outline is being quality assured with sufficient time for any remedial work to be completed after feedback.
Q: How do I remove a not moderated subject from the OMS?
A: If you need to remove a subject as being not moderated then you need to go to the Subject Access page, find the subject code and click the box in the not moderated column and update list. Confirm the deletion and the subject is marked not moderated. The procedure can be reversed.
Q; My subject is not showing as completed in the sign off status report, what's wrong?
A: In most cases, the sign off button has not been clicked. If you have clicked the sign off button, and the subject does not appear as signed off, contact the SRS system for support.
Q: What is a sign off report? A: The Sign-Off Status Report provides an overview of all subjects in the School and the dates when each phase of moderation was signed-off. Within the Online Moderation System moderators are responsible for signing-off each moderation record. A HOS or Admin person can produce a sign off report to find out which subjects have been signed off.
Q: Is it essential to moderate my subject every session?A: This question should be referred to your Faculty but the Moderation policy states that all subjects must be moderated at least once per year