Charles Sturt University
Charles Sturt University

What is quality assessment design?

Quality assessment design plays a vital role in the education process. A well-designed and well-communicated assessment task sets clear expectations for students. The task should provide opportunities for students to self-monitor, rehearse, practice, and receive feedback. It also has an influence on what, how, and how much a student will learn. Quality assessment design promotes lifelong learning practices by helping students gain a deep understanding of their subjects, and develop their capacity to judge their own and others' work

Criterion-referenced standards-based assessment (CRSBA) at CSU allows students to judge the quality of their work against pre-determined benchmarks. Judgment is untainted by prior performance and is independent of how other students perform in the same task. Each grade is assigned as a measure of their achievement of the subject and course learning outcomes (Sadler, 2005).

Designing CRSBA involves the alignment of assessment tasks to subject learning outcomes and to pre-defined criteria with related standards of skills, knowledge and competencies. In other words, learning outcomes for courses and subjects are linked to assessment tasks, and the marking criteria tell students what is expected of them to pass and perform well.

Quality assessment designing ensures that:

  1. Assessment requirements, and marking criteria and performance standards are clearly communicated to students before the session commences;
  2. Marking is more consistent across different offerings, campuses and cohorts;
  3. Feedback to students is more streamlined and results in fewer queries from students.

Use the following checklists to see how your subject aligns with the Assessment Principles Policy.

References

Sadler, R (2005). Interpretations of criteria-based assessment and grading in higher education.Assessment & Evaluation in Higher Education,30(2), 175-194.

Information about criterion referenced, standards based assessment can be found at:

Why use criterion referenced assessment?

Research shows that criterion referenced standards bases assessment works (see below). There are a range of benefits that criterion-referenced standards-based assessment provides for both teaching staff and students.

For students:

  • Provides greater transparency and satisfaction in the assessment process
  • Students clearly understand the expectation on what needs to be produced to attain a certain grade
  • Helps students understand where they sit on the continuum of performance measure
  • Criteria and standards provide the platform for students to be reflective on their own learning and assess their own work
  • The process is fairer as students are graded according to their own knowledge and achievements, not to the performance of their peers
  • Assessment is more predictable and consistent as grades are awarded bases on pre-defined criteria only
  • Encourages higher order learning as students understand the expectations.

For staff:

  • Provides shared understanding between students and teaching staff as to what is requires in an assessment.
  • Describing clear expectation of performance across different grade levels often results in less student queries over grading decisions.
  • Greater consistency of marking across different markers, campuses and cohorts.
  • Clearly defined criteria provide a good guide for developing your teaching strategies
  • Reviewing the criteria and standards post-delivery helps staff develop batter assessment strategies for next time
  • Working collaboratively in course teams ensures assessment strategy is aligned across the course and helps everyone with professional development
  • Provides an opportunity to review and improve assessment at both the course and subject level.
  • Provides an opportunity for professional growth and sharing amongst peers.

So what evidence is there that CRSBA is better than what we were already doing?

A lot. For example, Jonsson and Svingby (2007) conducted a literature review to identify evidence of the benefits of criterion referenced standards based assessment. While their work focuses on rubrics, using the definition that a rubric is a scoring tool for the qualitative rating of complex student work, and "includes criteria for rating important dimensions of performance, as well as standards of attainment for those criteria", this definition fits well with the requirements of the CSU Assessment Principles policy. The review included 75 studies and drew the following conclusions:

"the reliable scoring of performance assessments can be enhanced by the use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training; ... rubrics seem to have the potential of promoting learning and/or improve instruction. The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment."

Wiggins (1998) suggests that students' ability to adjust their work by self-assessing against the criteria and standards is one of the key benefits of CRSBA. The feedback then received as part of the marking process completes the process; students can see whether their self-assessment matches up with the expert (or lecturer's) judgement of their performance. This builds a student's capacity for judging and identifying high quality work (both in themselves and others), which Boud (2010b) argues is necessary for effective lifelong learning. Students, particularly in first year, may need guidance from lecturers/teaching staff on how to read, use and apply a rubric while they prepare and edit their assignments.

Another review of empirical studies by Pandero and Jonsson (2013), which uses the same definition of a rubric as Jonsson and Svingby (2007), identified that criteria and standards can improve students' performance through:

"(a) providing transparency to the assessment, which in turn may (b) reduce student anxiety. The use of rubrics may also (c) aid the feedback process, (d) improve student self-efficacy, and (e) support student self-regulation; all of which may indirectly facilitate improved student performance."

What's just as important is that students like the guidance provided by criteria and standards, and feel more confident in the fairness of the marking, as found by Laurian and Fitzgerald's (2013) study. After experiencing assessment tasks with and without criteria and standards, a class of university students were asked to complete a survey rating their thoughts about using rubrics. A summary of responses is presented below.

StatementResponses
I like to have a rubric to help me in my work 90% agree or totally agree, 10% neutral
A rubric helps me to raise the standards for my work 81% agree or totally agree, 19% neutral
A rubric stifles my creativity 70% totally disagree or disagree, 20% neutral, 10% agree
In this class the rubric helped the professor to grade more fairly 85% agree or totally agree, 15% neutral
A rubric helps me to organise my work 85% agree or totally agree, 15% neutral
A rubric helps me to self assess my work before I pass it in to the professor 95% agree or totally agree, 5% neutral
In this class my work was better when I used a rubric 80% agree or totally agree, 20% neutral

With all this talk of rubrics, many people assume CRSBA requires a rubric, but this isn't the case; it is up to you (or your School or Faculty policy) whether you use a rubric or not, a rubric is simply a different way of organising your criteria and their associated standards. What is crucial under the CSU policy is that you provide marking criteriaanddescriptions of the performance standards.

Critics of CRSBA, and especially of rubrics, believe it stifles innovation, however Wiggins (1998) argues only badly designed rubrics do this and instead we should design criteria that assesses impact and purpose – has the student's performance achieved the desired end result? – rather than "mandating ... process, format, method, or approach".

As with all things in life, attitude plays a large role in the effectiveness of using criteria and standards. A study by Kutlu, Yildirim and Bilican (2010) found that among 292 primary school teachers in Ankara, Turkey, those who had a positive attitude to rubrics found a variety of uses for them, including providing feedback to students (60%), observing the extent that students used their knowledge and skills in real life situations (56%), identifying deficiencies in classroom teaching (50%), grading students (50%), observing the development of students' high level thinking skills (45%) and restructuring in-class teaching activities (28%). While teachers with negative attitudes towards rubrics used all of these approaches to a much lesser extent, interestingly grading students – which came in fourth place for teachers with positive attitudes – was top-ranked by those teachers with negative attitudes towards rubrics. The study also found that those teachers who felt they had the most knowledge about rubrics were more positive, suggesting more knowledge equates to higher satisfaction with and acceptance of criteria and standards.

The resources listed below will add to your knowledge of this area, and they may also help you find a rationale for CRSBA that fits with your personal philosophies of learning and teaching in higher education. As Kutlu, Yildirim and Bilican (2010) found, greater knowledge brings with it greater positivity, and given that criteria and standards are here to stay, taking some time to examine the literature may help you feel more comfortable with criterion reference standard base assessment.

References    

Boud, D. (2010a).Assessment Futures.Boud, D. (2010b).Assessment 2020: Seven propositions for assessment reform in higher education.Jonsson, A., & Svingby, G. (2007).consequences. Educational Research Review, 2,130-144.

Kutlu, O., Yildirim, O., & Bilican, S. (2010).The comparison of the views of teachers with positive and negative attitudes towards rubrics. Procedia – Social and Behavioral Sciences, 9, 1566-1573.

Laurian, S., & Fitzgerald, C. J. (2013).Effects of using rubrics in a university academic level Romanian literature class. Procedia – Social and Behavioral Sciences, 76, 431-440.

Pandero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129-144.

Wiggins, G. (1998).Educative assessment: Designing assessment to inform and improve student performance (pp. 153-185). San Francisco: Jossey-Bass Publishers.