Pdf standard work in higher education creating rubrics to

Pdf File 44.27 KByte,

  • PDF Police Officer Perspectives on Higher Education: Is the ...
  • PDF A Pocket Style Manual, 2016 MLA Update Edition Free Ebooks
  • Making Apprenticeships Work - Inside Higher Ed

Making Apprenticeships Work - Inside Higher Ed

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
Standard Work in Higher Education:
Creating Rubrics to Improve Student Success
Heather McCain
University of Kansas
Abstract
Quality requirements are not easy to define no matter what business you are in. Defining quality is manufacturing can be difficult and trying to define quality in higher education is equally, if not more, difficult. In manufacturing standard work is used to create consistency and define quality. In higher education standard work are those procedures and practices that could help create consistency and define quality. Items such as a course syllabus and course timeline are used to help students understand what to do and when to complete assignments. Although students receive the syllabus they still may not know what it takes to get a good grade on an assignment. Lecture notes may help clarify the quality requirements for assignments beyond the syllabus. Another way instructors can convey requirements is by developing rubrics. This paper suggests that rubrics can help improve student satisfaction by creating consistency and providing quality requirements.
Keywords
Rubrics; Quality; Standard Work
Introduction
Quality requirements are not easy to define no matter what business you are in. Defining quality is manufacturing can be difficult and trying to define quality for education is equally, if not more, difficult. Philip Crosby (1979) defined quality as "conformance to requirements".1 Russ Westcott suggests that "Quality - I'll know it when I see it"2 is used by customers to define quality. As a student at a university, conformance to requirements typically means following the course syllabus and completing the assignments for the course. As an instructor, quality of a student's work sometimes follows the "I'll know it when I see it."
In manufacturing standard work is used to create consistency and define quality. Standard work is defined as documented and agreed-upon procedures and practices.3 In higher education, standard work are those procedures and practices that could help create consistency and define quality. Items such as a course syllabus and course timeline are used to help students understand what to do and when to complete assignments. Although students receive the syllabus they still may not know what it takes to get a good grade on an assignment. Lecture notes may help clarify the quality requirements for assignments beyond the syllabus. Another way instructors can convey requirements is by developing rubrics. Quality requirements should be the specific criteria as to what is required for the student to receive a specific grade. Criteria are abstract ideals to which students (ought to) aspire, and against which one hopes to assess student performance.4
1 ? American Society for Engineering Education, 2015

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
Rubrics
A rubric articulates the expectations for the object being evaluated by listing the critical criteria of what is deemed as necessary and assesses the levels of quality from poor to excellent.5 Much of the literature on evaluation criteria and feedback to students is found in articles written by D. Royce Sadler. He is also referenced in many articles on the subject. He found that to develop knowledge students must understand the meaning of feedback with relationship to task compliance, quality, and criteria. Students need to identify the particular aspects of their work that need attention. Feedback should help the student understand more about the learning goal, more about their own achievement status in relation to that goal, and more about ways to bridge the gap between their current status and the desired status. Although the students may accept a teacher's judgment without question, they need more than summary grades if they are to develop expertise intelligently.6
Just as standard work helps manufacturing employees understand requirements, rubrics help students, as well as other instructors, understand the requirements. Researchers have noted that scoring rubrics help define "quality",7 provide expectations up-front,8 provide feedback about strengths and weaknesses in student work,4 monitor student performance,7 and support assessment for learning9 including student self-assessment.4 A rubric structure is usually presented in the form of a matrix, where the criteria and the levels of performance associated with each criterion indicates the explicit task and learning requirements.10 Mertler discusses the two types of rubrics: holistic and analytic.11 The difference between a holistic rubric and an analytic rubric is how the assignment is scored. Holistic rubrics grade the assignment as a whole. Analytic rubrics grade individual parts of the assignment, then sum the individual scores to obtain a total score.8
If you decide to use a holistic rubric, Mertler suggests writing narrative descriptions for excellent work and poor work incorporating each attribute in the description. For holistic rubrics, he suggests completing the rubric by describing other levels on the continuum that range from excellent to poor work for the collective attributes. For analytic rubrics, narrative descriptions are developed for each individual attribute. Levels are created on a continuum that range from poor to excellent work for each attribute. 11 To create a meaningful continuum, Walser suggests using a rubric ranging from 0 to 4 points, with a score of "3" serving as the fulcrum, representing the instructional goal, and a score of "4" representing work that goes beyond level 3 performance (Table 1).12 This idea came from a book called Making Standards Useful in the Classroom. The score of "4" represented "in-depth inferences and applications that go beyond what was taught."13 This rubric scale accommodates, rewards, and motivates more creative, innovative, and in-depth student performance beyond what was described as meeting an instructional goal. Once the rubric is created, Mertler suggests collecting samples of student work that exemplify each level. After reviewing student work, determine if the rubric is acceptable. If it is not, revise the rubric as necessary for improvement.11
2 ? American Society for Engineering Education, 2015

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
Table 1. Standard Rubric for Study Guide Assignments for Teacher, School and Society Course.12

Grade 4 3
2 1 0

Criteria Demonstrates in-depth understanding of Study Guide content that goes beyond "3" performance criteria. Clearly demonstrates understanding of Study Guide content.
Responses to questions are complete, accurate and appropriate.
Communication is clear with minimal spelling and grammatical errors.
Study Guide is completed and submitted in the specified format by the deadline.
For the most part, demonstrates understanding of Study Guide content. For the most part, does not demonstrate understanding of Study Guide content. Does not demonstrate understanding of Study Guide content at all OR does not complete and submit Study Guide within 48 hours of the deadline.

Atkinson and Lim found that students benefitted from the use of the rubric as they were clearly able to see what they needed to do, what they had achieved, and what they needed to do to improve.10 A rubric clarifies the instructor's key categories of emphasis because the criteria and their weighting indicate the required content and the focus. The levels of performance and the associated descriptors provide further detail of what is required to meet and exceed expectations. Making the rubric available early to students in the assessment process allows time for them to consider and reflect upon what is required based on factors such as time given, resources and their aptitude. The weighting of criteria and the levels of achievement allow students to plan their workload appropriately.
Rubrics assist accountability by providing a measure to which works are compared. By creating an object that serves as an example of quality and using a scale from poor to excellent, all the stakeholders are able to make decisions based on a standard.14 According to Popham, "Rubrics represent not only scoring tools but also, more important, instructional illuminators. Appropriately designed rubrics can make an enormous contribution to instructional quality."15 Rubrics maintain and motivate high standards as well as promote fairness and assessment for learning. The structure and detail of the rubric allows the instructor to communicate to students which criteria to focus on and to motivate them to aspire to higher levels of performance. The learning is more directed and focused than without such a guide. The rubric can be used during the process of students undertaking an assignment, and then to assess the final product. If the rubric is well constructed, then the structure should also aid the provision of feedback and the efficiency of assessment.10 Students should not be surprised by feedback and grades if a rubric was utilized.

3 ? American Society for Engineering Education, 2015

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
Standard Work in Higher Education
Without standard work in manufacturing, quality and consistency can be difficult to preserve. The same issue prevails for large universities where multiple instructors are involved. Quality control can be difficult to maintain when multiple graders are involved. Many courses have multiple sessions so student grades are determined often using different criteria thus causing inconsistency. Rubrics provide information that can assist multiple instructors in grading assignments. With common information, grades will be more consistent. A rubric can also be of benefit where a single grader, making an assessment over a period of time, may be influenced by fatigue. From a management point of view, the moderation time saved through the use of a rubric in maintaining consistency, can also contribute to quality assurance associated with university accreditation processes.15
Rubrics have a long history of use in educational assessment16 and researchers have shown the added benefits of computerization using either personal computing software17, 18 or dedicated assessment systems.19 Many universities now use computerized Learning Management Systems. The BlackBoard Learning Management System is an example of a system that contains rubrics. Rubrics in BlackBoard are grids consisting of rows and columns. The rows are the criteria and the columns are the levels of success. Once setup, students can see the criteria for each assignment and instructors can add notes to the students. The calculations are automated and results with summary statistics are available for general feedback to students.10 Once created, rubrics in BlackBoard can be shared with other instructors to create standardization.
Technology tools such as BlackBoard have made the utilization of rubrics easier for instructors. Rubrics can help improve student satisfaction by creating consistency and providing quality requirements. Rubrics have become the part of standard work for higher education.
References
1. Crosby, Philip, Quality is free: the art of making quality certain / Philip B. Crosby. New York : McGraw-Hill, c1979.
2. Westcott, Russell, The Certified Manager of Quality/organizational Excellence Handbook. Milwaukee, Wisconsin: ASQ Quality Press, 2014, 261.
3. Westcott, R. and Duffy, G., The Certified Quality Improvement Associate Handbook. Milwaukee, Wisconsin: ASQ Quality Press, 2015.
4. Andrade, H., Using rubrics to promote thinking and learning. Educational Leadership, 57(5), 2000, 13?19.
5. Reddy, M. and Andrade, H., A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 2010, 435-448.
6. Sadler, D., Formative assessment and the design of instructional systems. Instructional Science, 18(2), 1989, 119?144.
4 ? American Society for Engineering Education, 2015

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
7. Goodrich, H., Understanding rubrics. Educational Leadership, 54(4), 1996, 14-17.
8. Moskal, B., Recommendations for developing classroom performance assessments and scoring rubrics. Practical Assessment, Research & Evaluation, 8(14), 2003, Retrieved from
9. Tierney, R. and Simon, M., What's still wrong with rubrics: Focusing on the consistency of performance criteria across scale levels. Practical Assessment, Research & Evaluation, 9(2), 2004. Retrieved from
10. Atkinson, D. and Lim, S., Improving assessment processes in Higher Education: Student and teacher perceptions of the effectiveness of a rubric embedded in a LMS. Australasian Journal Of Educational Technology, 29(5), 2013, 651-666.
11. Mertler, C., Designing scoring rubrics for your classroom. Practical Assessment, Research & Evaluation, 7(25), 2001. Retrieved November 12, 2014 from
12. Walser, T., Using a Standard Rubric to Promote High Standards, Fairness, Student Motivation, and Assessment for Learning. Mountainrise, 6(3), 2011, 1-13.
13. Marzano, R. and Haystead, M., Making standards useful in the classroom. Alexandria, VA: ASCD, 2008.
14. Fitzpatrick, J., Sanders, J., & Worthen, B,. Program evaluation: Alternative approaches and practical guidelines. Princeton, NJ: Pearson, 2001.
15. Popham, J., What's wrong--and what's right--with rubrics. Educational Leadership, 55, 1997, 7275.
16. Dirlam, D. and Byrne, M., Standardized developmental ratings. Paper presented at the Annual Meeting of the American Educational Research Association, Toronto, Ontario, Canada, 1978.
17. Mitri, M., Automated rubric generation and analysis for assurance of learning tasks. Issues in Information Systems, 6(1), 2005, 296-302. Retrieved from .
18. Czaplewski, A., Computer-assisted grading rubrics: Automating the process of providing comments and student feedback. Marketing Education Review, 19(1), 2009, 29-36.
19. Anglin, L., Anglin, K., Schumann, P., & Kaliski, J., Improving the efficiency and effectiveness of grading through the use of computer-assisted grading rubrics. Decision Sciences Journal of Innovative Education, 6(1), 2008, 51-73.
Heather McCain
Heather McCain is currently a Professor of the Practice at the University of Kansas and the sole proprietor of hjm Training and Consulting. She works with companies to implement continuous improvement methodologies and provides training on quality, project management, and process improvement. She was the Quality Manager for Consumer Product at Garmin International, a Consulting Engineer at Hallmark Cards, and held various positions AlliedSignal (now Honeywell) Aerospace and Automotive. She has a BS in Electrical Engineering from Kansas
5 ? American Society for Engineering Education, 2015

2015 ASEE Zone III Conference (Gulf Southwest ? Midwest ? North Midwest Sections)
State University and a MS in Engineering Management from The University of Kansas. She is pursuing her PhD in Technology Management from Indiana State University.
6 ? American Society for Engineering Education, 2015

Download Pdf File Online Preview