Mastery Grading: System Updates

My first three posts were about my motivation, initial design, and first implementation of my mastery-based grading system and now I want to talk about the purposeful changes I made last semester and how the system was impacted by the shift to remote learning.

I taught the same course (Math and Politics – a general education course, see this post for more information and my initial mastery grading setup) so I was able to make direct changes to various components of the mastery system based on the initial experience and my reflections (see this post).

Objectives

I made a few minor changes to specific objectives (consolidating, eliminating, rewording, etc.) but a more significant change was converting assignments to objectives to (hopefully) give them more value by identifying the main learning goals I sought from the assignments and placing them alongside the other content goals for each of the units. The assignments included completing practice problems (students chose which ones to write up from a larger set) and writing assignments similar to those I had assigned in the past.

As mentioned in my previous reflection, I also added a category of “overall objectives” which included attendance, participation, and four brief reflections that were not tied to a specific unit in the course but that I view as valuable in a general education math course (the relationship between math and privilege, productive struggle, growth mindset language, and a final reflection on the course).

This time around I had a total of 80 course objectives:

  • Voting Theory – 7 procedural, 10 conceptual, 3 from assignments (20 total)
  • Apportionment – 7 procedural, 9 conceptual, 3 from assignments (19 total)
  • Redistricting – 9 procedural, 7 conceptual, 3 from assignments (19 total)
  • Game Theory – 8 procedural, 8 conceptual (16 total)
  • Overall Objectives – attendance, participation, 4 reflections (6 total)

Final Grade Conversion

Rather than using a “bundle” approach as last semester, I decided to simplify the process by having a total number of objectives mastered correspond to a final letter grade. This makes it much clearer to students where they stand at all times and does not distinguish (grade-wise) between the procedural and conceptual objectives. After going through the spreadsheets from last semester I didn’t think it was necessary to separate them in order to translate to a final grade but I still labeled them in their respective categories to identify the different types of (mathematical) thinking that correspond to the objectives.

The updated grade system was as follows:
80 (out of 80) objectives mastered = A
77-79 = A-
74-76 = B+

59-61 = C-

50-52 = D-
< 50 = F

Tokens

I gave each student 10 (digital) tokens for the semester where each token could be used to retake any objective, submit an assignment late, or resubmit an assignment. In an attempt to motivate higher quality assignments (beyond the minimum needed to satisfy the objectives), I also awarded an additional token to students that submitted excellent work.

Spreadsheet

This looked similar to last semester with a couple of changes. I added a “progress bar” on the top with boundaries for the different letter grades and a display of their total number of objectives mastered and the corresponding final grade. I also added checkboxes and formatting to the “token bank” to make it easier for myself and add another visual element for students. (Click to enlarge the screenshot below.)

Remote Learning

Given the circumstances, the transition to remote learning amidst the pandemic actually happened at a good time in the course because I had just given the exam on the second unit (it was right before our scheduled Spring Break) meaning I could start fresh on the remaining two units in a new format. For the first two units, I had constructed guided note packets that we would work through in class and quizzes and exams were used to assess the objectives.

For this class, I decided to go the asynchronous route and make videos that went through the notes and examples and then created assignments that corresponded to the videos and the objectives. I also wanted to give students as much flexibility as possible with all the challenges they faced with remote learning and various life situations. As a result, I turned the content into weekly modules where I would post some videos (myself narrating and working through problems on my tablet) and an assignment on Monday which would be due the following Sunday. With 6 weeks of remote learning in the semester and two full units remaining, this worked out nicely in terms of having 3 modules for each unit. (Notice if you look closely at the spreadsheet screenshot above, there are no “procedural” and “conceptual” labels for the last two units as the objectives are organized by the weekly modules instead.) I did not give any quizzes or exams and mastery of objectives was completely based on grading their assignments and the questions that corresponded to the objectives. In terms of accessibility I had virtual office hours each week at set times where students could discuss that week’s assignment with myself and each other.

Another outcome of remote learning is that I did not give a final exam which I did last semester as a “final opportunity” to earn any missing objectives and instead I scheduled individual virtual meetings with students for them to use their tokens. Given the circumstances and stress I did not give them new problems to demonstrate mastery and I simply had them go through their old quizzes, exams, and/or assignments and explain to me why they got certain problems wrong and how the problem would be corrected. The students knew this ahead of time so that they could adequately prepare and this token use is not a structure I would use in “standard” semesters but my human response was to ease some of the expectations in the trying times.

As an anecdote, I’ll add that the final distribution of grades was the most bimodal that I’ve ever had as about 60% of the students got A’s and the other 40% received C’s. I attribute this to students having the ability to change grades to pass/fail and some students with less favorable learning conditions had no problem taking this route to satisfy their general education math requirement with no negative impact on their GPA while others were motivated to earn an A that would boost their GPA.

Student Feedback

I just wanted to share a few things that students shared about their experience with mastery-based grading this semester:

  • “Tokens were great to fall back on but they were limited so it pushed me to study”
  • “The system allowed mistakes to be made and actually learned from”
  • “It was nice to study for understanding rather than for a grade”

Reaction to the grading system this semester was unanimously positive which is extremely encouraging and only continues to motivate me to extend such a system to other classes as well. Thanks for reading and I will continue to share my experiences in math education.

If you’re interested in trying mastery-based grading and want a spreadsheet template, I’m sharing mine with instructions as well: bit.ly/MasteryTemplate

An Apprentice in Mastery (Part 3)

Now that I explained the components of my mastery-based system to you in my last post and to the students on the first day of the semester, I want to share some thoughts about how this all played out in practice.

Changing the language and the narrative. This probably isn’t surprising, but it is interesting – students quickly adapted their language to match the language of the grading system. “Checkboxes” or just “checks” became the predominant vocabulary along with “tokens” and “understanding.” Early on, students knew what was expected to achieve mastery of an objective and most of their language matched my goal of improving student mindset (and aligned with the mindset language I regularly used in the classroom).

Homework Assignments. The initial plan was for these assignments to be a great opportunity to earn feedback towards your mastery of objectives and not impact your final grade, but because they were de-emphasized in the grades and only grading for completion, they slid further down my own list of priorities, especially with two other preps this semester. I moved these responsibilities to my TA for the class and I am also more convinced now than before that it is okay for homework to be optional if students are able to demonstrate mastery in the end and that I should provide them with practice problems but not force them to turn in the problems. The shift in language to “practice problems” might have a positive effect as well, especially if there are ways for students to quickly self-check their answers/progress (such as through an online “quiz”).

Writing Assignments. Unsurprisingly, the quality of these assignments decreased since students just needed to efficiently answer the questions to earn a procedural objective rather than need a more comprehensive write-up as a percentage of their grade. With that being said, I still believe that the content in these assignments brings the important political context to my course, so my plan is to adjust and combine these assignments to mini-research projects with more corresponding checkboxes to increase the quality and emphasize the importance of the political context. My initial thought is to create one of these per unit (four in total) while keeping the Redistricting project as one of the four assignments.

Grading. I can only remember two small instances of students contesting any grading and both were cases of them arguing that they demonstrated enough mastery to be awarded a check on an objective from an exam question. As you might imagine, the grading process itself was certainly quicker than in previous semesters. While there are some cases that are borderline in terms of determining whether a student’s reasoning sufficiently demonstrates mastery of an objective, many are clear-cut and the lack of any partial credit accelerates the grading process.

“What’s my current grade in the class?” It’s a question we get all the time, and one that isn’t usually too hard to answer – it’s whatever the grade formula gives you based on the assignments and tests so far. Unfortunately, this question was more difficult for me to answer (other than in the extreme cases) using this system because it was so dependent on your performance on each upcoming objective and unit and because most of the students still had tokens in the bank until the end of the semester. As long as I can find a nice way to do this in Google Sheets, my current plan is to implement a “progress bar” that automatically shows your progress towards final letter grades based on how many objectives you have mastered at any point.

Token system. A few students chose the strategy of retaking objectives shortly after exams because they were fresher in their minds, but the vast majority saved all of their tokens until Finals week as I allowed them to use the tokens at any point before the final. I always talked about token strategies with the whole class and with individuals to guide them in ways that I felt made the most sense for each student. Common strategies I emphasized were (i) save them until the end, (ii) retake procedures first (based on final grade implications), and (iii) retake objectives from the same unit (less content to review if taking the final).

Up until Finals week, I had no issues managing token use as not too many were used and I would generally write out a couple of questions or have an oral exam/conversation for students to demonstrate their understanding of whatever objective(s) they were coming in to retake. But with a class of 30 students and roughly 100 tokens collectively still to use (some students didn’t need any or needed less than 5 because they had already achieved all of the objectives), Finals week was overwhelming to say the least. My “solution” was to create a sign-up so that no more than 4-5 students would be in the office at once, but I ultimately needed 12 hours (4 hours each on 3 straight days) to complete all the retakes. This was by far the most inefficient element of the system and my plan is to create a digital repository of questions (rather than a mental one) to draw from for retakes and essentially just print out quizzes to make this process go (hopefully) smoother than this first time.

A smaller change in the system is to have a 1-1 ratio instead of a 1-2 ratio of tokens to objectives that can be retaken. I still think that retaking up to 10 objectives “feels right” and although it is arbitrary to an extent, again my goal was to balance initial motivation with growth mindset and opportunities to learn at your own pace over time so my plan is to give 10 tokens this time around. This also has the effect of slightly decreasing the impact of any “bonus tokens” which I will probably add a couple more opportunities to earn as well to hopefully increase motivation on assignments.

Overall Objectives. To further streamline the final grade conversion, I believe that it is more sensible to convert holistic aspects such as attendance and participation from final grading categories to “overall objectives” that would carry the same weight as a procedural or conceptual objective. Placing a small amount of these overall objectives on the spreadsheet would allow every component of the class to be visible (thus “tangible”) to the students.

Student Reflections. At the end of the semester, I gave a survey to the students about their thoughts and experiences on various components of the mastery-based grading system and are some pieces of feedback that best capture the class as a whole:

  • “clear visual of what I accomplished and what I needed to work on”
  • “less stressed because I know I can redo checks”
  • “focused more on understanding instead of memorizing material”

The transparency of the system is what students talked about most, and this is encouraging as one reason to use this system is to keep everyone closer to the content of the course. Lowering test anxiety is also a critical feature as I always try my best to not only be responsive and reactive, but proactive to the mental health concerns of students. And setting up the two categories of procedural and conceptual objectives was done to emphasize the importance of conceptual understanding.

Students also offered suggestions for improvements to the system such as additional tokens and partial credit opportunities, but no single suggestion appeared more than twice. I should also mention that 20 of the 24 students that responded to the survey (from a class of 30) said that they liked or really liked (the top 2 choices from a 4-point Likert scale) the use of mastery-based grading.

Stay tuned for more updates as I implement changes to the system this semester based on my experiences. Thanks for reading and I hope this makes you think more about your own course objectives and assessment. From one reflective practitioner to another.

An Apprentice in Mastery (Part 2)

For my second post, I want to describe for you the mastery-based system that I used for my first attempt, but before I do that, let me give you a quick overview of the course in which it was used.

Course: Math 114 – “Math and Politics”

This is a general education course and one that I taught in each of my previous two semesters at Trinity. The course content and assessments have evolved through the semesters, with the course covering the following four units (some associated topics):

  • Voting Theory (unweighted voting methods, fairness criteria, weighted voting)
  • Apportionment (methods, history, paradoxes, voting power)
  • Redistricting (overview, efficiency gap, compactness)
  • Game Theory (Nash equilibrium, simultaneous and sequential games, mixed strategies)

Before implementing the mastery-based system, assessment in this course consisted of unit exams, regular homework assignments, written responses to articles relating the topics to politics, and a group project.

Keeping in mind my goals from the last post of procedures vs. concepts, simple and transparent, and including a retake system, here’s the grading system I used in Fall 2019:

Learning Objectives

I translated content goals from having taught the course before into explicit learning objectives in which students must display mastery. Each unit had their own set of objectives with half of them categorized as “Procedural” and the other half categorized as “Conceptual” objectives based on the associated level of thinking. There was a total of 76 objectives split almost equally across the four units:

  • Voting Theory – 10 Procedural, 10 Conceptual
  • Apportionment – 10 Procedural, 10 Conceptual
  • Redistricting – 10 Procedural, 10 Conceptual
  • Game Theory – 8 Procedural, 8 Conceptual

Assessments

These objectives were assessed through in-class quizzes and exams. For the first two units, two quizzes were given only covering procedural objectives and the exam covered all of the conceptual objectives. Part of the motivation for this was having 50 minute class periods and not wanting students to use any of that time going through any potentially tedious (procedural) calculations. For the last two units, the quizzes and exams covered a mix of objectives, but this was a result of the way the content was organized as opposed to an intentional shift in structure.

The regular homework assignments were maintained but could not count towards demonstrating mastery. They were graded simply on based on completion and were used as an opportunity for students to gain practice and feedback on their progress towards the objectives rather than as a higher-stakes portion of their grade based on correctness. See below for how these assignments contributed to their final letter grade.

The written responses were maintained but streamlined for students to only answer questions that I viewed as the most crucial to get out of the article and to briefly reflect on the political context. The main point of each article (one per unit) was included as one of the procedural objectives and successful completion of the assignment was enough for students to demonstrate mastery of that objective.

The group project was also maintained and streamlined compared to the previous semesters. The project was for students to create their own new congressional district map for a state using an online web application and to write a brief report on the history of redistricting in that state and to show the efficiency gap and compactness calculations of their proposal which they had learned in that unit. The main goals of the project were translated into 3 of the 10 conceptual objectives for that unit.

Spreadsheet and Checkboxes

To ensure that students had easy access to their objectives at all times, I created a Google spreadsheet with all objectives separated by unit and separated by procedural/conceptual. Each objective had a checkbox next to it that I would mark when students achieved that objective and I set up conditional formatting to highlight the objective green when it was mastered.

Logistical note: I created one master spreadsheet file with a tab for each student and then a separate file for each student that was shared with them and automatically synced to their tab on the master file. This is why I had to use Google Sheets because Excel Online doesn’t support syncing from another file. Below is a screenshot of the spreadsheet (click to enlarge).

Tokens

To address retakes and support student’s control over their learning, I created a digital token system and gave each student 5 tokens to use for the semester. Each token could be used to retake up to 2 objectives or to submit a homework assignment later than the deadline. There was also an opportunity to earn an extra token for an excellent group project report. Students could retake objectives at any time during the semester and could retake any objective multiple times if needed. Using a token to retake objectives consisted of coming to office hours and me providing them with new questions related to the objectives for them to demonstrate mastery. As you may have noticed, tracking their tokens was also a part of each individual’s spreadsheet.

As you might imagine, many of these tokens were used at the end of the semester as students saved them to see how many objectives they would need to retake. More on this and other reflections coming in the next post!

Final Grades

To convert a student’s mastery of objectives into a final grade, I created and used the conversion table below which factored in the number of objectives checked off in the procedural and conceptual categories separately as well as the number of assignments completed and their attendance and participation in the class. You can see the table below taken from the course syllabus (click to enlarge) where your final grade would be the first row in which you met all four categories. For example, if you completed all assignments, attended and participated in class, mastered 35 procedural objectives, and mastered 29 conceptual objectives, your final grade would be a C+ as you would not have achieved every column in any of the rows above that grade.

Part 3 will be some of my reflections on how various components of this system actually worked and thinking about some changes for next semester!

An Apprentice in Mastery (Part 1)

In addition to creating a new course this past semester, I also took on another creative teaching endeavor by implementing mastery-based grading in one of my courses for the first time. I’m going to write a short series of posts talking about my approach and my experiences:

  1. Motivation guiding my design (this post)
  2. Implementation of my system
  3. Reflections and moving forward

If you’re reading this, I’m not assuming you have any background in what mastery-based grading is, but I don’t want to spend too much time explaining the various components because they will mostly be revealed through my reasoning and approach. However, it is important that I would describe the key tenets of such a system as (a) assessment and grades are based on the understanding (“mastery”) of learning objectives rather than a traditional numerical scale, and (b) there should be a way for students to have additional attempts to demonstrate mastery.

So why did I want to do something like this in the first place?

  • Giving grades more meaning. When you give a final grade in a course, how does it correspond to what the student actually has accomplished? For example, in a traditional grading system a B could translate to (i) getting everything correct the entire semester but missing a few assignments, (ii) getting B’s on the vast majority of assessments, or (iii) having a wide variety of grades that average to a B in the end. A mastery-based approach treats a course like a license or certification in that you have to demonstrate certain skills to earn the license. Grades then translate to how many of the skills you have demonstrated mastery in and no matter the grade, you will have a tangible list of objectives that you have learned as a result of taking the class.
  • Student mindset. It is always important to attend to students’ mental dispositions and a mastery-based system (in theory) has the ability to make positive impacts on the mindset and anxieties of our students. It is one thing to preach the value of a growth mindset and it is another to align your practices with this value. Allowing students additional opportunities to demonstrate their understanding of learning objectives embodies the growth mindset philosophy of encouraging productive struggle and that learning (especially at the level of mastery) can take different amounts of time for different types of learners. In addition, such a system would hopefully reduce test anxiety where students would know that it is okay to not understand everything at the time of an exam and that they won’t be penalized or judged as inferior for attempting some of the objectives at a later time (and hopefully with additional motivation to achieve mastery).
  • Discourage academic dishonesty. When reading perspectives of others who have utilized mastery-based grading, you probably won’t see this as a reason to employ the system, but I try to be as transparent as possible about my own thought process. In math, homework traditionally counts for 15-30% of your final grade and in my opinion, this only encourages the academic dishonesty that is rampant today (this could be a whole blog post on its own). From a student’s perspective, imagine sitting down the night before a homework assignment is due and not knowing how to do half of it. If the homework is graded for correctness, you would ask a friend for the answers or Google them too, wouldn’t you? For the record, I’m not faulting the student at all for doing this, especially given their stress associated with courses, extra-curriculars, and social life. So academic dishonesty is probably the wrong term, but why use a system that encourages a strategy that is not conducive to making progress towards actually understanding course content? I think it is fair to say that we would like final grades to be (as much as possible) associated with a student’s level of understanding and not the understanding of their friends or people online. We should be encouraging the use of these resources as a means of developing understanding rather than having students feel anxious because they are “cheating” on their homework.
  • Instructor time? In reading perspectives of others, “saving the instructor time” is a commonly cited reason for implementing a mastery-based system, but in my opinion, this isn’t a reason to make a pedagogical change. Yes, as professors we have a lot on our plates and would like to be more efficient, but I would never prioritize this over doing something because I believe it will benefit students in one way or another. In reality, any changes we make will take a lot of time initially and take less time as we gain experience and fine-tune our approaches. I will be expanding on this in one of my next posts, but for now I will say that my implementation this semester added time on the front end in determining objectives, took less time to grade during the semester, and added time in administering retakes.

With these motivations in mind, here were the initial principles I had in mind for designing the structure of my mastery-based grading system:

  • Separating procedural and conceptual thinking. I believe that being explicit about the cognitive demand level we expect from our students is important and I wanted to honor the way in which the mathematics education community differentiates procedural (following steps) and conceptual (displaying reasoning / non-routine problems) thinking.
  • Simple and transparent. The last thing students want is a complicated grading system, especially when it is different from what they are used to, so I wanted to make sure that it was easy to understand how their understanding of objectives would translate to a final grade and that they could see (online) at all times which objectives they have (and have not) mastered.
  • Retake system. This is a key feature of mastery-based grading, so I knew I wanted to incorporate a retake system with a personal goal of trying to find the “sweet spot” of not having too few opportunities for retakes (wouldn’t alleviate the test anxiety issue) and not having too many opportunities for retakes (could take away initial motivation to learn).

I also wanted to figure out a way to incorporate my existing assignments into the system, either as a category or as their own objectives. This is a good time to mention that I did this in a course I had taught the previous two semesters and I think it is much easier to go through the challenging process of identifying learning objectives if you have taught the course before so that you are closer to the content and student outcomes.

In my next post, I will talk about the course, the specifics of the mastery-based system that I created and used, and how the system played out during the semester. Thanks for reading!