An Apprentice in Mastery (Part 3)

Now that I explained the components of my mastery-based system to you in my last post and to the students on the first day of the semester, I want to share some thoughts about how this all played out in practice.

Changing the language and the narrative. This probably isn’t surprising, but it is interesting – students quickly adapted their language to match the language of the grading system. “Checkboxes” or just “checks” became the predominant vocabulary along with “tokens” and “understanding.” Early on, students knew what was expected to achieve mastery of an objective and most of their language matched my goal of improving student mindset (and aligned with the mindset language I regularly used in the classroom).

Homework Assignments. The initial plan was for these assignments to be a great opportunity to earn feedback towards your mastery of objectives and not impact your final grade, but because they were de-emphasized in the grades and only grading for completion, they slid further down my own list of priorities, especially with two other preps this semester. I moved these responsibilities to my TA for the class and I am also more convinced now than before that it is okay for homework to be optional if students are able to demonstrate mastery in the end and that I should provide them with practice problems but not force them to turn in the problems. The shift in language to “practice problems” might have a positive effect as well, especially if there are ways for students to quickly self-check their answers/progress (such as through an online “quiz”).

Writing Assignments. Unsurprisingly, the quality of these assignments decreased since students just needed to efficiently answer the questions to earn a procedural objective rather than need a more comprehensive write-up as a percentage of their grade. With that being said, I still believe that the content in these assignments brings the important political context to my course, so my plan is to adjust and combine these assignments to mini-research projects with more corresponding checkboxes to increase the quality and emphasize the importance of the political context. My initial thought is to create one of these per unit (four in total) while keeping the Redistricting project as one of the four assignments.

Grading. I can only remember two small instances of students contesting any grading and both were cases of them arguing that they demonstrated enough mastery to be awarded a check on an objective from an exam question. As you might imagine, the grading process itself was certainly quicker than in previous semesters. While there are some cases that are borderline in terms of determining whether a student’s reasoning sufficiently demonstrates mastery of an objective, many are clear-cut and the lack of any partial credit accelerates the grading process.

“What’s my current grade in the class?” It’s a question we get all the time, and one that isn’t usually too hard to answer – it’s whatever the grade formula gives you based on the assignments and tests so far. Unfortunately, this question was more difficult for me to answer (other than in the extreme cases) using this system because it was so dependent on your performance on each upcoming objective and unit and because most of the students still had tokens in the bank until the end of the semester. As long as I can find a nice way to do this in Google Sheets, my current plan is to implement a “progress bar” that automatically shows your progress towards final letter grades based on how many objectives you have mastered at any point.

Token system. A few students chose the strategy of retaking objectives shortly after exams because they were fresher in their minds, but the vast majority saved all of their tokens until Finals week as I allowed them to use the tokens at any point before the final. I always talked about token strategies with the whole class and with individuals to guide them in ways that I felt made the most sense for each student. Common strategies I emphasized were (i) save them until the end, (ii) retake procedures first (based on final grade implications), and (iii) retake objectives from the same unit (less content to review if taking the final).

Up until Finals week, I had no issues managing token use as not too many were used and I would generally write out a couple of questions or have an oral exam/conversation for students to demonstrate their understanding of whatever objective(s) they were coming in to retake. But with a class of 30 students and roughly 100 tokens collectively still to use (some students didn’t need any or needed less than 5 because they had already achieved all of the objectives), Finals week was overwhelming to say the least. My “solution” was to create a sign-up so that no more than 4-5 students would be in the office at once, but I ultimately needed 12 hours (4 hours each on 3 straight days) to complete all the retakes. This was by far the most inefficient element of the system and my plan is to create a digital repository of questions (rather than a mental one) to draw from for retakes and essentially just print out quizzes to make this process go (hopefully) smoother than this first time.

A smaller change in the system is to have a 1-1 ratio instead of a 1-2 ratio of tokens to objectives that can be retaken. I still think that retaking up to 10 objectives “feels right” and although it is arbitrary to an extent, again my goal was to balance initial motivation with growth mindset and opportunities to learn at your own pace over time so my plan is to give 10 tokens this time around. This also has the effect of slightly decreasing the impact of any “bonus tokens” which I will probably add a couple more opportunities to earn as well to hopefully increase motivation on assignments.

Overall Objectives. To further streamline the final grade conversion, I believe that it is more sensible to convert holistic aspects such as attendance and participation from final grading categories to “overall objectives” that would carry the same weight as a procedural or conceptual objective. Placing a small amount of these overall objectives on the spreadsheet would allow every component of the class to be visible (thus “tangible”) to the students.

Student Reflections. At the end of the semester, I gave a survey to the students about their thoughts and experiences on various components of the mastery-based grading system and are some pieces of feedback that best capture the class as a whole:

  • “clear visual of what I accomplished and what I needed to work on”
  • “less stressed because I know I can redo checks”
  • “focused more on understanding instead of memorizing material”

The transparency of the system is what students talked about most, and this is encouraging as one reason to use this system is to keep everyone closer to the content of the course. Lowering test anxiety is also a critical feature as I always try my best to not only be responsive and reactive, but proactive to the mental health concerns of students. And setting up the two categories of procedural and conceptual objectives was done to emphasize the importance of conceptual understanding.

Students also offered suggestions for improvements to the system such as additional tokens and partial credit opportunities, but no single suggestion appeared more than twice. I should also mention that 20 of the 24 students that responded to the survey (from a class of 30) said that they liked or really liked (the top 2 choices from a 4-point Likert scale) the use of mastery-based grading.

Stay tuned for more updates as I implement changes to the system this semester based on my experiences. Thanks for reading and I hope this makes you think more about your own course objectives and assessment. From one reflective practitioner to another.

An Apprentice in Mastery (Part 2)

For my second post, I want to describe for you the mastery-based system that I used for my first attempt, but before I do that, let me give you a quick overview of the course in which it was used.

Course: Math 114 – “Math and Politics”

This is a general education course and one that I taught in each of my previous two semesters at Trinity. The course content and assessments have evolved through the semesters, with the course covering the following four units (some associated topics):

  • Voting Theory (unweighted voting methods, fairness criteria, weighted voting)
  • Apportionment (methods, history, paradoxes, voting power)
  • Redistricting (overview, efficiency gap, compactness)
  • Game Theory (Nash equilibrium, simultaneous and sequential games, mixed strategies)

Before implementing the mastery-based system, assessment in this course consisted of unit exams, regular homework assignments, written responses to articles relating the topics to politics, and a group project.

Keeping in mind my goals from the last post of procedures vs. concepts, simple and transparent, and including a retake system, here’s the grading system I used in Fall 2019:

Learning Objectives

I translated content goals from having taught the course before into explicit learning objectives in which students must display mastery. Each unit had their own set of objectives with half of them categorized as “Procedural” and the other half categorized as “Conceptual” objectives based on the associated level of thinking. There was a total of 76 objectives split almost equally across the four units:

  • Voting Theory – 10 Procedural, 10 Conceptual
  • Apportionment – 10 Procedural, 10 Conceptual
  • Redistricting – 10 Procedural, 10 Conceptual
  • Game Theory – 8 Procedural, 8 Conceptual

Assessments

These objectives were assessed through in-class quizzes and exams. For the first two units, two quizzes were given only covering procedural objectives and the exam covered all of the conceptual objectives. Part of the motivation for this was having 50 minute class periods and not wanting students to use any of that time going through any potentially tedious (procedural) calculations. For the last two units, the quizzes and exams covered a mix of objectives, but this was a result of the way the content was organized as opposed to an intentional shift in structure.

The regular homework assignments were maintained but could not count towards demonstrating mastery. They were graded simply on based on completion and were used as an opportunity for students to gain practice and feedback on their progress towards the objectives rather than as a higher-stakes portion of their grade based on correctness. See below for how these assignments contributed to their final letter grade.

The written responses were maintained but streamlined for students to only answer questions that I viewed as the most crucial to get out of the article and to briefly reflect on the political context. The main point of each article (one per unit) was included as one of the procedural objectives and successful completion of the assignment was enough for students to demonstrate mastery of that objective.

The group project was also maintained and streamlined compared to the previous semesters. The project was for students to create their own new congressional district map for a state using an online web application and to write a brief report on the history of redistricting in that state and to show the efficiency gap and compactness calculations of their proposal which they had learned in that unit. The main goals of the project were translated into 3 of the 10 conceptual objectives for that unit.

Spreadsheet and Checkboxes

To ensure that students had easy access to their objectives at all times, I created a Google spreadsheet with all objectives separated by unit and separated by procedural/conceptual. Each objective had a checkbox next to it that I would mark when students achieved that objective and I set up conditional formatting to highlight the objective green when it was mastered.

Logistical note: I created one master spreadsheet file with a tab for each student and then a separate file for each student that was shared with them and automatically synced to their tab on the master file. This is why I had to use Google Sheets because Excel Online doesn’t support syncing from another file. Below is a screenshot of the spreadsheet (click to enlarge).

Tokens

To address retakes and support student’s control over their learning, I created a digital token system and gave each student 5 tokens to use for the semester. Each token could be used to retake up to 2 objectives or to submit a homework assignment later than the deadline. There was also an opportunity to earn an extra token for an excellent group project report. Students could retake objectives at any time during the semester and could retake any objective multiple times if needed. Using a token to retake objectives consisted of coming to office hours and me providing them with new questions related to the objectives for them to demonstrate mastery. As you may have noticed, tracking their tokens was also a part of each individual’s spreadsheet.

As you might imagine, many of these tokens were used at the end of the semester as students saved them to see how many objectives they would need to retake. More on this and other reflections coming in the next post!

Final Grades

To convert a student’s mastery of objectives into a final grade, I created and used the conversion table below which factored in the number of objectives checked off in the procedural and conceptual categories separately as well as the number of assignments completed and their attendance and participation in the class. You can see the table below taken from the course syllabus (click to enlarge) where your final grade would be the first row in which you met all four categories. For example, if you completed all assignments, attended and participated in class, mastered 35 procedural objectives, and mastered 29 conceptual objectives, your final grade would be a C+ as you would not have achieved every column in any of the rows above that grade.

Part 3 will be some of my reflections on how various components of this system actually worked and thinking about some changes for next semester!

An Apprentice in Mastery (Part 1)

In addition to creating a new course this past semester, I also took on another creative teaching endeavor by implementing mastery-based grading in one of my courses for the first time. I’m going to write a short series of posts talking about my approach and my experiences:

  1. Motivation guiding my design (this post)
  2. Implementation of my system
  3. Reflections and moving forward

If you’re reading this, I’m not assuming you have any background in what mastery-based grading is, but I don’t want to spend too much time explaining the various components because they will mostly be revealed through my reasoning and approach. However, it is important that I would describe the key tenets of such a system as (a) assessment and grades are based on the understanding (“mastery”) of learning objectives rather than a traditional numerical scale, and (b) there should be a way for students to have additional attempts to demonstrate mastery.

So why did I want to do something like this in the first place?

  • Giving grades more meaning. When you give a final grade in a course, how does it correspond to what the student actually has accomplished? For example, in a traditional grading system a B could translate to (i) getting everything correct the entire semester but missing a few assignments, (ii) getting B’s on the vast majority of assessments, or (iii) having a wide variety of grades that average to a B in the end. A mastery-based approach treats a course like a license or certification in that you have to demonstrate certain skills to earn the license. Grades then translate to how many of the skills you have demonstrated mastery in and no matter the grade, you will have a tangible list of objectives that you have learned as a result of taking the class.
  • Student mindset. It is always important to attend to students’ mental dispositions and a mastery-based system (in theory) has the ability to make positive impacts on the mindset and anxieties of our students. It is one thing to preach the value of a growth mindset and it is another to align your practices with this value. Allowing students additional opportunities to demonstrate their understanding of learning objectives embodies the growth mindset philosophy of encouraging productive struggle and that learning (especially at the level of mastery) can take different amounts of time for different types of learners. In addition, such a system would hopefully reduce test anxiety where students would know that it is okay to not understand everything at the time of an exam and that they won’t be penalized or judged as inferior for attempting some of the objectives at a later time (and hopefully with additional motivation to achieve mastery).
  • Discourage academic dishonesty. When reading perspectives of others who have utilized mastery-based grading, you probably won’t see this as a reason to employ the system, but I try to be as transparent as possible about my own thought process. In math, homework traditionally counts for 15-30% of your final grade and in my opinion, this only encourages the academic dishonesty that is rampant today (this could be a whole blog post on its own). From a student’s perspective, imagine sitting down the night before a homework assignment is due and not knowing how to do half of it. If the homework is graded for correctness, you would ask a friend for the answers or Google them too, wouldn’t you? For the record, I’m not faulting the student at all for doing this, especially given their stress associated with courses, extra-curriculars, and social life. So academic dishonesty is probably the wrong term, but why use a system that encourages a strategy that is not conducive to making progress towards actually understanding course content? I think it is fair to say that we would like final grades to be (as much as possible) associated with a student’s level of understanding and not the understanding of their friends or people online. We should be encouraging the use of these resources as a means of developing understanding rather than having students feel anxious because they are “cheating” on their homework.
  • Instructor time? In reading perspectives of others, “saving the instructor time” is a commonly cited reason for implementing a mastery-based system, but in my opinion, this isn’t a reason to make a pedagogical change. Yes, as professors we have a lot on our plates and would like to be more efficient, but I would never prioritize this over doing something because I believe it will benefit students in one way or another. In reality, any changes we make will take a lot of time initially and take less time as we gain experience and fine-tune our approaches. I will be expanding on this in one of my next posts, but for now I will say that my implementation this semester added time on the front end in determining objectives, took less time to grade during the semester, and added time in administering retakes.

With these motivations in mind, here were the initial principles I had in mind for designing the structure of my mastery-based grading system:

  • Separating procedural and conceptual thinking. I believe that being explicit about the cognitive demand level we expect from our students is important and I wanted to honor the way in which the mathematics education community differentiates procedural (following steps) and conceptual (displaying reasoning / non-routine problems) thinking.
  • Simple and transparent. The last thing students want is a complicated grading system, especially when it is different from what they are used to, so I wanted to make sure that it was easy to understand how their understanding of objectives would translate to a final grade and that they could see (online) at all times which objectives they have (and have not) mastered.
  • Retake system. This is a key feature of mastery-based grading, so I knew I wanted to incorporate a retake system with a personal goal of trying to find the “sweet spot” of not having too few opportunities for retakes (wouldn’t alleviate the test anxiety issue) and not having too many opportunities for retakes (could take away initial motivation to learn).

I also wanted to figure out a way to incorporate my existing assignments into the system, either as a category or as their own objectives. This is a good time to mention that I did this in a course I had taught the previous two semesters and I think it is much easier to go through the challenging process of identifying learning objectives if you have taught the course before so that you are closer to the content and student outcomes.

In my next post, I will talk about the course, the specifics of the mastery-based system that I created and used, and how the system played out during the semester. Thanks for reading!

The Journey of a New Course

As yet another semester comes to a close, this one was even more special than usual because my vision of creating a new class finally came to life. Considering I assigned my students a final reflection assignment, I thought I’d share my thoughts and experiences as well.

First some background if you haven’t been following along: the goal of creating an entire class on redistricting that authentically uses math in the context of a relevant (and often contentious) political issue was almost two years in the making. In my last semester at UConn (Spring 2018), Adam Giambrone (now teaching at Elmira) and I tried the “let’s throw some things [content] at the wall [students] and see what sticks” approach to teaching this topic which for my money is an underrated approach to education – you don’t always need to have a plan! (More on that later.)

Anyway, upon being hired at Trinity, I was connected to Jack Dougherty who made me quickly realize that my goal of creating such a class was far from a dream, in fact it was exactly the sort of thing the institution wanted and that it had the potential to be brought to life even more through community / government partners. I was also fortunate enough to have the support of my department (Mathematics) who easily could have asked for something more concrete for a new course offering.

Through the help of Jack and Megan Hartline (who both do an amazing job organizing and supporting Trinity’s Community Learning Initiative) I was put in touch with various local professionals who had some connection to redistricting or politics and voting in general. This led me to a government employee (who would prefer to remain anonymous) who is simply incredible and whose knowledge and passion further inspired my own. Our first meeting is when my vision started to feel real for the first time.

Ok so I had a government partner supporting my vision and the course was quickly approved by the school – with the Political Science and Public Policy & Law departments even counting it towards their major! – and my passion for the topic and for education were always there, but creating a class from scratch isn’t easy. To be honest, up until the beginning of this semester I had nothing prepared beyond a list of potential topics, an idea for the final project (having them redistrict Connecticut as authentically as possible), and some materials I developed and used for a 2-3 week unit on the topic in my general education “Math and Politics” class.

Back to not having a plan…here’s the thing: It wasn’t that I didn’t know what to do and it certainly wasn’t laziness. For me part of education is being responsive to your audience and I wanted to gauge the mathematical and political science background of my students and use that to guide my instruction. This was especially important because this was my first time teaching a subject area outside of math or education and I had no prior experiences to rely on to somewhat estimate the level of the students.

As you might imagine, this “approach” led to a lot of late nights making lecture slides, handouts, readings, etc. I always joked with the students that twice a week I felt like one of them working on a class presentation assignment. Creating a course organically and in real-time is not something I would recommend (especially when you have two other preps in the same semester) but it did keep me very close to the content and to current events. On that note, thanks to the state of North Carolina for providing excellent content for this class!

What ended up happening is that effectively all of my course content was front-loaded into the first eight weeks (out of 14) of the semester. At that point I think both myself and the students (we affected each other) sort of “ran out of steam” when it came to content. There were definitely a few other related topics I could have covered, but I’m confident that I covered all of the crucial content and in a way that the students successfully grasped.

At the same time, I was communicating with various stakeholders to create and schedule the authentic learning experiences that I believed were absolutely essential for the course. Managing these logistics takes time and naturally forced these experiences into the second half of the semester, but I think this worked out because students went into the experiences with a strong foundational knowledge of redistricting policy and the associated mathematics.

What experiences am I referring to? In addition to two excellent guest speakers in the first half of the semester, there were three experiences I was able to put together and add (in my and the students’ opinion) incredible value to the course. First, I’m extremely thankful for the Caliper Corporation that generously gave my class a license to the software program Maptitude for Redistricting, which is the actual program used by a majority of the states. Thanks to Cheryl Cape and others in Information Technology, we were able to put together a “Redistricting Lab” in the library for students to grind away and act as independent redistricting commissions to propose a new State Senate map for Connecticut. There were certainly challenges and a learning curve for all of us in operating the software, but it was more than worth it for the authentic experience and the ability to empower students as some of the most qualified individuals for future redistricting in any state.

Second, I can’t thank State Representative (and Chief of Staff to Trinity’s President) Jason Rojas enough for leading a behind-the-scenes tour of the State Capitol and Legislative Office Building. While the focus was not directly on redistricting, so many students reflected with incredible positive feedback about the experience and how it humanized the people working in the government. And it’s not every day you get a chance encounter with the Governor!

Third, I was hoping that students could get an even more personalized meeting with elected officials involved in the redistricting process both to add anecdotes that I or any reading wouldn’t be able to provide and to humanize the process as well. Fortunately with Jason’s help, I was able to have smaller groups of students meet with two former Democratic Speakers of the House (Chris Donovan and Brendan Sharkey) and a former and current Republican Minority Leader of the House (Themis Klarides and Larry Cafero). I am thankful for each of them giving some of their time to our class and I was personally involved in two of the four meetings and I know I learned so much more from them than in researching my lecture slides.

I want to clarify that this class isn’t about me. Anyone can have an idea or vision for a new course but without interested and engaged students it’s simply not possible. This is a good time to mention that registering for this new class was by permission only and that the initial interest of my 22 students (from an initial cap of 19) actually did carryover throughout the semester. I can only imagine our collective attitudes toward education if all of our classes were this way.

With that being said, I do want to share a couple of personal anecdotes. First, I don’t have any formal background in Political Science (I completed half of a minor as an undergraduate!) and I never pretended that I did. I joked that I was a “Math professor masquerading as a Political Science professor” and it was no secret that the advanced Political Science majors were more well-spoken about the field in general than I was. And that’s ok! There’s something to be said about learning together and using our unique knowledge as individuals to advance our collective knowledge. As for my own knowledge, I certainly noticed a shift as the semester progressed. Thanks to the students, the guest speakers, the legislators, and hours of research in preparing lecture slides and activities, I noticed myself transition from someone who is knowledgeable and interested in redistricting to someone who is an expert in redistricting (particularly in the context of Connecticut).

My final takeaway: In looking back, I’m proud of what I was able to provide my students both in terms of content and experiences in my first attempt at doing something like this. It’s really amazing to think back to an initial vision and realize everything that contributed to its reality. Thanks to my experiences and my students’ input, my head has been spinning at all of the tweaks I can make to improve this class when (not if) I teach this class again. I firmly believe that this type of class represents something we should do more often in education, but clearly it’s not easy. Most importantly, we can’t do this alone. It takes (a lot of) communication and collaboration with others at your institution and the right people in your community. It takes others to trust and support your vision. It takes the right students. It takes the Supreme Court making a key ruling two months before the semester (just kidding, but it does help). It takes a belief that math truly matters. And finally, it takes a commitment to the philosophy that the sharing of perspectives and experiences are at the core of education.

For more information about the course content, see my page about the course.