Land Acknowledgement, August 2021

My practice as a scholar is entirely shaped by relationships: with my students and colleagues, with my family, my neighbours, and with the land. This short introduction describes some of those relations.

I live with my wife and twin teenage sons in Hamilton, Ontario, on the traditional territory of Haudenosonee and Anishnaabe nations. I grew up in Ottawa, Ontario, also the traditional territory of Anishnaabe nations. My relationship to the land cannot be separated from my relationship to my ancestors. My several-greats-grandfather on my mother’s side was Jakob Treitz, who travelled from what is now Germany and arrived on Wolastoqiyik and Mi’kmaq land in 1766. The house that Jakob Treitz built still stands today in Moncton, New Brunswick. It is a museum that bears his name. My several-greats-grandfather on my father’s side was Alexander MacDonald, who travelled from Scotland and arrived on the land governed by the Wabanaki Confederacy in 1783. The house that Alexander MacDonald built still stands today in Lower Newcastle, New Brunswick. It is a museum that bears his name.

To my knowledge, there are no museums bearing the names of those on whose land my grandfathers built their houses, yet my family and I continue to benefit from the actions of my settler ancestors. I therefore make the following commitments:

  • I will strive for a healthy, just relationship with the land, by eating locally and choosing human-powered transport and renewable energy.
  • I will strive for a healthy, just relationship with my local community, contributing to grassroots organizations and buying from small, local businesses.
  • I will teach my children and my students about the ongoing effects of colonization in Canada and the world. I will teach my Linguistics students about the ways that linguists can support Indigenous people in sustaining and reclaiming their languages.
  • I will continue to learn from Indigenous scholars, artists, activists and community members.

I still have a great deal to learn to undo my colonial ways of thinking and being. I am grateful for my relations with colleagues and students, from whom and with whom I learn every day.

Listening to Students’ Voices

This is an article I wrote for my Faculty Association‘s newsletter. Some of it is specific to my institution, but the overall principles hold for many undergraduate instructors.

All semester long, our students listen to our voices (or watch our hands, in the case of ASL courses). And at the end of the semester, it has long been our tradition to ask students to voice their experience in the course, in the form of a course evaluation questionnaire. More than a decade ago, one of my students took the opportunity to make it clear that listening to me had not been a positive experience for them, by commenting, “She has an annoying voice.” I’ve often wondered what that student’s goal was in writing that: Did they think I should have acquired a different voice to make their experience more pleasant? Would they have learned more or better if my voice were different? Did they want my Chair to discipline me for my voice? The only plausible goal I can think of is that they wanted to hurt me. If so, they achieved some modest success – after all, I still remember their barb all these years later. But I know that my white skin and monolingual anglophone Ontario accent shield me from the hate that many colleagues receive in this medium, anonymized and typed up in a formal report issued by the employer. The bigger question, more important than what that student hoped to achieve, is what we hope to achieve: what is our goal in surveying students at the end of each course? 

What do we want to know from student surveys?

Ostensibly, the goal of these surveys is to measure Effective Teaching, or, in the case of Teaching Professors, Excellent Teaching. According to the Tenure & Promotion Policy, “A candidate for re-appointment, tenure and/or promotion must demonstrate that he or she is an effective teacher,” and, “A candidate for permanence must demonstrate that he or she is an excellent teacher”. And SPS B1 Procedures for the Assessment of Teaching opens by stipulating that “Effective teaching is a condition for […]  salary increments based on merit.” Our existing policies require the numerical data from student surveys to be considered in determining whether an instructor’s teaching is effective, which is necessary for making decisions about tenure, permanence, promotion, and merit. 

Who could argue with the goal of ensuring effective teaching? McMaster’s world-class reputation for creativity, innovation and excellence rests in no small part on the quality of our teaching, and it is clear from the immense efforts we have invested in redesigning courses for pandemic virtual learning conditions that McMaster faculty are deeply invested in providing high-quality learning experiences to our students. 

What can we know from student surveys?

The problem is, the numerical scores from student surveys have very little relationship to teaching effectiveness. A large and growing literature has repeatedly found that these surveys are strongly biased by factors such as the instructor’s race, accent, and perceived gender and by students’ grade expectations. So convincing is the evidence that a 2018 decision by arbitrator William Kaplan established the precedent that “averages [of survey responses] establish nothing relevant or useful about teaching effectiveness”, a ruling that required Ryerson University to stop using survey results for promotion or tenure decisions. And in the time since that ruling, pandemic teaching and learning conditions have introduced even more factors that affect students’ learning experiences, such as the quality of their instructor’s home internet connection or their dislike of remote proctoring.

According to our policies, we value effective and excellent teaching. According to the university’s vision statement, we value “excellence, inclusion and community”. But over the years we’ve built a system that calculates a faculty member’s annual merit and makes high-stakes decisions about tenure, permanence and promotion using a number that not only does not measure effectiveness or excellence, but actively works against promoting inclusion and community by reinforcing existing hierarchies of race and gender. The system also does real harm to equity-seeking members of our community by subjecting them to anonymous hateful comments that are irrelevant to their teaching.

If we claim to offer excellent teaching, we have a responsibility to listen to what students say about their learning. If we claim to offer excellent teaching, we have a responsibility to avoid relying on invalid, biased data as evidence.

What has MUFA done about it? 

At the time of the Kaplan ruling, MUFA began collaborating with scholars at the MacPherson Institute to develop innovative and equitable ways to observe teaching effectiveness. A 2019 report made initial recommendations, and an ongoing committee is developing an evidence base that will inform a redesigned system. 

In the absence of an unbiased, valid tool for observing effective teaching, the members of the MUFA executive thought it imperative to attempt to mitigate the biasing effects of the current surveys. To that end, we negotiated a revision to the so-called “summative question” on the surveys. Instead of asking students their opinion of the instructor’s effectiveness, that question now asks, “Overall for this course, how would you describe your learning experience?”. Furthermore, under pandemic circumstances, we negotiated an agreement that these scores should not be used at all in the assessment of merit for 2020. We hope these temporary changes go some way to reducing the biases of the student survey process, but they were only ever meant to be short-term measures.

What still needs to be done?

Excellent teaching is too complex to be tracked by a single number that gets compared across instructors. As members at a recent MUFA Council meeting pointed out, simply replacing student survey numbers with peer scores or with student focus groups is unlikely to eliminate the bias. If our goal is excellence, or even something more achievable like effectiveness, we need complex, high-quality ways of assessing progress towards that goal. It will take time to achieve this, and we’re working on it. 

Once we’ve developed a new process, we’ll need to update all our policies to make sure they’re consistent with each other. We should also build in a regular schedule for reviewing the process, to make sure we don’t inadvertently regress to overly simple, biased metrics over time. 

One MUFA Council member asked whether we had taken any steps to redress historical consequences of using biased data. The MUFA executive have not discussed this specifically with respect to student survey data, but we are continuing to investigate equity issues in our members’ compensation. The 2015 salary adjustment for women faculty was one outcome of this ongoing process. 

While all that work is going on, we can and should strive for continuous improvement of our teaching. And here is where listening to our students is vital. Students are the only ones who can provide first-hand data on their experiences of our courses. We could design the most rigorous, comprehensive courses in the world that still might not support our students’ learning. Within a semester, listening to our students might prompt us to move due dates, reweight assessments, or return to a topic that we thought we had finished. By listening to our students at the end of a semester, we can make improvements to the course for the next cohort of students, like converting a timed test to a take-home assignment, or reducing the number of low-stakes quizzes. It was listening to my students saying, “I don’t have the textbook. Is it required? Is it on reserve? It’s awfully expensive,” that led me to create an OER that is freely available to all learners everywhere. 

If we claim to offer excellent teaching, we have a responsibility to listen to what students say about their learning. If we claim to offer excellent teaching, we have a responsibility to avoid relying on invalid, biased data as evidence. MUFA is working on ways to do both of these. We welcome your contributions to this work. To get involved with the work of the MUFA Executive, please contact . 

Making undergraduate research opportunities accessible

Our department has several opportunities for undergrads to conduct original research. In the Research Practicum (Ling 3RP3), students work as lab members on a faculty member’s research project. In the Honours Thesis (4Y06), students conduct an original research project of their own, guided by a faculty supervisor. Students might also conduct a smaller-scale research project under the Independent Study (4II3) course code, though this course code also covers individual reading courses and the like. I created the Honours Thesis course and guidelines in 2009, but in recent years we learned that many students (and indeed, some faculty members!) were not aware of the process for requesting access to these courses. In 2017, I re-wrote the instructions for applying for these courses, and revised the suggested course outlines. The updated guidelines are now available on the department website.

Blended Learning for a Very Large Class

Introduction to Linguistics is a very large class — usually about 600 students in Ling 1A03 and 300-400 in Ling 1AA3. The size of the class can be overwhelming to first-year students, and the format of the class, which blends on-line components with in-person class sessions, might also be unfamiliar to them. Having a clear structure in Avenue is important to helping students stay on track in the course. In the very large classroom it’s also important to create a sense of community, to alleviate some of the sensation that students might experience of being just a face in the crowd. In the first couple of classes, I remind students that learning can work well in community and I give them a few minutes to get to know each other. In each class period there are always opportunities for students to talk to each other while analyzing data or arguing over the answer to a clicker question. I often remind them that if they don’t know the person they’re sitting next to, it’s worth taking a minute to introduce themselves before jumping into the discussion question. And I often open class with a couple of sentences about myself or my family, so that they remember that I’m a human with a life outside the classroom just as they are.

The screencast embedded below walks you through some of the on-line components of the course from the student’s point of view. The first semester that we used the blended format was Fall 2014. Each year I’ve surveyed students anonymously about their experience in the course and made changes based on their feedback. The first year, their most frequent ask was that we correct the errors in the quizzes, an entirely fair request since that first version did contain more errors than tolerable (partly because we were still learning how to use the Avenue quizzing software, and partly because we were developing materials on a short timeline). So the primary job for my TAs that next summer was to go through the quizzes very carefully and correct any errors. We also added a lot more quiz questions to each topic that summer, so there was a larger bank to randomly draw questions from.

The following year, the thing students wanted most was more practice assignments. As I gradually detached the course from its dependence on the commercial textbook, I created my own new assignments every year. I do this so I can provide them with full answer keys and commentary once the assignment is over, while not having to worry about future students getting access to the answer key. This means that I can post the previous year’s assignments and tests plus the answer keys for them, so that students get a chance to practice and check their own answers.

Here are some sample assignments and practice tests:

Sample Assignment for Ling 1A03

Sample Midterm Test + Answer Key for Ling 1A03

Sample Assignment for Ling 1AA3

Sample Midterm Test + Answer Key for Ling 1AA3

The biggest change for 2018 will be the new Open Access eBook, Essentials of Linguistics. Before September, my TAs are working to update the quiz questions and all the accompanying materials to be consistent with the notation conventions in the new book. In the Spring 2018 course, students have already expressed how pleased they are not to have to shell out $147 for a commercial textbook!