Professor Reviews and Learning Python

Last fall, I decided to learn Python, with a desire to analyze text and implement some machine learning. So, I decided to start by learning BeautifulSoup and using the tools there to scrape a professor rating site. The project went well, and I was able to write some code (that you can find on my GitHub). I got the hang of scraping and wrote code to collect numeric and text information from reviews of professors by school or by state.

Next, I began to analyze those reviews. I started the project intending to look gender differences in ratings, following other reports of differences, such as Sidanius & Crane, 1989, and and Anderson & Miller, 1997. So, I had to have the gender of the professors, something that was not available in the dataset that I had scraped. I decided to use pronouns to assess gender, and in case there were no pronouns in the text or the pronoun use was unclear, I assigned gender based on name.

I compiled reviews for schools in Rhode Island, New Hampshire, and Maine, discarded reviews with no gender assigned, and began by looking at differences in numeric rankings, which include an overall score and a difficulty score. Male and female professors have nearly identical mean scores — women’s mean overall is 3.71 and men’s is 3.75 and women’s difficulty is 2.91 and men’s is 2.90. The overall and difficulty means for each professor are correlated, as you can see here:

Plot of histograms, scatter with linear, and residuals

Interestingly, women seem to have fewer reviews than men. On average, female professors have about 9 reviews and male professors have about 12. This difference seems to be stable when looking at each individual year, and could be due to male professors teaching larger classes (but I have no data on that). The end result is that there are far fewer reviews of female professors. In my dataset, there are 107,930 reviews of male professors and just 64,799 reviews of female professors.

The data set also has a self-report of grades from most reviewers. You can see in the data overall scores go down when  reviewers get a bad grade, but women seem to be hit harder by this than men.

Bar graph showing overall mean score by grade and gender.

Note also that far more reviewers report receiving high grades. In fact, over 160,000 of the 173,000 reviews in my dataset report getting A’s.

The overall scores show a bimodal distribution, as you can see in the histogram of overall scores (reviewers can report scores from 0 to 5 with half-points possible). The next thing I decided to was to categorize these reviews into positive or negative, getting rid of reviews in the middle, and then to do some analysis of the text in those reviews. I’ll report on that next.

Histogram of overall scores, showing bimodal distributions for both men and women.

 

Disappointment and Hiding in the Classroom

I’ve been noticing lately my disappointment in students. I don’t want to feel disappointed in students. Honestly, I don’t want to feel disappointed in anyone. Who does? But you might argue that we have certain expectations for how the people around us will act, and that people don’t always meet those expectations. When they don’t, I am justified in feeling disappointed, at least provided that my expectations were reasonable. The trouble is that  disappointment is counterproductive, and for me it is part of an overall tendency I have to disconnect with people.

Let me look at this a little closer. I have certain expectations for my students. I set those out for the students by giving them specific assignments (“turn this worksheet in on Monday” or “write a blog post about your problem-solving process”), and I lay them out on the course syllabus by telling students to come to class, check their email regularly, participate, and so forth. There are also a collection of expectations that go unspoken by me. I expect that students will be thinking about what they need to do to prepare for upcoming exams, even if I don’t give them explicit assignments. I expect that students will ask for help and support when they don’t understand something after class. I expect that students will monitor what they do and don’t understand. I expect that students will give me their best work, and won’t piece together something at the last minute. I often say things which imply these expectations, but I’m not always explicit about them. Also notice that not all of these expectations are realistic.

If a student doesn’t meet these expectations, I get cranky. In between classes, if I am expecting work and participation from students that I don’t see, I start to worry, and to run my “disappointment tape.” Typically it involves me getting frustrated and making up a lot of things that I imagine to be happening with the students. I imagine them as uninterested in the course, not dedicated, not hard-working, wanting to get away with not doing work, not caring about thinking deeply, not caring about interacting with me or other students. Yes, there’s some really ugly stuff hiding in there. The thing is that I don’t know that any of that is really happening. Mostly, I think what is happening with me is that I want this connection with students, and most of what I have to connect with is their work. When the work isn’t there, I feel rejected. I imagine the students pulling away from me, and I rush to pull away from them first, by getting “disappointed” in them. Most of the time, I can get back my connection with the students simply by being around them — it is the time in between classes that provides a space for these feelings to grow.

Students don’t always do what we teachers what them to do. In fact, people in general don’t always do what other people what them to do. So we get anxious about our relationships and our standing with other people. In school, this means teachers get frustrated with and disappointed in students. What do students do? Students learn to hide from the disappointment of teachers. They hide and they lie so they can save themselves from the consequences of expectations unmet. Students hide so that they’re grades aren’t in jeopardy and they hide so that they can maintain positive relationships with the powerful people that are important to them. Students get into a habit of hiding, so that it seems as natural as breathing. I remember it well from the last time I was a student — doing work I wasn’t proud of and hoping it would slip by without notice, making up excuses for doing work late or stretching excuses that were technically true but not really accurate, trying to look good in order to get away with things. As a teacher, I know that students are doing these things, but I ignore it, acting as if students are going to meet all of my expectations, and then getting disappointed when they don’t. Because I am required to assign grades to students, I maintain and perpetuate the fiction that grades mean something objective, when the reality is that they’re just a somewhat arbitrary record of how well a student met my somewhat arbitrary standards about a somewhat arbitrary collection of activities and topics.

What if I stopped doing this? It’s hard to imagine. Could I stop having expectations of students? What would happen to me and to the students if I did? What if I kept having my expectations, but was more honest about the fact that I know students won’t always meet them? What’s so bad about the students not meeting them anyways? Could I keep the expectations, but let go of the disappointment, simply connecting with students about what happened and deciding what to do next? Could I let my students be honest with me about the unrealistic nature of my expectations and with what really happens for them in a class? Could I let students formulate their own expectations, help them to make those expectations realistic, and then help them to live up to those expectations? Could I create a classroom environment in which I helped my students evaluate themselves? Wouldn’t this cause the very foundation of objective and rational subjects like math and science crumble because students would start writing expressive poetry about how math makes them feel and giving themselves an A++ on every assignment?

 

Mid-term Evaluation

A lot has happened this summer while I have been completely absent from this blog. I’ll give some updates about everything as I ease back into the semester, and I’m starting today with a project I’ll be working on this semester, which grows out of a mid-term evaluation process that I have developed. My intention is do a small SoTL (scholarship of teaching and learning) project this fall on my midterm evaluation process.

Right now, I’m working on the literature review. Right away, I hit something both obvious and interesting. I started reading a paper about course evaluations and the opening words were about the purpose of course evaluations. The claim was that the purpose of course evaluations was, essentially, the improvement of teaching. They provide feedback to instructors that the instructors can use to improve their teaching, and they provide information to supervisors that can be used to insure that the right teachers are assigned to the right courses. Of course, that got me thinking about my purpose in doing midterm course evaluations. I think that improvement in teaching is certainly part of it, but I wouldn’t call it my primary goal. Here’s what I’ve come up with for my goals in doing midterm evaluations:

  • To develop insight into how students are perceiving and relating to the course and me as the instructor
  • To provide me with time to change aspects of the course that need improvement before final evaluation at the end of the semester — to improve the course and my teaching in “real time”
  • To give students a mechanism for taking charge of their education, and to have them practice using that power through impacting the course
  • To enhance the level of safety and improve relationships in the course

I also got a useful new perspective. Why would we do this improvement in teaching? Presumably to improve student learning and outcomes, but student evaluation instruments don’t tend to focus on learning or on student outcomes! This semester, in the class I am going to be doing this evaluation project in, I am also going to have the students do two reports on their achievement of the course objectives. Essentially, these are a self-evaluations, and I would love to find a way to connect these self-evaluations with the midterm evaluation of instruction.