September 20, 2018
Dear Faculty Colleagues,
Many of you are aware that changes are being introduced regarding teaching evaluations at USC. This process was initiated by the Academic Senate, and Faculty input has been sought in a number of ways over the last few years. In addition to working towards teaching excellence, there has been significant concern about the bias in the student evaluation process, especially against women and people of color. Moving towards reducing the emphasis on the use of student evaluation of teaching and towards peer review has therefore been encouraged by Senate efforts. More recent steps by the Center for Excellence in Teaching (CET) towards implementation, however, have been met with some questions. We have therefore asked Ginger Clark (Director of CET and Assistant Vice Provost for Academic and Faculty Affairs) to provide a Q&A on the topic (it describes the impetus for the changes as well as efforts to include faculty), and I have included that at the end of this email.
I am excited to engage our faculty on this important issue and believe that having productive conversations on the merits of all aspects of an issue is the right way forward as a University. As such, we will be discussing this issue at the next Academic Senate meeting (Wednesday, September 26, 2018, 2:00 – 4:00 pm at UVF 1100 in the University Village), and we have invited Provost Michael Quick and Assistant Vice Provost Ginger Clark to the discussion as well. The Senate meeting is open to anyone, and I encourage those who want to discuss this issue to come. Whether or not you can make it, I would also like to invite you to send comments you may have on the issue to your Faculty Council Chairs and Senators. Your school faculty leaders will attempt to collate the comments (both comments supporting the change or those suggesting alternative ideas for improvement) and relay these at the Senate meeting. Below is a list of each school’s representatives (with email addresses).
Sincerely,
Yaniv Bar-Cohen
Academic Senate President
Faculty Council Chairs and Senators Roster
2018 – 2019
Annenberg School for Communication
Faculty Council Chair: Laura Castañeda, lcastane@usc.edu
Senator: Francois Bar, fbar@usc.edu
School of Architecture
Faculty Council Chair and Senator: James Steele, jamesmcleodsteele@gmail.com
Marshall School of Business
Faculty Council Chair and Senator: Elissa Grossman, elissa.grossman@marshall.usc.edu
Senator: Sharoni Little, slittle@marshall.usc.edu
School of Cinematic Arts
Faculty Council Chair: Jeff Watson, jeff.watson@cinema.usc.edu
Senator: To be elected in coming days
Glorya Kaufman School of Dance
Faculty Council Chair and Senator: Margo Apostolos, margo@usc.edu
Herman Ostrow School of Dentistry
Faculty Council Chair and Senator: Mehdi Mohammadi, mmmohamm@usc.edu
Dornsife College of Letters, Arts and Sciences
Faculty Council Chair and Senator: Daniel Pecchenino, dpecchen@usc.edu
Senator: Jessica Cantiello, jessica.cantiello@usc.edu
Senator: David Crombeque, crombecq@usc.edu
Senator: Devin Griffiths, devin.griffiths@usc.edu
Senator: P.T. McNiff, mcniff@usc.edu
Senator: Jessica Parr, parr@usc.edu
School of Dramatic Arts
Faculty Council Co-Chair and Senator: Sibyl Wickersheimer, sawicker@usc.edu
Faculty Council Co-Chair: Meiling Cheng, meilingc@usc.edu
Rossier School of Education
Faculty Council Chair and Senator: Morgan S. Polikoff, polikoff@usc.edu
Viterbi School of Engineering
Senator (alternate): Victor S. Adamchik, adamchik@usc.edu
Faculty Council Chair and Senator: Todd Andrew Brun, tbrun@usc.edu
Faculty Council Vice-Chair and Senator: Elizabeth Fife, fife@usc.edu
Roski School of Fine Arts
Faculty Council Chair: Andrew Campbell, campbela@usc.edu
Senator: Ewa Wojciak, wojciak@usc.edu
Davis School of Gerontology
Faculty Council Chair: Susan Enguidanos, enguidan@usc.edu
Senator: Jennifer Ailshire, ailshire@usc.edu
Independent Health Professions
Faculty Council Chair and Senator: Julie McLaughlin Gray, jmgray@chan.usc.edu
Gould School of Law
Faculty Council Chair and Senator: Scott Altman, saltman@law.usc.edu
Keck School of Medicine
Faculty Council Chair and Senator: Shahab Asgharzadeh, SAsgharzadeh@chla.usc.edu
Senator: Mark Frey, Mfrey@chla.usc.edu
Senator: Launda Grazette, laundra.grazette@med.usc.edu
Senator: Rima Jubran, rjubran@chla.usc.edu
Senator: Alison G. Wilcox, awilcox@med.usc.edu
Senator: Gabriel Zada, Gabriel.Zada@med.usc.edu
Thornton School of Music
Faculty Council Chair and Senator: Adam Gilbert, akgilber@usc.edu
School of Pharmacy
Faculty Council Chair: Eunjoo Pacifici, epacific@usc.edu
Senator: Andrew Mackay, jamackay@usc.edu
Sol Price School of Public Policy
Faculty Council Chair and Senator: Juliet Musso, musso@price.usc.edu
Suzanne Dworak-Peck School of Social Work
Faculty Council Chair and Senator: Carl Castro, cacastro@usc.edu
USC Libraries
Faculty Council Chair and Senator: Andrew Justice, ajustice@usc.edu
Q&A from Provost’s Office Regarding Teaching Evaluations
Why are our recent teaching excellence efforts at USC?
Even at a research university where innovative research and scholarship is key, most would agree that teaching is one of our most important endeavors as faculty—and one that will have broad impact through the students we mentor and train. So, are we doing all we should to develop teaching excellence, evaluate it, and reward teaching appropriately? Over the past five years, there have been discussions and reports, led by the Academic Senate and by faculty task forces, on how to advance teaching excellence at USC. We have been working from the recommendations of those faculty governance groups to put into place resources for faculty and schools to advance teaching excellence.
Why are we changing the teaching evaluation process?
The recommendations from the Academic Senate and faculty committees indicated the need for a more meaningful review of teaching than student evaluations provide. Specific recommendations called for a revision of the student evaluation instrument, and making teaching evaluation more robust by adding peer-review components.
Why are student evaluations of teaching not enough?
While student evaluations can be informative, research on student evaluations show that results are not correlated with learning outcomes, or other valid measures of teaching effectiveness. They are also prone to systematic bias against women, and there are early indications the same holds true for faculty of color. Anecdotally, many faculty have a sense that student evaluations are influenced by how easy or hard the class is or how engaging a faculty member is, but they don’t reward faculty who use effective, but perhaps less entertaining, teaching practices.
How will teaching be evaluated?
Faculty and schools will need to figure this out for their particular discipline. At a high level, it would seem to make sense that teaching should be evaluated across several categories. Categories might include: Teaching effectiveness, evidence of teaching development, peer-review of colleagues’ teaching (including peer observation), inclusive practices, student engagement, etc.
It seems appropriate that “Teaching effectiveness” should make up the majority proportion of an evaluation, and best practices suggest it should be comprised of multiple measures. We are adopting the recommendation that those measures be primarily peer-reviewed. Merit review may be comprised of one or two of these measures, whereas tenure, promotion, and continuing appointment might require more categories be assessed.
As is the gold standard in research and clinical care, the recommendations that came from the Academic Senate and the faculty-led task forces recommended we include peer review. Faculty will need to decide what makes most sense for their school, but peer-reviewed measures might include:
- Teaching reflection statements
- Syllabus or course materials review
- Review of assessments and grading
- Classroom observation
Faculty and schools will need to select the other categories and measures that will make up the remaining proportions of teaching evaluation.
Will student evaluations be used to evaluate faculty at all?
Student evaluations provide important information and should be used by all faculty to improve their teaching. But, as described above, we should be very careful about being overly reliant on them. If faculty and schools decide that one component of their teaching evaluation is “student engagement” then student evaluations would fit appropriately in that category. (Schools should avoid using student evaluations as their only data point for student engagement, given the strong evidence that student evaluations are prone to bias.)
Another option for using student evaluations effectively might be the following: Through a peer-reviewed teaching reflection statement, faculty could be asked to examine their teaching practice over the past year, and talk about successes and failures in teaching, data they used to assess their work, and plans for improvement. Faculty might be asked to use their student evaluations and other sources of information to inform how they plan to adjust their teaching practices to enhance student engagement.
For schools, aggregated student evaluations can also tell us much about student engagement at the school or program level. They can be used to better understand students’ experience of inclusive practices, as one data point to assess schools’ diversity and inclusion efforts. They can be used to identify faculty who are very good at engaging students, as well as those who may need support in this area. And of course, student evaluations can alert us to problematic behaviors that require further evaluation or investigation.
How have faculty been involved in the process thus far?
The Academic Senate convened a Faculty Evaluation Task Force in 2013. The resulting white paper was reviewed by the full Academic Senate in December, 2013. The final paper was sent to the Provost’s office in February, 2014. Teaching is covered on pages 3 and 4.
The Provost convened a 2016 Task Force on Teaching Effectiveness populated by faculty from across schools and disciplines to consider ways to improve and assess teaching effectiveness. A report was submitted that included recommendations for promoting and assessing teaching effectiveness.
The Senate/Provost Committee on Teaching and Academic Programs built on those recommendations, and developed a set of recommendations for teaching excellence at USC. This report was presented to the Academic Senate in May 2017, and submitted to the Provost. From those reports, an advisory board of faculty experts from across disciplines was convened to provide guidance and feedback on the university elements that would support teaching and changing the student evaluations. It also helped CET develop the resources needed to help schools and faculty move forward in advancing teaching excellence.
The Academic Senate was consulted during the 2017-2018 academic year on various aspects of the model, input was sought from faculty councils, and offers made to council chairs to hold information/feedback sessions in every school; 10 have been held to date. Some chairs preferred to wait to hold information/feedback sessions until or if their faculty felt it was needed, while others preferred to get the information from their vice deans.
Has everything already been decided or is there room for changes?
Actually, very little has been decided. The university has provided schools with general guidelines and extensive teaching development and evaluation services and resources through CET. But the process has now been turned over to the schools and their faculty. Faculty will be asked to work with their school’s leadership to define teaching excellence in their school; determine what categories should be measured, using what tools and processes; and shape how investment in teaching development and evaluation, as well as excellence in teaching, should be rewarded.
The Provost’s office has been working with school leaders to help them devise a faculty-led teaching plan development process. We provided a plan template to help faculty envision what particular parts of the plan might include. The examples given in the template are intended only to guide discussion or provide options. CET’s resources are noted often in the template because they exist already, but schools do not have to use them. University guidance is indicated with asterisks. The first part of the plan template requires schools to document how faculty governance, feedback, and agreement on the plan occurred. Faculty leadership in the process is required for every section of the plan template.
What is the timeline of the changes?
Getting this right is more important than hitting any particular deadline. That said, given that student evaluations are likely to be disadvantaging a large segment of our faculty, it is important that we work diligently on this.
We have envisioned a year-long process for developing the plans. We have asked school leaders to work with a faculty taskforce to sketch out a framework their faculty can support, and submit that by end of fall, while they continue to work on the plans through spring. We expect many schools will opt for a phased implementation. For example, with regard to peer-review, schools may choose to use a teaching reflection statement for merit this year, and next year will institute another tool, such as syllabus review. Perhaps the next year review of assessments will be launched, and the following year class observation. This will give schools time to fully flesh out criteria, processes, and plans for implementation.
How do we know the new evaluation process will be better than the last? Will there be training?
The rationale for peer-review has many layers:
- It promotes teaching development. Rather than having evaluation based on two scores from vague items on student evaluation, faculty will receive feedback on their strengths and where they can improve their instruction, their course design, their assessments, their inclusive practices, and other elements of teaching. This makes the evaluation process both meaningful and informative, providing both formative and summative evaluation.
- The act of reviewing one’s peers can also promote teaching development, allowing faculty to learn from each other’s work, much like in peer-review of scholarship.
- Learning to use peer-review tools is informative to one’s own teaching, as it allows faculty to learn about best practices and to do critical self-reflection on their teaching.
- And of course, peer-review is the standard used in academia. We should be evaluating one another’s work, providing both support and challenge.
There is early empirical support for peer-review evaluation of teaching, but there is room for us to conduct more in-depth examination of its multifaceted impact. The literature is in agreement that training for reviewers is critical for success, which is why CET is providing training for the tools it developed, should schools wish to use them. Schools are asked to ensure training occurs for any peer-review tools they implement.
How do we know the new process will not be biased?
Bias, while sometimes explicit, is more often implicit. We will never eliminate bias completely, but we are able to minimize it much more with evaluations that use multiple measures and which are led by faculty. To reduce bias in peer-review we are doing the following:
- The university has implemented bias training. The School Diversity and Inclusion Plans must include bias training for any faculty member who influences the hiring, merit, promotion, tenure, or continuing appointment decisions of another faculty member; this effectively means most or all faculty.
- All CET peer-review tools measure either the presence or absence of a best teaching practice. The evaluations are not comprised solely of subjective narrative, but of observable, measurable criteria.
- CET will train faculty that choose to use their tools so that there is greater accuracy in identifying best practices (validity) and better inter-rater agreement (reliability), so that biased ratings can be addressed and remedied.
- Schools must provide training to faculty who will use peer-review tools, whether they are tools developed by CET or not.
Will there be resources to help?
Extensive resources have been committed to making CET a full-service teaching resource center. It has developed teaching development opportunities for early-career faculty and graduate students, as well as senior faculty who wish to lead development opportunities in their school. It has created editable, customizable peer-review tools based on best teaching practices, and consultation on ensuring incentive structures (merit, promotion, etc. criteria) are sufficiently rewarding as to motivate faculty to invest in all aspects of teaching. CET is available for training on CET peer-review tools. CET is also available to consult with schools as they map out their plan, and can work with them to customize their evaluation tools (whether they were developed by CET or not) to the teaching practices in their discipline. Finally, CET is available for individual and group faculty consultations on courses or curriculum.
Should we be using any of these new systems for current merit reviews?
We are recommending that schools use teaching reflection statements for this year.