Guidance for Internal Faculty Evaluators

Overview

Internal faculty evaluators—department colleagues, special committee members, P&T committee members—play a critical role in the review process for Lehigh faculty. As an internal faculty evaluator, you are a key voice in assessing the quality of the research, teaching, and service of review candidates. You provide a perspective that external reviewers cannot provide, because of your unique insight into Lehigh, and into parts of the candidate’s dossier (e.g., teaching quality) that are hard to assess externally.

At the same time, you are responsible for providing a fair assessment, with a careful eye toward ensuring that outcomes are inclusive, equitable, and transparent. The goal of this page is to provide evidence-based practices to help you in this endeavor, highlighting research about differential impacts of biases within different parts of the faculty review, and empowering you to foster inclusive, equitable evaluations.

This page provides evidence- based on national guides such as the AAUP’s Good Practice in Tenure Evaluation and work by ADVANCE groups at Lehigh and elsewhere, as well as Lehigh’s Rules and Procedures, applicable legal requirements, recommendations from Lehigh’s Faculty Personnel Committee, and discussions with college promotion and tenure (P&T) committees.

For more information on specific types of reviews, consult the individual pages for those reviews in the navigation menu. 

Best Practices for Internal Review

The following six “C”s should govern faculty evaluation at every level:

• Clarity • Consultation • Consistency • Confidentiality • Candor • Caring

1. Follow department, college, and University guidelines. In all deliberations and recommendations, evaluators at every level (department/program faculty, P&T committee, Dean, and Provost) must equitably use and adhere to all applicable University policies and procedures, including R&P and the college and/or department guidelines on criteria for tenure and promotion (e.g., the departmental “Appendix”). If the department or college guidelines contradict R&P, the university-wide R&P prevails.

2. Use appropriate channels to handle questions. When a question or “gray area” arises regarding the criteria or procedures (and interpretation questions are not unusual), the Department Chair should consult with the Deputy Provost for Faculty Affairs (DPFA). The DPFA will consult with others as needed. In the College of Arts and Sciences, the Department Chair should first contact the Associate Dean for Faculty and Staff.

3. Maintain confidentiality. Confidentiality must be maintained throughout the process as well as after the process is complete. This includes all confidential materials in the candidate’s file, meetings of the department/program faculty to discuss the case, and any meetings related to the case with the P&T committee, Dean, and Provost.

4. Only review complete files. Ensure you can access the entire dossier before engaging in a review. Review the entire file before meeting with colleagues to discuss the case. The Department Chair must ensure that the review dossier is complete before sending it to the Dean’s office.

5. Consider new information that becomes available during the review. If the candidate has published a new book or article, gotten a new grant, etc., they may update their dossier. The candidate should inform the Department Chair, who can help with the logistics of updating the dossier.

6. Make clear, unambiguous recommendations in letters. Provide evidence regarding each of the domains of research and scholarship, teaching, and service to support the recommendation. It is fine for letters to discuss both strengths and weaknesses of the dossier, but the recommendation itself (e.g., grant or not grant tenure) should be unambiguous.

7. Ensure that letters and votes are consistent. For review types that include a vote (reappointment, tenure, promotion), faculty members’ individual letters should align with their votes. If a letter expresses many negative opinions of the dossier but the faculty member votes “yes,” the letter should also explain the positive factors in the dossier and why the positives outweigh the negatives. Similarly, a “no” vote should be well supported by the letter, including specific reasons why the faculty member feels the dossier falls short of meriting reappointment/tenure/promotion. When letters are not well aligned with votes, it often makes it difficult for the Chair, Dean, and the Provost to interpret the basis for the faculty’s votes and to make their own recommendation.

8. Do not consider personal situations. Personal issues such as family, health, parenthood, or other situations must not be considered or discussed during reviews. Faculty must not request access to such information, even when the probationary period has been extended.

9. Use the same standards for candidates who received tenure-clock extensions. The standards for tenure are the same for candidates who received extensions and for those who did not. If the candidate reveals the reason for an FMLA leave or tenure extension, you must still use the same standards for tenure as for candidates who do not receive extensions or take FMLA leaves.

10. Personal characteristics such as race, ethnicity, gender, disability, etc. must not be considered. Discrimination is unacceptable and illegal. Familiarize yourself with and adhere to the University’s Policy on Harassment and Non-Discrimination.

11. Lack of a response from an external evaluator is not a negative evaluation. If an external evaluator ignores a request or declines to provide a letter, this should not be interpreted as a comment on the case itself. 

12. The Department Chair is the contact person for requests from the P&T committee or Dean’s office for additional information. The Chair will also coordinate the response of the department faculty when the P&T committee is considering a recommendation contrary to that of the department faculty.

13. Address weaknesses of the case if you are recommending for promotion/tenure and strengths of the case if you are recommending against. For example, if there are differences between external evaluators’ comments/recommendations and the committee’s recommendation, this should be addressed directly.

Cornerstones of Fair and Equitable Evaluation

1. Frame your evaluation in terms of how the scholar’s accomplishments align with the stated criteria and contribute to the mission of the department, college, and university. Refresh your knowledge of the University Mission, Diversity, Equity, and Inclusion Plan, and Strategic Plan and related statements from your department and college to orient your perspective. Consider the relevance and significance of the candidate’s efforts not only for the intended scholarly audience but also for our students, institution, and external communities.

2. Lehigh faculty should be reviewed holistically. A holistic review takes into account the totality of contributions and impacts the candidate has made that contribute to excellence in research, teaching and service/leadership, as well as the totality of resources, opportunities, and constraints the candidate has been presented with. Remember that a candidate’s work might not fit neatly into a traditional category for evaluation. 

3. Recognize and commit to intentional, inclusive, equity-minded process and practice. Lehigh University expects all faculty evaluations to be inclusive, equitable, and fair. Evaluators should acknowledge that the entries on a CV reflect the scholar’s accomplishments, as well as years of hidden opportunities, challenges, choices, and biases: The portfolio reflects an entire ecosystem of human decisions and processes. Evaluators, as individuals and throughout group deliberations, must be aware of the potential for biases, from the evaluation criteria, through the decisions, and the way feedback is returned to the candidate. Conducting a faculty evaluation involves committing to this entire process. It involves self-reflection and education, devoting the requisite time, and holistically examining the evidence within the portfolio against the transparent, equitable, and inclusive criteria.

4. Grow and apply your understanding and competencies for recognizing and responding to bias and equity barriers. Biases in evaluating teaching, research, and service are widespread in academia. Implicit biases, stemming from stereotypes and expectations, are displayed by everyone, regardless of race or gender, and can be compounded by factors like country of origin, discipline, and degree institution. These biases subtly advantage majority group members, accumulating over their careers. Recognizing and addressing these biases is critical to ensuring equity. People and teams who are unaware of barriers for women and BIPOC faculty tend to promote fewer of them, but regular education on biases mitigates this effect (Regner et al. 2018). 

5. Prioritize equity over equality during the evaluation process. Emphasizing equity in faculty evaluations acknowledges unequal workloads in teaching and service, so research expectations may vary accordingly. An equitable review values and rewards exceptional service or teaching beyond the department’s basic expectations.  For example, a faculty member who (perhaps by virtue of their race or gender) has taken on disproportionate service roles, and therefore far exceeds the service excellence bar, might clear the research excellence bar with a different margin than a faculty member whose research time has been protected and who therefore far exceeds the research excellence bar but more narrowly exceeds the teaching and service excellence bars. This is expected and acceptable, and both cases might be worthy of a positive review outcome.

Preparing to Evaluate Faculty

1. Review and refer to all department, college, and university guidelines for the administrative and procedural aspects of faculty review, as well as for information about the standards of excellence to be applied.

2. Apply clear expectations for department and/or committee members’ roles. Understand your role in the faculty evaluation process. Department and committee chairs should clearly communicate expectations for members. All evaluators should attend all meetings if at all possible, participate actively in the process, treat all applicants fairly, maintain confidentiality, and understand how decisions and reporting will take place. See also the ADVANCE guide for P&T committee chairs. 

3. Review and refer to established departmental criteria when reviewing portfolios. Every department should have established criteria for the review of research, teaching, and service records (e.g., a departmental “Appendix”.) Apply these criteria during the evaluation process, and avoid the use of criteria that are not relevant. Evaluators should have a common understanding of how to interpret the criteria.

4. Establish and apply ground rules, norms, and accountability for:

   • Evaluation discussions and deliberations. Group practices should ensure that all voices are heard and shared. Faculty should be empowered to speak up and to turn the conversation back to shared agreements, questioning assumptions or conclusions put forward without evidence. Case studies have shown that a single strong opinion can unduly affect the views of other members of the committee (Niemann 2012). Anonymous voting is one way to ensure equal consideration (though this is not practical in all voting situations). 

   • Confidentiality. Confidentiality is crucial in faculty evaluations. Discussions and materials must not be shared with staff, students, non-committee faculty, or those from other departments. This includes all confidential documents and meetings related to the case, even after the process is completed. 

   • Conflicts of interest. Prior to reviewing a candidate, discuss the types of circumstances that might constitute a conflict of interest and possible recusal. See R&P §2.2.2.2.4.

Mitigating the Impact of Implicit Bias

1. Allow sufficient time for the review. Implicit bias is most likely to creep in when people work quickly or are under time pressure. Candidate portfolio materials should be reviewed methodically. 

2. Tenure clock extensions do not change tenure standards. Do not penalize candidates for taking tenure clock extensions by expecting a stronger record relative to their years in rank. The same tenure standards and criteria apply, regardless of the number of or types of extensions. Accomplishments relative to opportunity must be considered against the criteria, and personal aspects about the candidate should not be discussed among evaluators.

3. Time to promotion does not change promotion standards. Candidates for promotion to Full Professor should be evaluated using the same expectations and criteria, regardless of the number of years since their last promotion or the number of triennial reviews they have had.

4. The mechanism by which a candidate was hired is irrelevant to a review. Reviews should focus on whether and how well the candidate meets the review criteria, not hiring circumstances. Do not revisit how someone joined the department during the evaluation; doing so runs the risk of shifting the standards (Niemann 2012). 

5. Interpret criteria consistently, even for candidates with joint appointments. Faculty with joint appointments may face complications in reviews due to differing norms between disciplines, with work done for one department/program possibly going unrecognized or unrewarded by the other. The MOU and annual review process should establish expectations between the units to ensure evaluations are conducted in a fair, equitable, and consistent way, and that joint appointees are not expected to do double duty.

6. Avoid tokenism when reviewing a candidate from a marginalized group. In departments with an overrepresentation of white or male faculty, historically underrepresented faculty may face tokenism, by which they are seen as representatives of a particular group they belong to. Tokenism can include drawing on stereotypes or labeling someone as a “diversity hire,” implying they were hired for symbolic reasons rather than their own merit (Dovidio et al. 2010, Fiske and Taylor 1984, Niemann 2012).

7. Speak up. Use questions to call for evidence or rationale if you don’t understand a colleague’s comment, whether positive or negative. Notice inconsistencies. Some examples can include: “Why is this coming up?” “How should we consider this information relative to the case?” “Why is this coming up when X is discussed but not when Y is discussed?”

8. Be careful of the tendency to compare a candidate to others. The candidate you are reviewing may reflect a very different life history and path than other candidates you are familiar with. Your own expertise and experiences may not be a reliable point of comparison. Evaluate candidates against the criteria, not a mental image of an ideal professor. Different journeys yield different priorities and perspectives, and these differences add richness and value to our shared work and life.

9. Avoid informal discussions of a candidate’s record prior to or outside of the formal review. Informal conversations about whether a candidate has a strong review case are often based on dated information, assumptions about the quality of the candidate’s work, and even gossip. Even well-intentioned conversations can introduce unfair and biased perceptions; they can also be overheard. Informal discussions also lack documentation to support or refute a position or recommendation.

10. Evaluate scholars based on their achievements relative to their opportunities, especially with regard to COVID-19. Acknowledge that the COVID-19 pandemic shifted faculty experiences and workloads in a variety of ways, with a variety of impacts. Read and consider the COVID impact statement, if any, in the dossier. (See the page on COVID Impact Statements, especially the section on Guidance for Evaluators, for guidance on how to interpret such statements.) An important reminder from a report by the University of Michigan’s ADVANCE program: Do not let the 25% of faculty who were able to be more productive during the pandemic set the standard for the 75% who were not able to do so.

Best Practices for Reviewing Dossiers

Research, Scholarship, and Creative Works 

Definitions and modes of scholarship have broadened in recent decades, yet faculty reviews often tend to favor sole-author, discovery-based research at the epistemic center of a field over other forms of scholarship. This preference can disadvantage faculty who are involved in community-engaged, integrative, applied, translational, and/or interdisciplinary scholarship. (Material in this section is adapted from guidance from University of Massachusetts Amherst ADVANCE Program.)

1. Recognize the potential for bias in external review letters. External review letters are important for addressing scholarly impact; however, much research shows that they can introduce bias, often manifesting as gender or institutional prestige bias. An institution’s promotion decision often reflects the letter writers’ personal characteristics rather than the candidate’s performance (Madera et al. 2024). Ensure consistent weighting and valuation of the letters across all reviewees.

2. Acknowledge the limits of existing indicators of impact and broaden what is considered. 

   • Decenter citation metrics (e.g., journal impact factor, citation counts, and h-index) as the main indicator of impact. Place more emphasis on the quality, originality, and impact of the work on society or the discipline. Metrics like citation counts and h-index offer limited value for comparing faculty and are particularly flawed for early-career scholars. They introduce false precision, emphasizing quantity over quality, without reflecting the creativity of someone’s work, and often reflecting personal biases. They can be exaggerated by self-citation and network citation practices that often disadvantage women, scholars of color, and international scholars (Mitchneck 2021, Lariviere et al. 2013). Use citation statistics cautiously due to these limitations and biases (Milard and Tanguy, 2018, Zare 2012). 

   • Consider citation statistics within the context of the field of study. One approach is to compare citation statistics with comparable scholars in the same area of specialty who are at a similar stage in their career. Another approach is to use the Field Citation Ratio (FCR), which is calculated by dividing the number of citations a paper has received by the average number received by papers published in the same year and in the same fields of research. 

   • Consider multiple or newer metrics of scholarly impact. Almetrics provide an alternative (and also imperfect) way to document impact that captures scholarly research cited in public policy documents, course syllabi, social media, and more (Herbert et al, 2016, 2023;  Mitchneck 2021). 

3. Consider context regarding journal choice. Acknowledge that some scholarship will have a greater impact if published in a subdiscipline journal or other venue with a lower impact factor but a more appropriate audience. Restricting perceptions of impact to a narrow list of publications can disadvantage some faculty. Departments should ensure that target journals cover all faculty research areas and recognize that those listed venues’ impact factors may arise from biased editorial choices.

4. Focus on the quality of the candidate’s research accomplishments, not the candidate’s research area or mode. Case studies, especially of minoritized faculty, suggest that some faculty are reviewed negatively based on their area of specialization, rather than their actual accomplishments (Niemann 2012, Settles and Linderman 2020). 

Teaching

1. Minimize reliance on student evaluations of teaching (SETs). Studies find that SETs are not a reliable indicator of teaching effectiveness or student learning (Braga et al. 2014, Lee et al. 2018) and can be biased, particularly against faculty from historically underrepresented groups. Students often evaluate men based on qualifications and competence but women based on personality and appearance. Despite similar student performance, women, especially older women, receive lower ratings, particularly from male students. Black and Asian faculty also receive lower evaluations compared to their white counterparts. (See O’Meara and Templeton 2022, and references therein.) 

2. Take a holistic view of teaching, with greater emphasis on peer evaluations and student outcomes. Evidence beyond SETs include syllabi reviews, classroom peer observations, student work artifacts, and discussions on pedagogy. Peer teaching reviews convey constructive feedback about the quality of course content, organization, instructional materials, assessment methods, and concern for student learning. Student outcomes include information about completed undergraduate honors theses, high-impact learning experiences, graduate student accomplishments and job placement, and so on. Faculty should have the opportunity to present diverse evidence of their teaching effectiveness. (See Aruguete et al. 2017, Boring et al. 2016, Boring 2017, Fan et al. 2019, MacNell et al. 2015.) 

3. Select a balance of positive and critical comments from SETs, rather than selectively considering negative comments. Case studies of faculty members from systemically minoritized groups find that P&T committees can misrepresent a faculty member’s teaching effectiveness by selectively presenting negative comments (Niemann 2012). 

Service 

1. Acknowledge that faculty from historically underrepresented groups are likely taking on extra service responsibilities based on their social identity. For example, BIPOC faculty, female faculty in many disciplines, etc. may be asked to serve on more committees than other faculty members so that the committees are more diverse. Such faculty are also disproportionately asked to serve on committees that focus on diversity and inclusion. Finally, they are disproportionately expected to mentor junior faculty and students from underrepresented groups. Taken together these practices have been referred to as a minority tax, which typically comes at the expense of time that could otherwise be spent on research (Padilla and Chavez 1995). This service should be recognized and rewarded. 

2. Give credit to the impact of both formal and informal or “invisible” service. When service work described in the dossier is above the departmental baseline, it should be included as an asset to the overall assessment. Evaluators can demonstrate that they value these forms of service by understanding that colleagues may have additional, often invisible but impactful, labor, and acknowledging that the time spent on this service may reduce time on scholarship. Recognize the importance of such invisible service in the department summary letter, individual faculty letters, and P&T committee letter, acknowledging the impact that such service has had on the department, on colleagues, and on students.

3. Acknowledge that identities are complex, and disadvantages accumulate in intersectional ways. For example, the experiences of a Black woman in a STEM discipline will be different from the experiences of a white woman in the same discipline, and also different from those of a Black man.

4. Poor service can be reflected in the recommendation. All faculty are expected to engage meaningfully in service; therefore, it is reasonable for poor service to be a demerit in a review case, just as it is reasonable for exceptional service to be a boost to the case.

Notes

• This overview is intended to supplement the information contained in Lehigh’s Rules and Procedures of the Faculty (R&P). If there is a discrepancy between the guidance on this page and R&P, the provisions of R&P govern.

• Feel free to contact your Department Chair, Associate Dean for Faculty, the Deputy Provost for Faculty Affairs, or the Director of Faculty Affairs with any questions or concerns.