Forum on External Peer Review
The Forum on External Peer Review was held on March 21, 2002 at Seton Hall under the joint sponsorship of the Faculty Senate, The Center for College Teaching, and the University Library. The Forum was organized and moderated by Dr. Albert Hakim and Dr. Richard Stern. The featured speaker was Dr. Shelley Rhoades-Catanach from Villanova University. At this site you will find the following information on the forum and related information on external peer review:
A copy of Dr. Rhoades-Catanach's Powerpoint presentation, "External Peer Review for Promotion & Tenure Decisions".
Notes of the question and answer portion of the Forum.
Wording of the Provost's proposed change to 1. Guidelines for Applications for Promotion, Tenure, Sabbaticals and 2. The Faculty Guide to include external peer review.
A study of external peer review requirements as posted on academic websites conducted by Dr. James Hanson.
A link to Dr. Rhoades-Catanach's article and other articles and book chapters on external peer review through E-reserves.
A link to Dr. Shelley Rhoades-Catanach's website.
A copy of Dr. Rhoades-Catanach's Powerpoint presentation, "External Peer Review for Promotion & Tenure Decisions".
Notes of the question and answer portion of the Forum.
A question or comment from a member of the audience is denoted as “Q”, and a response from the speaker, Dr. Shelley Rhoades-Catanach, is denoted as “A”.
Q: What was her experience in going through external peer review during tenure review at Villanova?
A: She became more aware of external expectations and what a good record is, and she became aware of the need for research to be coherent and explainable to an outside person.
Q: Is it necessary to be advised of the external peer review requirement several years in advance of the tenure and promotion process?
A: Without preparation the candidate is at an unfair disadvantage.
Q: Should external peer review be put into effect only when it’s part of the faculty guide? Should it be made mandatory only for faculty who are notified of the requirement when they are hired?
A: The process changes people’s perspectives. It could be made optional for several years and made mandatory from some point forward.
Q: Faculty and career development: money to go to conferences and editorial support to get published: Are these important for external peer review?
A: Face to face contact at conferences is helpful. To send a packet of materials to a reviewer who doesn’t know you is tough.
Q: What is the distribution of requirements for external peer review by level of program?
A: In accounting among the top 30 schools 90% require it.
For next tier, PhD granting, 75 programs, 80% required.
Don’t grant PhDs, 50% require it.
Q: What is release time from teaching to conduct research is provided at Villanova?
A: In business school, the regular load is 4/4. Anyone doing research does 3/3. Those who get highest ratings in research can get 3/2. At Villanova external peer review is required for promotion and tenure even though Villanova is primarily a teaching institution.
Q: What is the teaching load in the first year at the business school at Villanova?
A: 3/3 teach load. Research is supported for first two years.
Q: What materials did she send to external reviewers?
A: Vita, four research publications, personal statement, and cover letter from department chair.
Q: Is there a standard cover letter?
A: Varies by institution. Some have university-mandated letters.
In some institution letters are drafted by department chairs and may not be available to candidate. In some cases candidates know reviewers and others not
Q: Is there any study on the weight given to external review in promotion and tenure process?
A: No studies.
Q: How many external reviews do individuals conduct in a year?
A: Some people write as many as 10 to 15 external reviews per year.
Q: Who contacts external reviewers, candidate or chair of department?
A: Chair or head of review committee by phone or writing. Candidate is not usually allowed to contact reviewer. Often candidates talk to reviewers in advance of the promotion and tenure process. Some institutions will ask reviewers if candidates contacted them.
Q: Does the institutional affiliation of the reviewer matter?
A: Reviewers are usually asked for their vita. Judgments are made about reviewers.
Q: In SHU’s College of Arts & Sciences the use of external peer review has been unofficial policy put forward by the Dean. The candidate is provided with whatever is written about her or him. In this kind of open process reviewers decline to write letters if they feel don’t have something positive to write. They decline the request to review, but they do not state reasons. Most external peer review letters are positive. Reviewers may be reluctant to write negative letters because they fear that if the candidates know their identities, they might be sued.
Q: Is the external peer review process valid and useful?
A: It’s worthwhile because it forces junior faculty to think about how they will be perceived and what their scholarship is about.
Q: Scholarship is already peer reviewed when submitted for publication. Why is external peer review needed?
A: The journal peer review process is focused on a single publication whereas external peer review is about a body of scholarship.
Q: The external peer review is supposed to be unbiased. If it is necessary that the review be familiar with you in order to review your work? How can the external peer review be characterized as unbiased if they know you?
A: Scholars do not risk their academic reputations to give a positive that is not deserved.
Q: What is the problem that external peer review addresses?
Is it that we want junior faculty to plan their careers from day one?
Is it essential that a faculty development process be in place before an external peer review process is implemented?
A: To put a faculty development process in place before external peer review is implemented may take more time than a university administration is willing to wait.
Q: Faculty should resist the implementation of external peer review until requisite support is provided for faculty development at Seton Hall.
Q: At Villanova what level of support is provided to junior faculty for research during the summer?
A: In the Villanova school of business research support is guaranteed for two years at $12,000 per summer with no summer teaching.
Q: The addition of external peer review to the evaluation of scholarship is leading to overemphasis of research in a predominantly teaching university. We should ask ourselves: Who are we? Who do we want to be?
A: The purpose of external peer review is to have an external person to evaluate whether a candidate has met an institution’s expectations of scholarship. Expectations should be clear. If junior faculty have met expectations, they shouldn’t fear external peer review. To some it may seem that external review is used as a weapon to turn down candidates.
Q: In the SHU College of Arts & Sciences, new faculty are told that they need 4 to 5 refereed publications. New faculty are told that publication is primary focus of tenure and promotion review. The problem is whether is there is an adequate connection between resources and expectations, and therefore the process seems onerous to junior faculty. Teaching release time and financial support would bring expectations and resources in line.
Q: What information about the institution should be communicated to external reviewers?
A: Among the institutional characteristics to be communicated are teaching load and course preparations.
Q: Is there any data on negative external peer reviews and impact on receipt of tenure and promotion?
A: She is not aware of any hard data.
Wording of the Provost's proposed change to 1. Guidelines for Applications for Promotion, Tenure, Sabbaticals and 2. The Faculty Guide to include external peer review.
(Copied from Memo from Mel Shay, Provost, to Faculty Senate, November 2, 2001):
Proposed Changes to the Guidelines for APPLICATIONS FOR PORMOTION, TENURE, SABBATICALS & FACULTY GUIDE
Under Section I. I., Guidelines for Tank and Tenure Committees and Deans, [and Departments]
Subsection 1. states: “The application for tenure and promotion to associate professor is a single application decided in a single vote. Therefore, no distinction may be made between tenuring and promotion (Faculty Guide Article 3.3k).
Subsection 6 states: “In applications for tenure and for promotion to professor, external assessments of the application should be sought. A list of names may be provided by the applicant. PROPOSED: CHANGE: A list of names may be provided by the applicant: to: External assessment process to be followed as described in Faculty Guide Article 4.3b
PROPOSED ADDITION to Faculty Guide Article 4.3b:
Any material of substance that would disclose the level of scholarship of the candidate may be submitted to individuals outside the University for external review. A sampling of the candidate’s primary publications submitted with the tenure/promotion application shall be evaluated. The candidate shall submit to the department a list of names of persons outside of the University who are qualified to furnish evaluations of the candidate’s scholarship. It should be noted that the department is not limited to only these evaluations, but may make its own judgment as to those persons outside the University who are qualified to make such evaluations. After consultation with the candidate, the department may reduce the list of outside evaluators submitted to only a portion of the list. The candidate may object, for cause, to the use of any particular evaluator. Inquiry shall be limited to evaluations of (i) the quality of the scholarship, (ii) reputation as a scholar, (iii) impact of scholarship within the candidate’s academic discipline or field of study. External review of scholarship in the manner described above should be completed in time to submit evaluations in line with the October 1 application deadline.
A study of external peer review requirements as posted on academic websites conducted by Dr. James Hanson.
Online Survey of External Review at 25 schools surveyed by visiting their websites and searching for Faculty Guide or similar documents conducted by Dr. James Hanson, Chemistry Department, SHU & Chair, Faculty Guide Committee
No information available online (5 schools, 20% of sample)
Duquesne University (Tier 2)
University of San Diego (Tier 2)
Saint John’s University (Big East, Tier 3)
Southern Methodist University (Tier 2) [online material restricted to University community]
Providence College (Big East, Tier 1 “Master’s Universities North”)
No Formal Requirement for External Review (5 Schools 20% of sample)
American University (Tier 2)
George Washington University (Tier 2)
University of the Pacific (Tier 2)
University of San Francisco (Tier 2)
Syracuse University (Tier 2)
No Explicit Requirement for External Review, Suggestive Language (5 schools 20% of sample)
Tulane University (Tier 1) [Full Professors have “international standing”]
Texas Christian University (Tier 2) [Full Professors only]
Fordham University (Tier 2) [Full Professors have “achievement recognized by outside scholars”
Marquette University (Tier 2) [Full Professors have “established reputation among scholars”]
Notre Dame University (Tier 1) [Very vague guidelines on tenure and promotion]
Explicit Language Allowing but not Requiring External Review (3 Schools, 12% of sample)
Boston College (Big East, Tier 1) [Promotion Committee may consult with qualified outside scholars]
Boston University (Tier 2) [Department, School, College, or University Comm. may consult outside]
Clark University (Tier 2) [Department shall “normally” secure four outside evaluations]
External Review Required or Apparently Required (3 schools, 12% of sample)
Catholic University (Tier 2) [Promotion application has section for department chair to list reviewers]
Loyola Chicago (Tier 2) [University regulations not available, some department regs require review]
St. Louis University (Tier 2) [Qualifications for promotions require recognition by outside colleagues]
External Review Required and Guidelines Specified (4 Schools, 16% of sample)
Georgetown (Big East, Tier 1) [Review handled by department]
Miami (FL) (Big East, Tier 2) [Review handled by department]
Villanova (Big East, Tier 1 Masters Universities North) [Review handled by department]
Akron (Tier 4) [Review handled by department]
Note: A Seton Hall email account is required for access.
If the hyperlink above does not connect you to the list of external peer review materials, follow these directions:
Go to the library’s homepage http://library.shu.edu.
Click on E-reserves on the left side. http://library.shu.edu/e-reserve-services.htm
Click on Catalog for Electronic Reserves.
Browse instructor names for Richard Stern.
Click search.
Click on hypertext title for the title you want to access.
Click on Full-text version online. Requires SHU E-mail ID/Password, and Adobe Reader.
Enter SHU user name and password.
A link to Dr. Shelley Rhoades-Catanach's website.
http://www39.homepage.villanova.edu/shelley.rhoades/
For further information regarding this page, contact:
Richard E. Stern, Ph.D.,
University Library
Seton Hall University
South Orange, NJ 07079-2671
(973) 275-2046