Report on Peer Review Committee Activities, 2002/03
June 19, 2003

Members: Frank Quinn (co-chair), Sylvia Turchyn (co-chair), Elizabeth Hanson, Joel Silver

Contents

• Introduction
• Ratings
• Observations and Recommendations

Introduction

The Committee read and evaluated the annual reports for the year 2002 submitted by 65 librarians, including 1 extra-system librarian. We used as our guidelines the procedures and criteria found in the Revised Peer Committee for Annual Merit Review (8/4/99; matrix amended by vote of BLF 7/01). We kept in mind the general principle of focusing "on the Librarian's position description in defining the arena for judging the accomplishment and nature of the composition." We did not have any information about last year's ratings for any librarians and kept in mind that we were making our judgments based on one year's work. We also did not have any supervisor's evaluations. We kept in mind the expected standards for each area (i.e., performance, professional development, and service) and for rank. These standards, taken from the Peer Review guidelines, are:

"In order to achieve a Level II rating in performance the following standard must be met. Assistant librarians: good, meets the requirements of operational standards. Associate/Full librarians: excellent, exceeding the requirements of operational standards."

"In order to achieve a Level II rating in professional development the following standard must be met. Assistant librarians: promising beginning; Associate/Full librarians: satisfactory, demonstrating responsiveness to the demands of the profession."

"In order to achieve a Level II rating in service, the following standard must be met: Assistant librarians: promising beginning; Associate/Full librarians: satisfactory, reflecting favorably on the University and the Libraries."

We want to thank the Libraries' Human Resources Department, especially Sheila Brock and Betty Davis, for their work in making copies of all the annual reviews, tracking down missing position descriptions and annual reviews, preparing letters to the librarians, and other associated work. We also want to thank the members of last year's Peer Review Committee for sharing their experiences and advice with us.

Ratings

The Committee used the following terms for ratings: Level IV-outstanding/exceptional; Level III-noteworthy; Level II-satisfactory/excellent, and Level I-not meeting expectations for rank. The Committee wrote brief comments whenever a Level III or Level IV rating was assigned.

It was frequently necessary for the Committee to restructure annual reviews in order to evaluate activities in their appropriate categories prior to the assignment of ratings. It was not uncommon for activities to be incorrectly assigned, and the Committee decided that the fairest procedure was to consider activities in their most appropriate categories. We discuss this issue further in the Observations and Recommendations section below.

The results, by rank, were as follows:

IV (Exceptional) = 8 (2 Assistant Librarians, 3 Associate Librarians, 3 Librarians)
III (Noteworthy) = 41 (4 Assistant Librarians, 24 Associate Librarians, 13 Librarians)
II (Satisfactory) = 16 (1 Assistant Librarian, 14 Associate Librarians, 1 Librarian)
I (Did not meet expectations for rank) = 0

Observations and Recommendations

Observations

a. We found that the position descriptions varied greatly, especially in the level of detail. Some people had one paragraph with a very general description, while other position descriptions were many pages long and listed a large number of responsibilities, many of which could have been subsumed under a more general area of responsibility.

b. The same discrepancies were found in the reviews. It was helpful when the librarian identified and explained particular achievements, e.g., the librarian's actual role in a committee.

c. Some librarians' reviews looked more like departmental annual reports than the review of an individual librarian. In such cases, it was often difficult for the committee to determine what the actual accomplishments of the individual librarian were.

d. The performance narrative should address the duties outlined in the job description. Some reviews discussed work not included in the review, and others did not address primary responsibilities listed in the job description. This made it difficult for the committee to evaluate how well the librarian fulfilled his or her job responsibilities.

e. The type and amount of documentation included was also a problem. Additional documentation should be included only when the narrative can't adequately explain the accomplishment. Much of the documentation provided little useful additional information, and it received only cursory examination, if that.

We would also like to note that our observations were so similar to those described by last year's Peer Review Committee that we decided to use their wording in the text above.

Recommendations

1. We recommend continuation of the workshops on preparing and writing of position descriptions and annual reviews/reports.

2. We recommend that all librarians who have ever submitted or who plan to submit a review to the Peer Review Committee be required to attend a workshop once every three years.

The Peer Review Committee was compelled to make this recommendation in light of this continuing situation, where the same problems have surfaced two consecutive years.

In addition, the Committee's experience led us to the inescapable conclusion that many librarians do not assign activities to their most appropriate categories. Theoretically, watchful supervisors catch such mistakes in annual reviews, but this Committee found too many errors to ignore. One of our most time-consuming tasks was in re-sorting annual review elements so that they made sense as "performance," "professional development," or "service." We think that these courses of action would be very much to the advantage of IUB librarians, and would considerably streamline the Peer Review Committee's deliberations. At the same time, some Committee members noted that the target audience for the workshops is primarily librarians who have already attended one or more of these workshops, yet the errors continue year after year.