Competency N

Evaluate programs and services using measurable criteria

Introduction

Evaluation as used in libraries can be defined as “the process of determining the results, costs, outcomes, impacts, successes, and other factors related to a library’s collections, programs, services, or resource use” (Matthews, 2018, P. 255). Evaluation provides managers a way to move decisions from the nebulous to the concrete by offering data driven decisions to lead instead of just thoughts or feelings. In this way, library management can clearly demonstrate value to library boards, library staff, and library patrons. 

Evaluation lives within a framework of good management. Peter Drucker, who has worked extensively in management theory, offers the following five essential questions that every organization, libraries included, should be able to answer (Matthews, 2018, P. 256).

  1. What is our mission?
  2. Who is our customer?
  3. What does the customer value?
  4. What are our results?
  5. What is our plan?

To begin, library management need a clearly defined mission statement and customer profile, including understanding what the customer values. Evaluation is the process by which library management can then answer the fourth item from Drucker’s list.  

Evaluation within a library follows many of the same steps as in the research design process. First, a clearly defined evaluation question needs to be formulated. By understanding the ultimate goal of the evaluation one is better able to decide on the most appropriate evaluation question. For example, the library looking to understand how a service can be improved requires a different research question than one that is looking to understand which patron population uses a service. After thoughtful formulation of the research question, one follows the other steps of the research design process. These include choosing research variables, deciding if qualitative (observations or interviews) or quantitative (surveys or counting and measuring) research will work best, and determining the sample population. One of the most important considerations when determining the sample is to remember that generalizations can only be drawn about the entire population when a sufficient sample size is collected. 

Evaluation is not complete without some sort of evaluation report being created. The report should include a review of current literature pertinent to the evaluation topic, a study methodology discussion, and acknowledge the limitations of the study. The intended audience for the report should be considered in the writing. But overall all evaluation reports should be concise, provide clear graphs and charts to support data visualization, and include useful recommendations for improvement.

Competency development

I received my formal training evaluating programs and services in my classwork at SJSU. INFO 210 provided an opportunity to evaluate reference interactions using the RUSA guidelines. Other evaluation opportunities were scattered throughout my coursework in INFO 282 and INFO 250. 

Evidence Description

INFO 210 – Reference Observation

I am submitting this reference observation paper as proof of my ability to evaluate services using clear criteria. In this paper I use the RUSA guidelines and guidelines from the Reference and Information Services textbook to evaluate an online reference interaction using a chat feature and an in person reference conversation.

INFO 282 – Metadata Analysis of a Video Collection

I am submitting this metadata analysis of a video collection to show my ability to evaluate a collection using the 15 Dublin Core fields as a video metadata standard.

INFO 250 – Instructional Design Examples Review

In this evaluation of instructional design examples I use the standard implied in Joan Kaplowitz’s book ‘Designing Information Literacy Instruction’ to critique two real world educational products.   

Concluding Remarks

All programs and services should be evaluated regularly to verify that these offerings are meeting the needs of the patron community. Evaluation also provides for decisions to be driven by data instead of just one manager’s thoughts or feelings. Proper evaluation provides a strong foundation for managers as they answer steak holder questions and concerns.

References

Matthews, J. R. (2018). Evaluation: An introduction to a crucial skill. In K. Haycock & M.-J. Romaniuk (Eds.), The portable MLIS: insights from the experts (Second edition, pp. 255-264). Libraries Unlimited, an imprint of ABC-CLIO, LLC. 

Last updated October 17, 2021 8:55 am.