Carman and Fredericks describe three types of organizations in terms of their evaluation capacity. If you are currently working in the social and behavioral sciences, use your current organization; ( You can use the Department of Social Services) if not, interview someone in your community who works in such an agency or organization about their experiences with program evaluation and collaboration. Assess and describe the program’s evaluation capacity. How important is program evaluation in this organization? What are the greatest challenges this organization faces in terms of evaluation? Why is evaluation an essential part of ethical program implementation?
In preparation for the Unit 8 discussion, also ask in your interview about community collaboration. With what organizations and groups do they collaborate? What are the benefits of collaboration? What are the biggest challenges?
unit reading: Implementing And Evaluating Programs
One of the “extra benefits” of this course is that, depending on your current and future leadership goals, the text and many of the assigned journal articles, such as the one in this unit by Carman and Fredericks (2009) offer insights into organizational management and administration as well as zeroing in on the management of a specific program or project. Evaluation capacity of the organization is an issue that will be discussed in more detail in the course, Advanced Program Evaluation. Evaluation capacity is the organization’s ability to carry out evaluation as an integral part of their operation. There are numerous models discussed by Carman and Fredericks (2009), as well as activities needed to carry out evaluation, and barriers to evaluation activities identified in this article.
As depicted in the Framework for Program Development and Evaluation (Comeau, 2011), evaluation is an ongoing process that feeds back into program improvements as well as communication with funders and other key stakeholders who support the program (resources/inputs). The two major types of evaluation are (Calley, 2011; Kellogg Foundation, 2004):
• Process or formative evaluation.
• Outcome or summative evaluation.
Evaluation can also be seen as part of the ethics of practice, with the obligation to not only be sensitive to the culture and changing needs within the population being served, but also to maintain ongoing accountability to and communication with stakeholders (Calley, 2011).
You may want to go back and scan Chapter 5 in Conyne (2010) as well as the Kellogg Foundation materials as you review the new material on evaluation in Chapter 12 (Calley, 2011). In Conyne’s (2010) Chapter 9 he introduces the question, how is something “evidenced?” This ties back to the Logic Model work and the identification of measurable objectives and outcomes. In research terms, the research would ask, how do we “operationalize” a concept or desired characteristic? This of course leads the program designer back to the research literature. Prevention programs are designed to reduce incidence by increasing protective factors and reducing risk factors; stakeholders want to know, how are participants “evidencing” the desired characteristics? While this is an evaluation question, it must also be a program design question, where developers work both backwards and forwards to build research-based activities that are believed to lead to certain outcomes, which can only be measured by identifying their logical indicators! How will we know that change or growth when we see it? How is it “evidenced?”
