Originally posted at Evaluspheric Perceptions
Just last week, one of my favorite evaluation blogs, Emery Evaluation, featured a guest post that got me thinking. Exploring the Non-Profit Paradox – Evaluation and Non-Profits [Guest post by Jamie Clearfield] reminded me that I’ve long thought there exists a dearth of program evaluation in public schools. As Jamie indicates for the world of non-profits and community-based organizations (CBOs), I too believe there is a lack of understanding of evaluation and its role in public education. How do I know this?
I have the good fortune to work in a PK-12 public school district by day, and teach program evaluation courses at a university at night. The majority of my doctoral students work in public education, most as school and district administrators. By and large, they are wonderful individuals, eager to learn and assiduously apply their learning not only in class, but also in their schools. However, semester after semester they tell me that they come into the course thinking they know evaluation, and leave the course with a completely different understanding of the field and its potential role in and impact on the work they do.
Virtually none of them has ever heard the terms “logic model” or “program theory” prior to taking the course. They come in lacking an understanding of evaluation questions as distinguished from questions asked on a measurement instrument (i.e. survey or interview questions). They lack a clear understanding of the purposes of evaluation and are surprised to find that you can evaluate program processes and implementation, in addition to outcomes or impacts. They have no idea that deceptively simple sounding questions, “Is the program working?” or “Is the intervention effective?” are indeed fraught with complexity when we make systematic attempts to answer them.
I don’t fault my students for this – in fact, I congratulate them for taking evaluation courses as it is evident there is much to learn. Their higher education preparatory experiences coupled with their work experiences obviously did not expose them to these concepts. It’s no wonder to me, as I was in the same boat prior to my first evaluation course. I had worked in public education for nearly two decades at the time and I too, had the same gaps in understanding and misconceptions about evaluation.
Jamie makes an excellent point as relevant to non-profits and CBOs as it is to public education: “herein lies the paradox…evaluation can help avoid the ongoing crisis cycle and can help organizations plan and prepare for the days ahead, while also identifying where programs are succeeding or need to be rethought.” So true, but how do we mitigate the insidious effects of the paradox – that organizations, including school districts, continue to push evaluation aside or only engage with it when called for by grant funders or other accountability requirements? Do organizations view evaluation as a waste of precious resources rather than as a vital nutrient of a healthy organization?
Like Jamie, I too “get it” and believe that evaluation is a “hard sell,” as is anything people don’t understand well or feel will benefit them. An often identified dysfunction of evaluation is that it is something done to and not for programs or people, and we’re still fighting the inherently negative connotation of the word itself. Other concerns catalyze the crisis cycle – most notably money and time, the two most needed and scarcest resources not only for evaluation, but for program development and implementation as well (the fact that evaluation can and should be an integral part of these from the very birth of a program idea is fodder for a separate post).
What is the answer? Jamie offers this: “At the heart of many of these questions is the need to develop strong working relationships.” I strongly concur, and will offer this: education about evaluation and its potential to enhance organizational health, learning, and efficiency can serve as rocket fuel for these relationships. But, as one commenter cautioned, “Instead of approaching an organization with the question, ‘How can I teach you about evaluation?’, I go into the conversation with the question, ‘What are you trying to do and how can I help you do it?’” Good advice.
As Jamie says, there are no “zero sum” answers, but Sheila says, there is available help. One resource that comes to mind is Real World Evaluation: Working Under Budget, Time, Data, and Political Constraints by Bamberger, Rugh, and Mabry. This comprehensive evaluation text for conducting successful evaluations in complex contexts is complemented by the site Real World Evaluation which offers a multitude of resources in addition to the text.
Please forgive the momentary visit to “clicheville,” but I do believe there is hope and knowledge is power. To this day, one of my students from years ago still says whenever she sees me, “Your course changed my thinking about everything. I can’t think about anything anymore without thinking in terms of evaluation!”
So, kudos to Jamie for sharing her thoughts and kudos to Ann for publishing them. Let’s keep the conversation going!
Filed under: Books, Education, Evaluation Resources, Evaluation Terminology, Uncategorized Tagged: books, education, evaluation resources, evaluation terminology, teaching evaluation
To Comment, visit the original post here: Evaluspheric Perceptions
