Assessment

As we outline the basic steps of organizing evaluation efforts, let’s define the essential terms.

Activity: The individual project or program components that people participate in (e.g. a community dinner).

Evaluation: Understanding the process, results, and outcomes of your work.

Outputs: Generally, answer the question “What difference will the project make?”. These changes are framed in terms of the specific actor (i.e. participants, residents, families, organizations, systems, etc.) and the specific area of change (i.e. awareness, actions, capacity, attitudes, etc.)

Outcomes: These are the quantifiable results achieved by a project—they identify specific activities done by a specific group of program participants. An output could also help to quantify outputs produced, or programming offered. Because outcomes are a basic measure of quantity, they are necessary for understanding the success of a program, though outcomes alone are not sufficient.

Short-term outcomes: the immediate effects of the project

Intermediate outcomes: initial effects that linger over time (e.g. “In the first six months of our project we saw...”)

Long-term outcomes: the longer term goals of the project are often difficult to identify early on, such as decreased neighborhood violence, improved economic viability, or increased social cohesion, but can be referenced when framing earlier outcomes

Indicators: These are qualitative (numbers) or quantitative (descriptive) measures of a specific outcome. Indicators are capable of being measured, through observation, conversation, reflections, et cetera, but also seek to answer the question, “How will we know when we’ve achieved our outcome?” Often an outcome can have several different indicators associated with it.

Once you have the right person or people in place to help with the evaluation process, you can begin walking through the different steps:

• Identify what you know already: Chances are, you already know quite a bit about the project, its context, and even the initial impacts it’s making. Even the project grant application itself can be a good source for forming the initial narrative. What you already know will become part of the larger story that the evaluation process will help to tell.

• Determine what you need to know: At this stage, the project team should discuss the kinds of outcomes you imagine seeing and the specific audiences the final report / story will need to speak to. Different audiences will need information in different formats. Policy makers might appreciate more quantitative results, funders might like a mix of qualitative and quantitative information, and local community advocates may feel the stories of impact are the most powerful. The audience you are trying to reach should inform the kinds of impacts you look for and how you present them. Creative placemaking evaluation isn’t a controlled scientific investigation. It’s about finding out specific information that will be helpful to specific people.

• Gather the information you need: Ideally this step happens throughout the life of the project. Many of the project activities you’re already doing for other aspects of your project, such as interviews as part of community engagement, can equally contribute to project evaluation efforts. Information gathering techniques include:

  • Story gathering

  • Interviews

  • Focus groups or group conversations

  • Polls and surveys

  • Social media analytics

  • Mapping, including asset mapping and network mapping

• Organize, analyze, and synthesize findings: At the end of the project — or even during the project — you’ll want to step back and look at the information you have. What story is beginning to emerge? Is there a pattern to the types of impacts that you’re seeing? Are certain types of evidence more helpful than others? This can be one of the most exciting parts of the evaluation process — take the time to work through ideas as a larger project team.

While the project evaluation may be about impacts of this art-infused work on your community, it is also possible to bring the arts into the evaluation process. Ideas include:

• Holding a community dinner at the end of the project and giving attendees guiding questions to reflect on together.

• Facilitating a creative process for community members to express the changes they felt such as with storytelling, theater/improv exercises or visually.

• Train student youth to interview community members as a part of a short film project.

• Create a walk through the neighborhood that captures the changes people see (or that will happen in the future) and work with a graphic artist to record those in a “neighborhood map.”

At the beginning of any creative placemaking effort, the project team should address how evaluation will be incorporated alongside the artistic process. To avoid duplicating efforts, or asking too much of the community members, it’s essential to harness everyone’s efforts for multiple purposes. Here are some ideas on how to integrate evaluation throughout the artistic process:

• Record interviews done as part of the community-focused artistic process in order to have access to them later as part of your larger evaluation/reflection process.

• When insights emerge in the field, have artist(s) text or email them to the person in charge of capturing information. You’ll be able to search for these emails later to have a record of what your thoughts were in the moment, and you’ll help support a longer-term conversation about what is happening within the project.

• Hire and train local youth and community members to interview folks that participated in the project, asking what peoples’ impressions were and how it impacted them. You’ll learn about the impacts they discovered and hear their own insights into the project. Through their eyes, your team can also learn things they may have been unaware of before or about the scale of personal impact the project had on them.