How Do We Evaluate the Impact of What We Do?

During meetings with our affiliated kehillot, many of our staff and lay leaders have been asked the uncomfortable question, “What has United Synagogue done that has made a measurable positive impact on congregations?”

We can easily give anecdotes about successful board workshops, education consultations or USY chapter events. Our Sulam for Presidents program garnered enthusiastic reviews for over 20 years. But these are examples of outcomes, in the language of the Kellogg Foundation’s Logic Model.

Outcomes tell about what changed in the participants of a program. For example, if a president at Sulam for Presidents learns about creating yearly board goals, one outcome might be that she arranges for her kehilla relationship manager to facilitate Session 1 of Sulam for Current Leaders to help the board create them.

United Synagogue is being asked, however, about impacts. Impacts are what happen – intentionally or not – to a kehilla over time as a result of program outcomes.

Here is an example of small impacts we might see in a kehilla if the president instituted the practice of creating board goals:

In the Sulam for Current Leaders workshop, the board sets a goal to increase their connections with the members of the community, beginning with one modest and achievable objective: that each board member will have Shabbat dinner with one other member family during the year. In the first year, an outcome might be that some board members will host those Shabbat dinners. By the end of the year, board members might enjoy hosting the dinners so much that they ask more than one family, engaging more of the community.  A small impact might be that those positive connections will bring new people into volunteer positions.

Kathy Elias, Chief Kehilla Officer

There might be additional outcomes and impacts, however.  A key concept from Sulam is that you need to check on progress towards goals, so the president might add a short agenda item to board meetings.  She institutes a “Kehilla Connection” moment, where board members are asked to describe one positive thing they heard that month at the Shabbat dinners.

As the “Kehilla Connection” moments at board meetings are documented, the board begins to build a library of testimonials about what is special and vibrant in its community.  In the strategic planning realm, this is similar to a consultants’ tool of “appreciative inquiry” — where people are asked to tell stories of what is going well in order to gather information about the strengths of a community.

Two years later, the next president and board might set a new goal of strategic planning. The stories reported during the “Kehilla Connection” portion of board meetings might provide the language describing what their kehilla does best, and will be used by the committee to create a vision and mission statement. There may even be new volunteers identified during the Shabbat dinners who will work on the committee.

In this scenario, what begins as a Sulam lesson about setting goals has impacts on membership engagement, volunteer recruitment, and in preparation for strategic planning.

The question we are asked about the impacts of our work is not answered by describing what United Synagogue is doing, or what people learn at our programs. It is answered by telling the story of what changes in our kehillot as a result. Where do we start? The first step requires that we gather baseline information about our kehillot.  Then we can step back and look for patterns of changes over time in each kehilla and in the aggregate.

This year, we began that process in a few significant ways:

  • Our kehilla strengthening program staff created a Leadership Succession Survey that gives us information about how leaders are recruited and trained. 143 kehilllot have completed this so far, and the survey will allow us to track how kehillot make improvements that let them answer “yes” to the question, “Do you know who your next president will be?”
  • An online survey about operational issues and needs was completed by nearly 40% of our kehillot this year.
  • All 52 graduates of our Sulam for Presidents program created a “90-day action plan” of changes they would make towards goals during their presidencies. The action plans were shared with their KRMs and give us a basis for follow up.
  • A Board Leadership Self Assessment in our Sulam for Current Leaders curriculum gives us a starting place for identifying strengths and weaknesses in governance. It is a requirement for coaching calls with our leadership development team and one of the intake documents for our Sulam for Strategic Planning, which will launch in the fall.
  • A Kehilla Strengthening Action Plan (KSP) allows our KRMs to record the needs and goals of kehilla leaders, and make a one-year plan for how they can maximize the impact of the services available from United Synagogue. One of our priorities for next year is for KRMs to create a KSAP for every kehilla.
  • Our Sulam for Emerging Leaders program is being formally evaluated by Dr. Steven Cohen and Dr. Ezra Kopelowicz. Baseline information about the 12 pilot kehillot and their participants has been collected and a report of the initial outcomes of the program will be available at our December board meeting. Data collection will continue with the 30 kehillot (and up to 400 participants) that will form the next cohort, and we will watch for impacts as these new leaders take their places at board tables in the next five years.

Join the Conversation

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: