The Short Answer

(1)    Population well-being is clearly beyond the responsibility of any one organization or any one level of government. It is beyond government itself. It requires a whole range of public and private partners. 

(2)    Programs and agencies serve client and customer groups that are (almost always) less than the total population.  It is possible to identify agency and program managers who can be held responsible.

(3) A given piece of data like “high school graduation rate” can play two different ROLES: the role of indicator and the role of performance measure.  This is like an actor playing two different parts;. same actor, but one day Hamlet and the next day Lear. So graduation rate is one day a performance measure for the school system and the next day an indicator of all children succeeding in school. (This does not mean that performance measures are sometimes indicators, anymore than it means that Hamlet is sometimes Lear.)

Full Answer

(1)    Population well-being is clearly beyond the responsibility of any one organization or any one level of government. It is beyond government itself. It requires a whole range of public and private partners. People can see this clearly when an example is used. 

TECHNIQUE: Help a group see how population accountability is beyond any one agency, organization or government itself: Ask the group to name the potential partners who have a role to play in “All Children Being Healthy.” (Or choose another result about children and families or for the entire population like “Clean Environment” or “Prosperous Economy.”) Ask the group if any one of these partners can or should be held responsible for the result. Note that, in the case of “healthy children,”it is common for this responsibility to be pinned on just one department, the Health Department. Now that we have a fuller list of partners we can see that this is not right. The Health Department should, perhaps, take the lead in convening the partners (around a table), and organizing the process, but can not assume full responsibility. 

(2)    Programs and agencies serve client and customer groups that are (almost always) less than the total population.  Itis possible to identify agency and program managers who can be held formally responsible for the performance of that program or agency.   

Chart:
Advanced View of the Relationship between Indicators and Performance Measures

(3) Advanced view of the relationship between indicators and performance measures: When a program or agency is small, it is not hard to distinquish the client population from the total population. But there are some times when a program (or agency’s) clients are close to or the same as the total population. As a program’s client population approaches the total population (of the state, county, city or community),  program performance measures begin to play a double role. First they are measures of how well the program is performing. And they can be used as indicators, proxies for the well-being of the whole population.

EXAMPLE 1: This most often happens in education and public health. The school superintendent and her senior staff go off on a retreat. The group discusses performance measures for the school district as an organizational entity that they are responsible for running. One of the most important performance measures is “high school graduation rate.”The next day the superintendent goes down the street to the monthly meeting of the family and children’s collaborative. Here, one of the population results the group has established is “all children succeeding in school,” and one of the indicators is “high school graduation rate.” The data for the graduation rate is playing two different roles, first as a management performance measure, and second as a proxy for the well-being of all school age children.

Note that the data element “graduation rate” is playing two different ROLES. This is like an actor playing two different parts;. same actor, but one day Hamlet and the next day Lear. So graduation rate is one day a performance measure and the next day an indicator. This not mean that performance measures are sometimes indicators, anymore than it means that Hamlet is sometimes Lear. The constant is the data. The roles are different.

EXAMPLE 2: The Public Health Department is operating a campaign to improve childhood immunizations. The campaign can be seen as a program to be managed. And as such it has performance measures. Such performance measures might include the unit cost of the vaccines or the accessibility of the service as measured by percentage of staff hours in mobile vans vs. clinics (upper right quadrant measures).  Client or customer well-being can be measured (in the lower quadrants) by the number of children immunized and the percent this represents of the total population. But it can also be measured by the percentage of the total population that is immunized or the childhood disease rate for immunizable illnesses. These are total population measures which usually serve as indicators. Here they are also used in the role of performance measure for the initiative.

(4) This double role of data helps explain why these ideas have been so mixed up together over the years. It has been quite common in past (and unfortunately many current) performance measurement efforts to hold agencies responsible for indicators. If it’s “safe community” then it must be the police department. If it’s “healthy children,” then it must be the health department. And so forth. This has lead to considerable, and well justified, cynicism about performance measures. Because the heads of these agencies can easily see that they are one of many players who must work together to do better. And yet the performance accountablity system pins it on them alone. One way to deal with this in agency presentations (to the public or the legislative branch) is as follows:

Make sure every presentation has two parts: Part 1 displays the community-wide results and indicators the agency is trying to impact as part of a broad partnership. Part 2 displays the specific performance measures for the agency and it’s component parts. 

EXAMPLE 3: The Health Department Director’s budget testimony: “We are here today to present the budget of the Healthy Department for the next fiscal year. On page one you can see at the top of the page the most important indicators of the health of our citizens in this (state, county, city). The Health Department is part of a health coalition addressing these indicators. This coalition includes the hospitals, doctors, nurses, managed care organizations, as well as schools, teachers and parents. Here’s what we as a coalition will be doing in the next year to turn the curve on the indicators you see before you. And here is the Health Department’s role in that effort. Some of these actions require your approval in the budget for the Department. The actions of other partners are contributed and paid for by those partners. On the lower portion of the page you can see a summary of the Department and its component divisions. Presented for each are 2 or 3 of the most important performance measures. These tell you whether that particular part of the organization is working well. We use these measures, and many others, to manage the department and work to provide the very best possible service. When we get to the budget for each division, we will show you the baselines for each of these measures, and what we propose to do to improve performance.”

The very structure of this presentation separates population accountability from agency and program performance accountability. This two part structure can be used in everything from press releases to the annual report. It helps keeps the department’s role clear. And it helps policy makers see that if they really want to make progress on population indicators (like immunization rates, high school graduation rates, juvenile crime rates, poverty rates, etc.) it will take the actions of many partners, and significant, not token, investment. 

(5)Another place where the boundary between performance measures and indicators is important is the discussion of service “systems.” Service systems involve many different agencies and service providers. The important thing to remember here is that these systems provide service and have clients or customers. In other words it is possible to distinguish people in the system receiving service from those outside the system not receiving service. This means that the measures for the service system performance are performance measures. So for example, take the entire child welfare system, the rate of repeat child abuse and child neglect (that is children who come back through the system a second time) is a performance measure for the system. This is distinct from the actual population rate of child abuse and repeat child abuse, which in theory can only be gathered from population surveys and studies. Again, services system performance measurement data will often play a double role as indicators.

(6) Prevention programs: prevention programs by their nature attempt to influence the behavior or condition of an entire population before they have need to enter the formal service system. In effect their client population is an entire population. In this case client results and population results are the same thing. So prevention programs must be judged on measures which are most often used only as indicators. It is also possible to measure the effects of prevention on the much smaller group of those people “contacted” by the prevention program (e.g. those children who attended a traveling theater production on violence prevention.) This kind of measure then cleanly follows the ruled for performance measures. Prevention programs should have both kinds of measures.

(7) It would be useful to help government leaders understand the different choices that go with these two kinds of work.  Most states have some version of performance measurement under way – efforts of various quality. Relatively few have population results efforts with an active process to engage communities.

(8) Is any framework that uses the word A result @ equally worth consideration by decision makers.  This is what many other publications on results have done.  Maybe it = s a good way to A position @ the organization sponsoring the publication, but there is a risk that the work will lose its conceptual center and point of view. It is necessary to be more explicit about the different kinds of results-based work. Rather than gloss over these differences, take them head on. For example:

A Results based decision making includes many different types of efforts that share in common the plain language articulation of desired conditions of well-being for agency customers or population groups, the business-like use of data to drive decision making, and the development of partnerships to steer the work and contribute to its success. Not all efforts which use the word A result @ in fact have these characteristics. The two most important types of results-based decision making efforts are (1) those which focus inside government or private agencies to improve the performance of services provided to customers of those agencies, and (2) those more broadly based efforts which address the well-being of whole populations, like the work in many states to make A all children healthy and ready to learn. @