The Short Answer

1. Start with ends, work backward to means. What do we want? How will we recognize it? What will it take to get there?
2. Be clear and disciplined about language.
3. Use plain language, not exclusionary jargon.
   • Keep accountability for populations separate from accountability for programs and agencies.
   • Results are end conditions of well-being for populations in a geographic area: children, adults, families and communities. They are the responsibility of partnerships
4. Customer or client results are end conditions of well-being for customers of a program, agency or service system. They are the responsibility of the managers of the program or agency.
5. Use data (indicators and performance measures) to gauge success or failure against a baseline.
6. Use data to drive a disciplined business-like decision making process to get better.
7. Involve a broad set of partners.
8. Get from talk to action as quickly as possible.

Full Answer

 

1. Results-Based Accountability starts with a crucial distinction: between results for whole populations like all children, all elders, all citizens in a geographic area – and results for the customers or clients of a particular program, agency or service system. The most important reason for this distinction is the difference in “who is accountable.” Performance accountability can be assigned to the managers who run the various programs, agencies or service systems. Population accountability can not be assigned to any one individual, organization or level of government. The whole community, public and private sectors, must share responsibility for population results.

2. Results-Based Accountability is made up of two parts: Population Accountability and Performance Accountability. Results-Based Accountability is the overarching idea which includes results-based decision making and results-based budgeting. Accountability is by someone to someone for something important.

Population accountability is accountability BY THE COMMUNITY (city, county or state) TO THE COMMUNITY (city, county or state) FOR THE WELL-BEING (RESULTS) OF A POPULATION (children, adults, families, all citizens…). Performance accountability is accountability BY THE MANAGERS (of the program, agency or service system) TO THE CUSTOMERS (and other stakeholders) FOR THE PERFORMANCE OF THE PROGRAM (agency or service system).

Results-Based Accountability is a different way of thinking. It organizes the work of programs, agencies, communities, cities, counties and states around the end conditions we seek for those who live in our community and those receiving service. It uses those end conditions as the grounding for all of the work, including decision making and budgeting.

Results-based decision-making uses results (the desired conditions of well-being) as the starting point for making decisions. It is a business-like process that starts with ends and works backwards to means. It first defines success in measurable terms and uses those measures to gauge success or failure.

Results-based Budgeting is a particular kind of decision making, using results to steer the development of budgets. This includes budgets for population results which span across departments and beyond the formal boundaries of the agencies governed by the budget process itself.  And it also includes performance budgets which use client results to drive budgets for the programs, agencies and service systems of the city county state or nation.

3. The Language of accountability: The most common problem in this work is the problem of language. People come to the table from many different disciplines and many different walks of life. And the way in which we talk about programs and services for children and families varies, literally, all over the map. This means that the usual state of affairs in planning for children and families is a Tower of Babel, where no one really knows what the other person is saying, but everyone politely pretends that they do. As a consequence, the work is slow, frustrating and often ineffective.

It is possible to exercise language discipline in this work. And the way to do this is to agree on a set of definitions that start with ideas and not words. Words are just labels for ideas. And the same idea can have many different labels. The following four ideas are the basis for definitions used at the beginning of this work. Alternative labels are offered:

Results (or outcomes or goals) are conditions of well-being for children, adults, families or communities, stated in plain English (or plain Spanish or plain Korean…). They are things that voters and taxpayers can understand. They are not about programs or agencies or government jargon. Results include: “healthy children, children ready for school, children succeeding in school, children staying out of trouble, strong families, safe communities.”Indicators (or benchmarks) are measures which help quantify the achievement of a result. They answer the question “How would we recognize these results in measurable terms if we fell over them?” So, for example, the rate of low-birthweight babies helps quantify whether we’re getting healthy births or not. Third grade reading scores help quantify whether children are succeeding in school today, and whether they were ready for school three years ago. The crime rate helps quantify whether we are living in safe communities, etc.Strategies are coherent collections of actions which have a reasoned chance of improving results. Strategies are made up of our best thinking about what works, and include the contributions of many partners. No single action by any one agency can create the improved results we want and need. (Note: the idea of “reasoned chance” is about the theory of change which underlies the strategy.) Performance Measures are measures of how well public and private programs and agencies are working. The most important performance measures tell us whether the clients or customers of the service are better off. We sometimes refer to these measures as client or customer results (to distinguish them from cross-community results for all children and families)

The principal distinction here is between ends and means. Results and indicators are about the ends we want for children and families. And strategies and performance measures are about the means to get there. Processes that fail to make these crucial distinctions often mix up ends and means. And such processes tend to get mired in the all-talk-no-action circles that have disillusioned countless participants in past efforts

For a discussion of these concepts applied to all children 0 to 5 healthy and ready for school, see “Results-Based Accountability for Proposition 10 Commissioners.”Click here to go to the UCLA site to read this paper.

It is possible to use the definitions above to “diagnose” the language usage of a group working on Results-Based Accountability. Two tools are included in this guide to help you do this. First is a diagnosis technique which provides a structure to analyze language usage and identify duplicative and inconsistent use of  language. Second is a crosswalk tool which allows any planning process to be crosswalked to the Results-Based Accountability framework in this guide. 4. The Results-Based Accountability Thinking Process Results-Based Accountability involves a very simple, but business-like, thinking process that can help direct the work of creating a strategy for improving the well-being of a population or subpopulation in a geographic area.  It is a simple set of notions: “What do we want for this population in plain language?” “How would we recognize it in measurable terms?” “What will it take to get there?”

Click here to see this process displayed in graphic form.  In the following paragraphs we will take a quick pass at the basic ideas in each step of the thinking process. And then in later sections, we will go back and explore how to do each step in more detail.

WHAT DO WE WANT?

Population: Results-Based Accountability starts with a whole population. A population might include all children in a county or state, or all elders, or all elders with a disability. Or it might include all the residents, and address the quality of life for all. The thinking process is the same whether the population in question is children, elders or whales. Results: Results are conditions of well-being we would like to say exist for this population. These conditions are stated in plain English (plain Spanish, plain Vietnamese, plain Farsi…) not bureaucratic jargon. Results include such statements as  “all children ready for school, all children safe, all children succeeding in school, strong and self sufficient families.”  They can also include such statements as: “our community with affordable housing for everyone,” “our community with an adequate sustainable water supply,” “our community without graffitti,” “forrests not prone to devastating fire,” “a prosperous economy,” “a clean environment.”  These are statements that the public can understand, that can be used to anchor and communicate the basic purpose of the work. In fact, a result is ANY plain language statement of a condition of well-being for children, adults, families and communities.

HOW WOULD WE RECOGNIZE IT?

The next two questions have to do with how we would recognize these conditions if we fell over them, first in terms of experience and then in terms of data. Experience: How would we recognize these results in our day to day lives in the community? What would we see, hear, feel, observe? If the results is children healthy and ready for school, we would see children playing outside. We would hear young children with good communication skills. We would feel that children were respected and loved in our community. Indicators: How would we recognize these conditions in measurable terms. Here we are looking for pieces of data that tell us whether these conditions exist or not. If the condition is child health, we might look at the rate of low birth-weight babies, or the rate of emergency room accidents. If the result is “children succeeding in school” we might look at the percent of children reading at grade level or graduating from high school. Some of this data we currently have and can use today in the planning process. Other data, which we would like to have, becomes part of our data development agenda. Baselines: For each indicator, we present a picture of where we’ve been and where we’re headed if we stay on our current course. These pictures are called baselines. They allow us to define success as doing better than the baseline.

WHAT WILL IT TAKE TO GET THERE?

Story behind the baselines: Why do these baseline pictures look the way they do? What are the causes and forces at work? This is the epidemiology part of the work. Digging behind the pictures helps us get a handle on what’s going on in our community and what might work to do better. As we do this work we bump up against things we wish we knew more about. This becomes part of our information/research agenda. We’ll gather this information as best we can between meetings.

Partners: Who are the potential partners (people and agencies, public and private) who have a role to play in doing better? What works: What do we think would work? What would it take to do better than the baselines in this community? What has worked in other places outside our community? What does the research tell us? Just importantly, what does our own personal experience tell us about what would work here? The answers should draw on the possible contributions of partners; and should involve no-cost and low-cost ideas. Criteria: If we come up with a long list of things that might work, how do we choose what to actually do? What criteria should guide this selection process? Some criteria to think about include: specificity (Is the idea about specific action not rhetoric?), leverage (Will it make a big or little difference?), values (Is it consistent with our personal and community values?), and reach (Is it feasible to do it this year, next year or 3 to 10 years). Strategy, action plan and budget: What do we propose to actually do? This should take the form of a multi-year strategy and action plan laying out what is to be accomplished by when.  We can then assign responsibilities and get started. Once you decide on things to be done, projects, programs, no-cost and low-cost actions etc. you can use performance measures to track their progress. This does not have to take forever. You can take a pass at this thinking process in an hour or so. And then go through it again each time you get together. Every time you iterate this process, your action plan gets better. 5. Performance Accountability Thinking Process: a. Be clear about what program or agency is being measured. The first order of business in picking the right performance measures is being clear about what program or agency is being measured. This is a “fence drawing” problem. First we draw a fence around the thing to be measured. It could be a program, like child care center , or a component of a program with some organizational identity, like infant child care. Or it could be an entire organization or agency, like a residential treatment center, or a department of social services. Or it could be an entire service system, like the entire child welfare or child care service system, involving many agencies and their programs. Next we ask ourselves a few questions about what’s inside the fence. Who are our customers? Customers include the direct recipients or beneficiaries of the service. But they also include others who depend on the program’s performance, like related programs and partners. For example, the customers of child care program include the children of the program, but also the parents of those children, and also the local elementary school where many of these children will enter kindergarten. It is important to consider the full range of customers, because, just like in business, success depends on doing a good job for your customers. b. Consider the four types of performance measures and choose the most important.

All performance measures fit into one of four categories, derived from the intersection of quantity and quality vs. effort and effect.

	QUANTITY	QUALITY
EFFORT	How much did we do? How much service did we deliver?	How well did we do it? How well did we deliver service?
EFFECT	Is anyone better off (#)? How much change for the better did we produce	Is anyone better off (%)? What quality of change for the better did we produce?

Not all performance measures are of equal importance. There are two general classes of performance measures that are most important: those that tell whether the service and its related functions are done well (upper right quadrant). These measures include such things as timeliness of service, accessibility, cultural competence, turnover rate and morale of staff. These measures are used by managers to steer the administration of the program. If things are late, they work to make them timely. If turnover is high, they work to retain staff. The most important measures tell us whether our clients or customers are better off as a consequence of receiving the service (lower right quadrant). We call these measures “client or customer results” These are measures which gauge the effect of the service on peoples lives. Usually, in programs which directly deliver services to people client results have to do with four dimensions of “better-offness.” Skills, attitude, behavior and circumstance. Did their skills improve; did their attitude change for the better, did their behavior change for the better, is their life circumstance improved in some demonstrable way? So, for example, if you are overseeing a child care program, you would want to measure such things as the percent of children with basic literacy skills (skills), the percent of children with a positive self image (attitude); the percent of children exhibiting disruptive behavior (behavior) and the percent of children who are up to date on their immunizations, and the percent who go on to succeed in 1st grade (circumstance). c. Don’t accept lack of control as an excuse. Now the first thing you’re going to say is “Wait a minute. What does child care have to do with whether or not children are up to date on immunizations? This is a good example of a performance measure where child care has very little control over whether the circumstance improves. Child care can make a contribution to the immunization status of its clients. Quality child care can help parents and children understand the importance of regular preventive health care and can help parents understand and access the health care system. But child care by itself can not do these things. So isn’t it unfair to track immunization rates for children in care? If you look at the other measures listed for child care (literacy skills, self image, disruptive behavior, first grade success) you will notice that these measures are also beyond the capacity of the child care provider to completely control. And the point is that all programs performance measures are affected by many factors beyond the particular program’s control. This lack of control is usually used as an excuse for not doing performance measurement at all. Turnover rate, staff morale, you name it is “beyond my control”. In fact, the more important the performance measure (e.g. children successful in 1st grade) the less control the program has over it. This is a paradox at the heart of doing performance measurement well. If control were the overriding criteria for performance measures then there would be noperformance measures at all. The first thing that we must do in performance measurement is get past the control excuse, and acknowledge that we must use measures we do not completely control. d. Create a performance accountability system useful to managers. – one that takes this control paradox into account. We do this in three ways. First, we ask managers to assess their performance on these measures – not on the basis of some absolute standard – or on how other providers are doing – but on whether they are doing better than their own history. We do this using the same technique used for cross community indicators: the notion of baseline. For each performance measure we ask managers to present a baseline of the history of their program’s performance, and where their performance is headed. We ask them to do better than their own baseline. This is the central way in which businesses use data. How are we doing compared to our own history. Later when you have the sophistication and the data, you can begin to develop and use comparisons to the performance of other similar providers with similar mixes of easy and hard cases. And later still, we can compare to standards, when we know what good performance looks like. In some services, like child care, we have progressed to the point where we have standards for the first type of performance measure above. In child care we know what quality service delivery looks like. We have standards for staffing ratios, percent of staff with certain qualifications, timeliness of service, safety etc. Next we ask managers to think about the partners who have a role to play in doing better. Programs cannot produce the most important results for customers by themselves. And, finally managers must ask and answer: What works to improve performance?” Out of this thinking we ask managers to present their best thinking about what needs to be done. This thinking process is summarized in the Seven Questions Central to Performance Accountability. These questions should be asked and answered at every intersection between a supervisor and a subordinate throughout the system.