2.10 How do we create a report card and what do we do with it?

The Short Answer

1. Gain organizational and political sponsorship, necessary to produce the document and give it standing in the decision making process.

2. Identify results and indicators, using a broad process to involve partners, and grounded in a conceptually clear framework. See 2.5 and 2.6, and 2.7.

3. Gather the data, starting with the best available and pursuing your data development agenda. Use your political sponsorship for cooperation and access. See 2.8

4. Analyze and present the data in simple and compelling ways. Focus on analyses with direct policy implications. Keep presentations short and visually interesting.

5. Decide on recommendations. This can be published as part of the document or separately.

Full Answer

There is of course a whole paper written about this if you’d like to read it. Click here to go to the Finance Project website to read “A Guide to Developing and Using Family and Children’s Budgets.”

What follows is a summary of some of the key steps and some new learnings about how to do this work.

(1) Gain sponsorship: Producing a Family and Children’s budget is a major undertaking, and needs the sponsorship and support. Two kinds of sponsorship are important: organizational and political. Organizational sponsorship follows from political sponsorship.

Possible political sponsorship is complicated, but it boils down to these options: Inside or outside government. Inside government in the executive or legislative branches; with majority minority or even individual elected official support. Outside government by coalition or collaborative or by a single organization usually an advocacy organization.

Organizational sponsorship or support (meaning production wherewithal) can be lodged in the executive branch or legislative branch or private sector. In the executive branch it can be housed in the budget office, a major department, or, perhaps best, a collaborative or children’s cabinet. In the legislative branch it is most likely to be housed in the staff arm or budget arm of the legislative body. In the private sector it can be housed in a single advocacy organization or a collaborative or council.

The purposes of sponsorship are twofold: First, organization support and financing is necessary to support the work and produce the products. But leverage, visibility, and utility may be even more important. Children’s report cards, like children’s budgets are political documents, and who sponsors them says a lot about how they will be seen and used. The obvious message here is to go fo the highest level sponsorship possible, preferably joint executive legislative authorization and support. Before this is possible the report card may need to go through its development in the private sector.

There is an argument to be made that the report card should be developed by and remain in the private sector, as a means of providing an outside, “objective” view. This is in fact the way most report cards have been produced. The Annie E. Casey Foundation Kids Count reports are mostly produced by private non-profit or university based organizations. The advantage here is flexibility. The possible disadvantage is more limited access to data and less powerful standing in the political and budget processes. United Ways are playing an increasingly important role in the development of report cards. These organizations often represent at least partially neutral ground, and have the organizational resources to pull it off. Foundation or other private sector funding is often necessary.

(2) Identify results and indicators: Once the project is authorized and funded, the fun begins. Many report card efforts of the past have skipped “results” gone directly to the businesses of selecting indicators, and did this mostly on the basis of data power (i.e. what was available). So you end up with a report card that is the best of available data. The problem with this is that the indicators are often a disembodies set of data with no anchor to the notion of well-being for the population being reported on. “Why these indicators?” one might reasonably ask. And the answer has more to do with the quality of data than the purpose of the data. The purpose of indicators is to help quantify results, conditions of well-being. Results themselves can help us be more specific about what we mean by well-being. So using results to select indicators produces a logic chain: “We’re concerned about the well-being of children in our community. Here’s what we mean by well-being in plain language (safe, healthy ready for school etc.). And here’s the data we have available which can best tell us how we’re doing on these conditions.”

The process of identifying results and indicators is described in other parts of this guide. See 2.5 and 2.6, and 2.7.

(3) Gather data: See 2.8 Where do we get the data for indicators? How do we get better data?

(4) Analyze and present the data: These are related tasks. The best presentations allow the reader to easily and visually reach their own conclusions. There should be two parts to the analysis and presentation. First is the basic data for each indicator. There are many formats to choose from. The best are compound pictures, that is, pictures with more than one data element. The National Casey Foundation Kids Count report provides one of the best, the principle display using bars to show the amount of improvement or decline compared to national averages. This is supported on the same page by the actual value of the indicator and the state’s ranking. One weakness in the Kid’s Count format is that the comparison involves just two points in time, usually ten years apart. This is a difficult baseline to interpret because it tells nothing about what happened in between or about recent trends. An arguably more powerful, but also more complex presentation, would show multi-year baselines with as much history as possible and forecasts of where we are heading if we stay on our current course. See 2.11 How do we create a baseline (trend line) for an indicator? There have been a great number of excellent family and children’s report cards produced in the last 10 years. Click here for a list of websites with click-on links.

Analyzing the data offers many options. This is the place where it is possible to tell the story behind the baselines, to describe causes and forces at work. Why does the data look the way it does? This is important because, just like in epidemiology, each cause is a pointer to potential action. The analysis should also include comparisons. First off are self comparisons, how much have things gotten better or worse; and secondly, comparisons to others, including national and state averages and trends. Comparisons can also be made to other comparable jurisdictions. And finally to standards if they exist and targets or goals if they have been set.

A word on setting goals: If you set the goal without regard to time (“Our goal is 95% high school graduation.”) then you can set the goal anywhere you like. But if your goal has a time component (“Our goal is 95% high school graduation by next year.”), make sure the goals are ambitious but reasonable in light of the baseline. Unrealistic goals undermine the credibility of the effort.

(5) Decide on recommendations: Report cards provide a platform and opportunity to influence decisions about children families and communities. The press coverage of report cards is sometime the most concentrated attention these issues get in the press. If the report card is produced by government, then it is not likely to allow much in the way of specific recommendations. In this case private advocacy organizations can and should issue a companion analysis offering specific policy recommendations.

(6) Produce and distribute the document: Paper and electronic distribution, as widely as resources permit.

(7) Let’s back up a step. Why do a Children’s Report Card or a report card on the well-being of any population? Here are two images which might help.

Families and Children Inc. :Imagine that your collaborative is a corporation, and the group that meets regularly is the board of directors. But the bottom line for your enterprise is not profit, it is the well-being of children and families as measured on the various indicator baselines. As a corporation, you would need information on a regular basis, just as a corporation does, on how well you are doing. Periodic report cards provide this information in a systematic and disciplined way, not just for your board, but for the many stakeholders (stockholders) in your community.

Consoles: You are the chief engineer at a nuclear power plant. In front of you is a large console with a thousand dials and gauges on it. If the plant is to be operated safely, you need to pay close attention to a handful of these gauges. The gauges that are most important are larger than the ones that are less important. The temperature guage in the main reactor is the biggest gauge. These large gauges are your indicators, and you run this literally life and death operation on the basis of that information. Those groups who come together to take responsibility for the well-being of children and families (or any other population) are faced with a system far more complex than a nuclear reactor. We too have many thousands of possible gauges to consider. When we select the most important measures to go into a report card we are selecting those that deserve to be larger than the others. And this report card can help us steer the enterprise of child and family well-being just as the plant manager steers the reactor and its many systems. So the report card is a console which can be used to steer the complex system (public and private, for profit and non-profit, paid professional and parent volunteer) supporting the well-being of children and families.

Report cards can help us stay on course when we’ve made progress. Take the example of Tillamook County where teen pregnancy rates rebounded after years of progress. Report cards can help us keep our eye on what is most important even when the numbers look good (or better) and take action to sustain progress.

Finally, one of the best arguments for a children’s report card is the picture of its absence. If we have no way to steer, we are left to literally wander. And this is much of the history of social policy. Throw money at problems and hope for the best. The early navigators made progress by charting their path against the stars. Results and indicators and the star charts of report cards (and budgets) are the way we can navigate. No one has yet created a better way. (Stay tuned for the Social Policy Global Positioning System.)

Marc2.10 How do we create a report card and what do we do with it?