February 15, 2007
HOW TO IMPROVE PUBLIC PERFORMANCE REPORTS
Major topic at conference chaired by CCAF

Ideas for improving public performance reports (PPRs) featured prominently at a conference on performance management that took place in Victoria, BC on January 16-17, 2007.

  • The conference, Adopting Best Practices in Public Sector Performance Management, was chaired by CCAF Executive Director Michael Eastman. CCAF organized a half-day session at the conference, and used the opportunity to highlight key findings from its recent research.
  • CCAF's underlying message was that public performance reporting, if it is to be of value, must focus more on the needs of report users, including legislators, the media and citizens.
  • To reinforce this message, the conference began with a panel of four users of PPRs:
    • Arn van Iersel, Acting Auditor General, Office of the Auditor General of BC;
    • Rob Fleming, MLA (Victoria-Hillside), Chair of the Select Standing Committee on Public Accounts, Province of BC;
    • Ken Stewart, Councillor, District of Maple Ridge, and former MLA; and
    • Paul Willcocks, Political Columnist, Victoria Times Colonist and Vancouver Sun.
  • They were followed by Rona Shaffran, a Director in the Office of the Auditor General of Canada who was on secondment in 2005 as Senior Research Associate to CCAF-FCVI. She spoke on Increasing the Demand for and Use of Public Performance Reporting.
  • Then Geoff Dubrow, CCAF's Director of Capacity Development, made a presentation on Improving Performance Reports to Better Address the Concerns of Users.
Providing assurance for PPRs

In his presentation to the conference, BC's Acting Auditor General Arn van Iersel noted that his office has provided assurance regarding the PPRs for a few BC agencies. Assurance refers to the external assessment of performance information and reporting by the agency, consistent with the reporting principles. Michael Macdonell, Director of Accountability Reporting in the Office of the Auditor General of BC, addressed the conference on the topic of Meeting User Needs for Relevance and Reliability. Some of his key points included these:

  • Internal and external groups use PPRs for different purposes and have different perspectives. Their needs for assurance may therefore differ, with external users likely requiring a higher level of assurance.
  • Assurance can be provided by several sources, including management, evaluators and academics, consultants, internal audit, and external audit (including the Office of the Auditor General).
  • Factors in determining an appropriate approach to assurance include the availability of other data sources, the strength of internal control systems, and the costs and benefits to users.
START WITH THE AUDIENCE

In introducing the panel, CCAF's Michael Eastman said public organizations fulfil their responsibility to account for their activities by reporting on their performance to elected representatives and to the public, including such intermediaries as interest groups and the media. Then elected representatives and the public can use that information to hold the government to account.

He said if organizations begin their performance management initiatives by thinking of the needs of these audiences – by considering what and to whom they want to report – then they can design their performance management processes with this goal in mind.

“If we think of public performance reporting as one critical output of our performance management activities,” he said, “we will be better able to ensure we focus those performance management activities on issues that are of interest and relevance to our governors and citizens.”

PANEL SAYS REPORTING MUST IMPROVE

BC's Acting Auditor General Arn van Iersel noted that BC and Alberta are recognized leaders in accountability reporting. However, he pointed out that the initial vision for enhanced accountability in BC, which had led to the emphasis on improved performance reporting, was almost a decade old. He said momentum may have reached a plateau. “We need to put more effort into this by following through on the initial vision and continuing to have BC take a leadership position in performance reporting and accountability,” he said.

Key issues include meeting legislators' needs, encouraging credible information linked to stated goals, and increasing public understanding of and engagement with organizational performance results.

Mr. van Iersel said his office would support improved reporting by developing examples of best practices, working to strengthen the use of reports by legislative committees, monitoring the application in BC of the Public Sector Accounting Board's new Statement of Recommended Practice regarding public performance reporting, and providing continued leadership on the assurance of annual reports.

BC MLA Rob Fleming talked about public performance reporting from the legislator's perspective. He said legislators are like “the canary in the coal mine” of public performance reporting – if they aren't using PPRs, the public likely isn't using them either. Despite some successes, he said research shows that MLAs are not in fact using the reports.

He noted several reasons for this:

  • Many MLAs don't value committee work, something that is unlikely to change.
  • The data in performance reports often lacks credibility.
  • Government operates in gray areas where it is difficult to identify good performance measures.

Mr. Fleming said the BC Public Accounts Committee (PAC), which he chairs, will likely revisit the issue of public performance reporting to see what it can do to help reinvigorate it. He said the committee will also work with the Auditor General to strengthen reporting; for example, the PAC expects to collaborate with the Auditor General and CCAF on the production of a retrospective study of public performance reporting in BC.

SIX QUESTIONS

Ken Stewart shared with the audience his “Six Questions for Accountability.” He developed the questions years ago when he was involved in a privatization initiative, and used them in holding Crown agencies to account in his time as chair of the Select Standing Committee on Crown Corporations in the last BC Legislature. The six questions, in essence, reflect BC's reporting principles.

The questions are:

  1. Tell us what you do.
  2. Tell why you do it.
  3. Who told you to do it?
  4. Show us that you are doing what you say.
  5. Prove it.
  6. What are your future plans ?

Mr. Stewart said the questions could provide a framework for good performance reporting.

Political columnist Paul Willcocks said most journalists don't use public performance reports - in part because they are as prone to laziness as anyone else, but particularly because most such reports are not useful.

He said a few organizations produce good reports (according to Mr. Willcocks, the BC Lottery Corporation does an excellent job of saying what they are trying to do, how they are doing, and why). But overall, the situation is “not good.”

One reason is that managers, especially in government, want “wiggle room” when it comes to setting performance targets, and they tend to set targets that are vague. For example, one BC Ministry set out to achieve “a reduction” in a particular indicator, then declared success when it achieved only a tiny reduction. Such information is of little use to journalists.

Mr. Willcocks said public performance reporting in BC has actually weakened over time. One ministry, for example, went from reporting on 35 measures in 2002 to 9 measures now – while tripling the size of its report. He urged MLAs to push for better reporting, saying it is in the interest of both the governing party and the Opposition to press for answers on performance from government.

A vigorous use of performance information

Consultant Heather Daynard of Prospect Management Enterprises described to the conference how the government of Washington State is putting performance information to work to improve the government and connect with citizens.

Since 2002, the state government has used a biennial process called Priorities of Government to identify key results that citizens expect from government. For each results area, planning teams work together to answer these questions:

  • What are the key indicators of success? How would citizens know if we are making progress toward the high-level results?
  • What would you identify as the proven (or most promising) strategies for achieving this result? What does our experience and research tell us about the factors most critical to success?
  • What changes in practice or costs do we need to make to maximize the results we deliver to citizens?
  • How will we measure our progress?

Each team then develops a list of recommended government activities, focused on maximizing results through evidenced-based strategies. Overall, the process responds to the question: How can Washington State government best spend taxpayer dollars to achieve the results that matter most to citizens? The result of this process is used to inform the government's biennial budget process, and is made public.

This approach is interesting for its emphasis on using performance information to communicate results to citizens - a key message in CCAF's public performance reporting activities. More information is available on the web site of Washington State: http://www.ofm.wa.gov/budget/pog/pdf/pogfinalreport2006.pdf.
If journalists are to use PPRs, Mr. Willcocks said, the reports need to be better. They need to present consistent information. They should be certified regularly by an outside source, such as the Auditor General. And they should be reviewed by legislative committees.

THE AVERAGE CITIZEN WANTS RELEVANT INFORMATION

Rona Shaffran's presentation explored the results of CCAF's 2006 research report Users and Uses: Towards Producing and Using Better Public Performance Reporting: Perspectives and Solutions.

The study found that the intended users of PPRs – legislators, media and the public – make little use of the reports. Although these audiences are interested in how government is performing, PPRs:

  • are not written from their perspective;
  • do not reflect their specific needs;
  • contain information that is generally not relevant; and
  • are in a language and format that is not easily understandable.

Ms. Shaffran said the average citizen makes limited use of PPRs, but will use government performance information when it is relevant to his or her life or important decisions. She said academics, researchers and public interest groups make some use of PPRs. Non-governmental organizations (NGOs) may use PPRs for:

  • policy development and advocacy;
  • tracking government follow-up;
  • assessing government;
  • marketing research; and
  • monitoring the use of programs.

“The time is ripe for governments to report measures of performance that matter to ordinary people,” she said. By considering the public's view on reporting, governments may obtain new perspectives.

Ms. Shaffran said legislators, media and the public have different needs for information on government performance than does government. The producers of performance reports should ask users what they need, and then report in ways that “speak” to users.

CCAF'S AGENDA FOR ACTION

Geoff Dubrow, CCAF's Director of Capacity Development, said he was encouraged by the suggestions of the panelists for moving public performance reporting forward in BC. He said moving forward would likely require a greater focus by governments on the needs of users of performance reports. At the same time, users should be more active in creating demand for relevant performance information, and in visibly using information in PPRs to scrutinize government.

Mr. Dubrow said CCAF is acting on a number of fronts to help realign the “supply” of public performance reports with the “demand” from users.

For example, CCAF is encouraging PACs and other legislative committees, which are users of PPRs, to:

  • look at the status of public performance reporting and make recommendations to government on next steps;
  • provide input to government on the content and readability of reports; and
  • provide feedback to government on the needs of other users.

With the media, CCAF plans to:

  • conduct focus groups to assess the needs of media as users of PPRs;
  • research and document best practices in reporting to media;
  • assess media usability of select PPRs and provide feedback to government; and
  • approach key journalists with PPRs, note their response regarding usability, and report back to government.

CCAF also intends to conduct focus groups to assess the needs of the public and NGOs as users of PPRs, and research and document best practices in reporting to the public. In conjunction with schools of public policy/administration, CCAF will have students (future NGO leaders) assess the usability of sectoral reports in several jurisdictions.

Finally, CCAF will facilitate government decisions about how to improve public performance reporting by apprising governments of best practices in Canada and abroad, providing them with user feedback, and presenting them with opportunities for low-risk communication with users.

By documenting each of these initiatives, CCAF hopes to provide both the producers and users of public performance reports with solid ideas for making PPRs a valued tool in the accountability process.

Return To Top of Page

Copyright © 2007 CCAF-FCVI
Privacy Policy