By Jon Christensen
Illustration ©Michael Gibbs
When The Nature Conservancy of California asked Silicon Valley venture capitalist Seth Neiman for a multimillion-dollar contribution to help protect local open space, no one involved had the slightest notion that they were about to step into one of the deepest and most difficult questions in conservation worldwide. The Conservancy’s fundraising team was just trying to raise enough money to buy conservation easements on Mount Hamilton, an island of natural habitat in an encroaching sea of suburbs south of San Jose.
Neiman asked how the Conservancy knew the investment would provide lasting protection for the oak woodlands and the creatures that live there. He wasn’t interested in preserving a piece of land for just 30 years. “That would be an act of vanity,” he says. He wanted to know whether it would be protected for hundreds of years.
“That stumped me,” says M.A. Sanjayan, who was called in to answer Neiman’s question. In fact, it touched a raw nerve for Sanjayan, the Conservancy’s chief scientist in California at the time and now a lead scientist at the headquarters in Washington DC. Sanjayan realized that Neiman was probing one of the most important unanswered questions in the science and practice of conservation biology. So he told Neiman the truth. They didn’t know. The fundraiser gripped the edge of the table. But Sanjayan pressed on. Maybe with Neiman’s help, The Nature Conservancy could find a way to begin answering the question of whether its conservation efforts are really conserving what they say they are for the long run.
That was nearly three years ago. Now, a new “measures and audit team” is putting the final touches on an ambitious program that will require Conservancy projects to measure whether they are achieving their goals and deploy auditors to verify the results for managers and donors. The idea has been field-tested at 15 Conservancy projects around the world. And it will be rolled out across the Conservancy beginning later this year.
Along the way, The Nature Conservancy has rediscovered something that businesses and other nonprofits have learned about measuring their performance: in addition to being an important tool for accountability, measuring and auditing can be powerful tools for continuous learning and improvement. As a result of the audits, some Conservancy projects have abandoned unproductive strategies. Others have realized the need to change strategies to counter new threats.
The Nature Conservancy’s effort is being watched closely by the entire conservation community. But the Conservancy is not alone in trying to figure out how to measure its results and open its conservation books to outside review, in much the same way its financial statements are publicly reported.
Four other major conservation organizations—Conservation International, the World Wide Fund for Nature/World Wildlife Fund, the Wildlife Conservation Society, and the African Wildlife Foundation—have joined the Conservancy in forming the Conservation Measures Partnership, coordinated by Foundations of Success, a nonprofit that helps conservation groups measure success. Together, they are working to establish a common framework for auditing their work—in much the same way that publicly traded companies use generally accepted accounting principles (GAAP) to report financial results on Wall Street.
But the conservation organizations promise to go beyond just sharing their financial accounting, like businesses do. They are preparing to share their management accounting, the performance measures that businesses usually keep close to the vest. And that is a brave step beyond business as usual.
There may never be a final answer to the vexing question, “Are we conserving what we say we are?” But measuring and auditing could provide a dynamic method for conservation projects to continue to hone their strategies in a changing world. And that, it turns out, is precisely what Neiman had in mind when he asked his fateful question.
Backing Up Success Stories
Sanjayan chose the Cosumnes River Preserve in California as the first project to audit because it had a documented history of conservation activities, a wealth of data from research and monitoring, and was well staffed and funded. If this project couldn’t be audited, no project could be.
The measures and audit team was made up of scientists and managers from around the Conservancy, an ecologist from the Cosumnes preserve, and a staff member from Foundations of Success. They arrived at the preserve on a summer morning in 2001 with the goal of answering the simple question: “Are we conserving what we say we are here?” But it soon became apparent that the answer would not be so simple.
For many years, The Nature Conservancy has had a simple metric for success: “bucks and acres”—how much money was raised and how much land was protected. By this measure, the Cosumnes River Preserve was a success. It protected one of the last remaining riparian oak forests along the last undammed river in the Sierra Nevada, home to sandhill cranes (Grus canadensis) and chinook salmon (Oncorhynchus tshawytscha).
The Cosumnes also had a great tale to tell about learning and adaptive management. One day, a staff ecologist was studying aerial photographs when he noticed that an oak forest that had grown up in a former farm field was missing in a 1985 photo, taken the year the river broke through a levee during a flood, carrying sand and debris on to the field. The seeds of the forest, he realized, must have been planted by the flood. Here they were painstakingly planting trees around the preserve with mixed results, and the river had done the work much more effectively. This soon led to a plan to breach the levee in other spots and use the river to enlarge the riparian forest.
But the measures and audit team wanted to know the numbers behind the story. They dug down into the computer spreadsheet that Conservancy projects are supposed to use to document the health of the ecosystem. Much like a corporate balance sheet, which tracks assets and liabilities, the Conservancy spreadsheet tracks critical factors such as the size, condition, and landscape context of species populations and habitat, and the threats they face. The team found that many of the entries were best guesses about whether to classify the health of a species such as salmon or a habitat such as the oak forest as “good,” “fair,” or “poor.” The judgments were entered into the spreadsheet simply by clicking on one of the categories without providing an underlying scientific rationale or data. Dan Salzer, the designer of the system, called this “mouse-based monitoring.”
The Cosumnes River Preserve had a lot of data that confirmed the project was making progress on important fronts, such as improving salmon habitat. But the information wasn’t accessible in one place where auditors could verify it and managers could use it to compare the effectiveness of their approaches. They asked project staff to go back and fill in the books, this time documenting the scientific evidence and rationale behind each of their judgments.
Over the next few months, it took nearly 600 hours for preserve staff to prepare a full report that could be audited by the rigorous standards the team demanded. Preparing for the audit turned out to be at least as valuable as the audit itself. The project realized it needed to devote more attention to emerging threats, such as groundwater pumping that is drawing water from the riverbed for nearby farms, vineyards, and suburbs. The detailed report set a new bar for the Conservancy, unfortunately not one likely to be cleared by many of their projects.
There Be Dragons
The measures and audit team then decided to test audit a project at the opposite end of the spectrum: Komodo National Park in Indonesia, a classic besieged “paper park.” If the audit worked in Komodo, it might work anywhere.
The park is home to the famous Komodo dragons, giant monitor lizards that live on remote islands fringed by some of the most diverse and beautiful coral reefs in the world. While the dragons have done well in the park, the reefs have not. They have been severely damaged by local fishermen and roving bands of pirates who use cyanide to stun big groupers for the live fish markets in Singapore and Hong Kong and aquarium specimens for the worldwide market. When the more valuable fish are gone or the fishermen are in too much of a hurry, they simply toss dynamite or homemade bombs overboard and harvest the fish that float to the top, leaving a rubble field of destroyed coral under the waves.
When the team met with Peter Mous, the project ecologist, on a diving boat sailing among the islands in late 2001, he told them he knew what needed to be done. Stop the bombing and overfishing. “It’s not rocket science,” he said.
Mous had little patience for the complex ecological models that were part of the audit. There was urgent work to be done. And the Komodo project could easily demonstrate its success. The number of bombing incidents had dropped after the Conservancy donated a patrol boat to the park. After two years, there was a four-percent increase in live coral cover, and the decline of large groupers had stopped.
But the team persisted. Mous had simple ecological models for the coral reefs, mangroves, and sea grass beds that are the main elements of the marine ecosystem. But the team insisted on drawing up a conceptual model of what the project was trying to protect and the threats it faced. As Mous worked with the team to fill in the threats and trace them to their sources, they all began to see the value of adding the human element to the model.
One of the causal chains ran right to the heart of the Muslim faith of many local communities. At least once in their lives, devout Muslims are expected to make the hajj, a pilgrimage to Mecca. This requires cash. And the quickest way to raise cash is blast fishing. One way to break the chain, the team discussed, might be a micro-loan program to help pilgrims make the hajj without resorting to destroying the reefs.
This was one small link in a complicated project that involves encouraging local aquaculture; educating villagers about the value of intact reefs, mangroves, and sea grass beds as a steady source of fish; and an ambitious long-term goal of improving park management through a concession that will channel increased revenue from tourists back to the park rather than to the central government.
Two years later, the Komodo project is still struggling. Offering micro-loans for the hajj, it turns out, may be meddling too much. The patrols have kept blasting down. But there have been pitched battles between park rangers and heavily armed poachers, resulting in many arrests and two deaths.
Tourism in Indonesia has plummeted after September 11th, the terrorist attack in Bali, and the SARS epidemic. Income for the park has declined. Some in the Conservancy have questioned how much longer the organization can afford to subsidize the park.
That is where the audit proved useful. It didn’t solve the park’s problems, but it documented that the project was making a difference in an extremely difficult situation. And recently, the government approved the concession plan that the audit had underscored as key to a long-term solution for Komodo.
Trial by Fire
The measures and audit team came away convinced that they were onto something worthwhile. So they worked to distill the lessons and simplify the process. Having a conceptual model was essential. It forced people to articulate their intuitive understanding of their projects as an explicit set of causes and effects that could be shared and analyzed critically with others. The team knew that the spreadsheet that they provided to projects had to be more user-friendly. At the same time, however, this conservation balance sheet had to be complex enough to be scientifically rigorous.
The balance between simplicity and complexity would remain in creative tension as the team asked 13 other Conservancy projects to test the measures on their own. One of those projects was the Lake Wales Ridge, the dry sandy spine of Florida. Fire is an essential part of this highly fragmented ecosystem, which is home to 13 endemic species. The Conservancy’s goal there is to buy subdivided parcels that have not been developed yet and to return fire to the ecosystem.
To that end, the Conservancy had spent US$170,000 a year over 5 years outfitting a fire strike team to help local agencies manage prescribed burns. Fire crews throughout the area have been trained to set and control the periodic fires that are necessary to maintain the oak shrub patches favored by native species like the Florida scrub jay (Aphelocoma coerulescens).
The area burned has doubled. “Sounds like a fabulous success,” Mary Huffman, the director of the project, told a recent gathering of Conservancy scientists. But it turns out that is not enough. Even with the Conservancy’s help, the area burned each year felt short. “It’s not about capacity,” Huffman said. The people who were being trained to manage burns are still afraid to set them because they could lose their jobs if a fire got out of control.
The project had missed an important link in the causal chain. Rather than training fire fighters, project managers should have focused on higher-level decision makers, convincing them to provide incentives for their crews to set more prescribed fires.
“I’m mad as hell,” Huffman said. But, she added, if they hadn’t gone through the audit, they might not have learned they needed a course correction.
Some scientists worry that this is still no better than trial and error learning. “It’s what kids do when they touch the stove,” says Frances James, an ecologist at Florida State University, who is also on the board of governors of the Conservancy. James has advised the auditing team to encourage projects to design scientific experiments into their work. Conservation cannot be done everywhere all at once. This affords opportunities for simple, inexpensive, controlled comparisons of conservation techniques.
Open Source Code for Conservation
While The Nature Conservancy was testing these new methods, other conservation groups were also trying to figure out how to measure and audit their own work. By the time the Conservation Measures Partnership came together, they had developed their own ideas, but there was a surprising convergence of thinking. They decided their first task would be developing common “open standards for the practice of conservation” modeled on “open source code” for software developers—programming that can be used and adapted by anyone. These are the steps that any conservation project should complete and could be audited on:
- Developing an explicit causal model of the ecosystem and threats;
- Developing a plan of action and monitoring and evaluation targeted at important threats and information needs;
- Implementing actions and monitoring and evaluation;
- Analyzing the data on an ongoing basis;
- Adapting strategies based on new information;
- Communicating what is learned; and
- Iterating the cycle of planning, acting, measuring, learning, and improving.
These may seem painfully obvious. But many projects neglect one or more of these important steps. In a recent survey, the William and Flora Hewlett Foundation found that less than one in ten projects it supported had an explicit causal model of how they hoped to affect the systems they were working in. Although many organizations measure what they do, few measure the effects of their work.
There are two obvious reasons for this. People intuitively respond to problems by doing what they know how to do. As the old saw goes: a surgeon will likely recommend surgery. And it is far easier to measure what you do and call it success than it is to take a hard look at whether you actually made a difference.
Activity-based Cost Accounting
The partners are also testing activity-based cost accounting on a variety of projects to see if the cost-effectiveness of different conservation approaches can be compared. Businesses have long used activity-based accounting to break out the costs of different parts of their operations. Rather than simply lumping all of the phone bills or rent or travel costs together, they are assigned to different activities.
Conservation projects have been slow to adopt this powerful accounting tool, in part because it requires more complicated bookkeeping, but also because it is hard to assign costs when one works on multiple strategies at the same time. But at the behest of the Gordon and Betty Moore Foundation, the Wildlife Conservation Society, Conservation International, and the World Wildlife Fund are implementing activity-based cost accounting in their multi-partner Gabon Parks Project.
Last year, the president of Gabon announced the creation of 13 new national parks covering close to 10 percent of the country. The three conservation organizations were already working in Gabon. But the Moore Foundation wanted them to collaborate to help Gabon manage the new parks—and it wanted comparable, measurable results.
This is a key element in measures and auditing. If the partners can generate comparable activity-based accounting, it could provide the basis for estimating a return-on-investment for different conservation actions. Then the circle will be complete and the potential of measures and auditing to drive strategic conservation efforts could be realized.
Learning to Audit, Auditing to Learn
Finally, later this year, the Conservation Measures Partnership will begin conducting pilot audits on each other’s projects—three this year with at least three more to come next year. This is akin to Adobe sending auditors to Microsoft, and it could signal the beginning of a new era of transparency, accountability, and collaboration among conservation organizations that until now have largely competed on the urgency of their appeals rather than their actual results.
As the center of gravity has shifted from the pioneering efforts of organizations like The Nature Conservancy to the collaboration of different conservation organizations around the world, the truly transformative power of Seth Neiman’s provocative question is just beginning to be realized.
“I didn’t realize it was such a loaded question,” Neiman says. But now, he believes, “if this is done right, it will change the whole discussion about conservation.” Instead of seeing conservation as just a good cause, he says, people will start asking, “What are your results?”
