Evaluating Megaprojects: What Constitutes Success?
Megaprojects always face challenges, but looking only at cost and timing may not be the only way to judge their success
October 2015 | by Steve Johnson
Megaprojects will always struggle with unforeseen events, regulatory requirements, technical difficulties, financial constraints, and politics. The costs—at least $1 billion—of megaprojects are high. The complexity is increased by the fact that there are many different stakeholders, including owners, managers, sponsors, and local communities, and they all have different perspectives. Moreover, delays and budget overruns are, admittedly, the norm rather than the exception. As a result, many megaprojects are remembered more for these issues than the lasting good they produce.
Defining success is complicated. It might seem straightforward to consider projects successful that come in on time and on budget, but what if the finished venture doesn’t solve the problem it was meant to? If a high-speed rail system meets its time and budget commitments, for example, but cannot attract the ridership necessary to alleviate traffic congestion and improve air quality, it is still a failure.
On the other hand, comprehensive research has found that projects that are both late and costly can still be considered successful if they deliver what is promised and if associated socioeconomic benefits are accounted for. Usually, however, such benefits are not even identified. For example, Boston’s Big Dig—a complex plan that rerouted an interstate and built a bridge, a tunnel, and a greenway—was a source of enormous frustration and controversy because it took many more years and many more billions of dollars than projected to finish. Now that it is done, though, residents and visitors are enjoying the benefits of dramatically reduced travel time, as well as improved waterfront access, wildlife conservation, new and expanded parks, and increased business development. Boston is a better place because of the Big Dig.
This is not to say cost and schedule do not matter; of course they do. Like everyone associated with the Big Dig, I wish it had gone more smoothly and less expensively. But its history also hints at another point: sponsors need to do a better job of assessing and then communicating the benefits a project will deliver.
Project-management Success versus Megaproject Success
There is a difference, in short, between successful project management and successful projects. Project-management success has traditionally been defined as meeting scope, schedule, and cost compliance requirements. These three factors constitute what is known as the “iron triangle” and are the traditional benchmarks used to evaluate most projects. There are additional attributes, however, such as socioeconomic improvements, technological innovation, and improved environmental conditions that could and should be part of the equation in determining whether a project is a success.
In recent years, “comprehensive benefits assessment” has been used to attract financing for big public projects and to build community support. This term means that all benefits, tangible and intangible, are taken into consideration in assessing a project’s justification. Intangibles include skill development, alleviation of poverty, knowledge sharing, and institution building. Undertaking a comprehensive benefits assessment is becoming the norm for determining the likely long-term success of projects and influencing decisions about priorities and resource allocation. I believe that all projects should incorporate a comprehensive assessment from the start and develop practices to implement and measure these benefits. For example, the worth of technological improvements can be measured by the increased value of the intellectual-property portfolio.
There are different ways to measure these benefits. One method is to do qualitative and quantitative analysis of project and industry data, including stakeholder surveys, screening, and observation. Typically, there is a base-case cost-benefit analysis to which investment alternatives are compared. The analysis addresses these questions: What additional benefits will accrue if this alternative is chosen? And what additional costs will it incur? The objective is to translate the effects of an investment into monetary terms and to account for the fact that benefits play out over a long period while capital costs mostly arise up front.
The World Bank Group, for example, seeks to link infrastructure-development projects to job creation, environmental improvements, and poverty reduction. From that point of view, a road becomes more than a means to get from point A to point B; it is a way to help the poor, and this benefit should be taken into account when considering whether to build. In the developing world, a number of studies point to a significant impact of roads on poverty reduction because of their effect on economic growth. A 1.0 percent increase in road investment is associated with a 0.3 percent drop in poverty incidence over five years.
Evaluated comprehensively, many large-scale projects deliver benefits above and beyond what was originally planned or even imagined. The original environmental assessment of the Big Dig focused on the economic and environmental benefits of alleviating traffic congestion. The Central Artery, which it replaced, was built to carry 75,000 vehicles a day and was instead supporting 190,000. As the project evolved, new opportunities arose and additional benefits became part of the thinking:
- innovative engineering, infrastructure, and technological advancements
- integration of isolated neighborhoods and transportation systems
- creation of a new island park from a mountain of decaying garbage
- redeveloped landfills and the expansion of wildlife population
- economic-development opportunities
The Big Dig also established the nation’s first innovation and advancements program; the program shared the knowledge gained from the project with transportation practitioners around the world. Since the Big Dig, this kind of knowledge transfer has been incorporated into every large-scale project that receives federal funding and is also required under many state statutes and regulations. Finally, while no one has yet counted up the benefits in monetary terms, the technological advancements that the Big Dig pioneered, such as environmentally enhancing context-sensitive design, safety-incentive programs, innovative ground-freezing techniques, and the largest use of urban slurry-wall modules, are now being used in other projects around the world.
The Sustainability Imperative
Sustainability normally refers to environmental practices. In megaprojects, a broader definition, including concepts of economic, social, and institutional sustainability, is appropriate.6 The San Francisco–Oakland Bay Bridge, which was damaged during the 1989 earthquake and reopened in 2013, was $5 billion over budget and took ten years longer than originally projected. But the bridge was built to last for 150 years—much longer than the typical 50 years of service—and to withstand earthquakes and seismic activity of the highest magnitude. Both factors will support substantial savings down the line. Although determining the bottom line on the Bay Bridge as constructed is difficult, the point is that cost and schedule are not the only ways to judge success. Other factors must be incorporated into the project’s cost-benefit analysis. There needs to be a framework to help governments—and the public—understand the larger benefits of a project and to include the impact of economic and social development in the final analysis. That doesn’t mean that residents will be less irritated at the daily disruptions of projects. But perhaps, looking at the bigger picture, they will consider them worth the trouble.