Is project forecasting done by liars and fools?
Apparently this is the case according to Professor Bent Flyvbjerg, of the BT Centre for Major Programme Management, Saїd Business School, University of Oxford. I was originally directed to this inflammatory statement from a post by Elizabeth Harrin on her popular PM blog “A Girl’s Guide to Project Management” which lead me to do a search for the article in questions. It can be found here and the title of the article is “Quality control and due diligence in project management: Getting decisions right by taking the outside view” published by the International Journal of Project Management, which is a pretty somber title in light of the provocative statement by the distinguished professor.
A good summary can be found on Saїd Business School website, and quoting directly from the site:
Professor Bent Flyvbjerg, a leading international expert in the management of major projects, delivers outspoken criticism of forecasters in a new paper which demonstrates the considerable inaccuracy of front-end estimates of costs and benefits that are typically used to support decisions on projects. ‘Estimates are commonly poor predictors of the actual value and viability of projects, and cannot be trusted as the basis for informed decision-making. These forecasts frequently misinform decision makers on projects instead of informing them. Some of the inaccuracy comes from genuine forecasting mistakes arising from over-optimism, but some estimates are deliberately misleading, designed to secure financial or political support for a project.’
Professor Flyvbjerg goes further and calls many of these forecasts ‘garbage’ – moreover, expensive garbage, both in terms of the fees of the consultancy firms involved, and in terms of the costs and risks to investors and other stakeholders, that arise from their misrepresentations. ‘Many forecasts are garbage and can be shown to be worse than garbage’ says Flyvbjerg. ‘These reports give the client, investors and others the impression that they are being informed about future demand, or the costs involved in a major project, when they are being misinformed. Instead of reducing risk, reports like this increase risk by systematically misleading decision-makers and investors about the real risks involved.’
I gotta say I like how direct he is and has no compunction for calling a spade a spade. And I have to agree with him quite a bit, for all one needs to do is read the news and you’ll hear of many major mega-project blunders such as the $1.3 billion dollar project the Air Force had to cancel that was a six year effort to modernize their processes and systems when they “realized that it would cost another $1 billion just to achieve one-quarter of the capabilities originally planned — and that even then the system would not be fully ready before 2020 — it decided to decamp”, according to a New York times article by Randall Stross. This project originated in 2006 and was forecast to cost no more than $628 million. To add insult to injury, this was for COTS (Commercial Off The Shelf) software so it wasn’t like they were building something elaborate from scratch!
So Flyvbjerg suggests that clients “should ask for their money back when they receive reports which later prove to be significantly inaccurate and misleading. In some cases, he urges them to seek compensation for decisions taken and occasionally, criminal action would be justified.” He even calls for organizations such as the “PMI, APM, APA, RTPI and others to use their codes of ethics to penalize and possibly exclude members who produce unethical forecasts, and chastises the associations for failing to address this adequately.”
I think this is a great wake up call for those who rely on project budget forecasts to wake up and acknowledge that the way project budgets are forecast needs a serious overhaul in many situations. I applaud his urgent call to action.
Don, I too think this is an interesting and challenging subject for an academic paper. I’ve written more about it on ProjectManagement.com today: http://www.projectmanagement.com/blog/The-Money-Files/6174/
Thanks Elizabeth for bringing this to my attention in the first place and I will definitely check out your article on projectmanagement.com!
Th Quantitative Project Manager (QPM)series on the New Leaf Project Management web site offers several ways to improve your project estimating (and avoid egregiously inaccurate estimates).