Call for transport consultants to be punished gets cautious response

A project management expert's call for transport consultants to be "punished" for producing inaccurate and misleading reports raises an issue that needs to be addressed but punishment would be counter-productive, according to practitioners. 

Bent Flyvbjerg, a professor and the founding director of BT’s Centre for Major Programme Management at the Saïd Business School, University of Oxford, said in a paper before Christmas that consultants should be penalised financially or even face criminal charges (LTT 21 Dec).

His comments have generated intense debate on LinkedIn over Christmas. Colby Brown, a planning technologist, said: "Not enough retrospective studies of forecast accuracy are done and on the whole I think Flyvberg's work is a laudable contribution. However, my concern is that draconian 'punishment' for faulty forecasts will not lead to better planning using better methods but more likely agencies and consultants reverting to capital investment decision-making based upon non-analytic methods such as Delphi, sketch-planning, or worse, just caving in to political force."

"Whereas what may be needed is the opposite: more forecasts using different models from a variety of analysts to get as many perspectives as possible before acting (in the limit, "crowd-sourcing" the planning process is one possible solution to the problem Flyvbjerg correctly identifies)." 

Roger Witte, a software engineer, commented: "In spite of the overly dramatic headlines that Flyvberg likes, the issues that he identifies are real. We, as a society, fail to collect data on actual outcomes against model outcomes so that forecasting can improve." He backed Flyvberg's call for classified databases of observations and forecasting errors that can be analysed statistically to create reference classes for new forecasts.

However, Witte cautioned that "the most obvious unintended consequence of shifting the forecasting risk onto consultants is the probable increase in consultancy costs".

Flyvbjerg’s international research has documented major inaccuracies in the costs and benefits of many transport projects, with a systematic bias of underestimated costs and exaggerated benefits, such as demand (LTT 10 May 07). He has advised the DfT on ways to combat the phenomenon known as optimism bias, whereby project costs estimates often start low but creep up. His comments were not welcomed by everybody, however.

Alan Howes, of Alan Howes Associates in Falkirk, said: "In every case I can think of, the major problem has been the quality of the input data - which mainly comes from the client. My work is mainly in the Arabian Gulf - we can't even get the clients to be realistic about 15-year population forecasts, and our main problem is curbing the clients' ambitions to spend vast sums on dubious infrastructure.

"In the UK though, the over-riding uncertainty is the direction the overall economy is going to take - how does Flyvbjerg propose accounting for that?"