Curiosity, honesty and a willingness to change your mind – the characteristics of a superforecaster

As a transport modeller and analyst, I am very pleased that the DfT corrects its mistakes, so strengthening the methodology

Tom van Vuren is Chairman of Modelling World

Tom van Vuren is Chairman of Modelling World

At the end of September 2022 the Department for Transport released a review of their minor road traffic estimates for the period between 2000 and 2020. The implications for the published growth figures were substantial. Whereas in the previous estimates traffic on minor roads had grown by more than 25% in the ten years to 2019, the revised figures only show an increase of just under 10%.

Join us at Modelling World 2023 on 2-3 November in Birmingham for essential modelling discussions and quality networking

What happened?

The DfT corrects their traffic estimates every 10 years or so, through a benchmarking exercise, but this 2021 correction was greater than that of previous benchmarking exercises; and therefore adjustments have also been made to historic figures going back as far as 2000, to ensure consistency and compatibility in the trend dataset.

Apart from a few errors in the use of the underpinning datasets, the main changes that were made by the Department was in their segmentation of the road segments used in the benchmarking analyses (around 10,000 of them). 

This time for the first time, the DfT divided the sample of roads into high and low flow sites, depending on whether they carry more than the median number of observations or less. It turned out that he sample that was used for the 2009 benchmark had a higher proportion of ‘low’ flow sites (44%) than the sample selected for the 2019 benchmark (39%) and hence growth had been artificially high.

Another change is how the updated methodology deals with cul-de-sacs. As users generally don’t drive the full length of the road, traffic on these types of road was halved in 2019 – a not unreasonable assumption. This was not originally taken into account when calculating traffic estimates from the 2009 benchmark.

Thirdly, the new methodology treats Inner and Outer London as two separate segments. Analyses of major road traffic estimates illustrate that traffic growth patterns differ between Inner London and Outer London and the review now analyses these as two separate levels in the minor roads benchmark regional stratification.

But errors were also made and subsequently corrected:

• Around 50 count points (out of the benchmark total of 10,000 sites) had been allocated to the wrong road type.

• The 2004 road traffic estimates were originally calculated using 2003 road lengths.

• Road traffic estimates were missing for small lengths of road for a few road categories in a few local authorities. 

• The original 2012 road traffic estimates were calculated using the change in traffic between 2010 and 2011, and the 2013 figures used a provisional rather than the final version of the 2012 traffic estimates.

My own analyses of the revised traffic growth figures provided in the tables available online here  show that traffic on minor roads grew by 10% between 2009 and 2019, a very similar number to (and actually a fraction less than) traffic growth in that same period on major roads and on the overall network.

Does it matter? I expect that some transport and planning arguments that relied on the previous figures will need watering down. But as a transport modeller and analyst I am very pleased that the Department has published its findings openly, and that as a result of the investigations the methodology has been strengthened.

Being able to change your beliefs is one of the essential traits of superforecasting.

Tom van Vuren is Chairman of Modelling World, a Visiting Professor at the University of Leeds and the Policy Director at the Transport Planning Society. He recently joined Amey as Strategic Business Partner