TransportXtra features news, opinion and analysis from the UK transport policy & planning;
passenger transport; urban development & parking industries.

Tackling uncertainty: superforecasting comes to Modelling World 2020

It turns out that the Superforecasters were quickly able to develop early, accurate forecasts on COVID-19. Time Magazine actually called them ‘eerily accurate’ in a piece back in June, and they have consistently outperformed epidemiologists and biostatisticians from around the world in predicting the course of the pandemic

Juliana O'Rourke
05 October 2020
Back in 2011, the Good Judgment Project, led by Philip Tetlock, author of Superforecasting: The Art and Science of Prediction, was the undisputed victor in a tournament led by the US-based Intelligence Advanced Research Projects Activity (IARPA) to gauge the accuracy of forecasts
Back in 2011, the Good Judgment Project, led by Philip Tetlock, author of Superforecasting: The Art and Science of Prediction, was the undisputed victor in a tournament led by the US-based Intelligence Advanced Research Projects Activity (IARPA) to gauge the accuracy of forecasts
Warren Hatch, CEO, Good Judgment
Warren Hatch, CEO, Good Judgment

 

Warren Hatch, CEO, Good Judgment, will be joining us at Modelling World 2020 on October 16, virtually, to share his skills, insights and experience and to engage with delegates in practical demonstrations. Back in 2011, the Good Judgment Project, led by Philip Tetlock, author of Superforecasting: The Art and Science of Prediction, was the undisputed victor in a tournament led by the US-based Intelligence Advanced Research Projects Activity (IARPA) to gauge the accuracy of forecasts. Good Judgment’s forecasts were so accurate that they outperformed intelligence analysts with access to classified data, said IARPA. 

Speaking to Forbes late in 2019, Warren Hatch said: ‘Good Judgment Inc is the commercial successor to the research project that was led by Philip Tetlock at the University of Pennsylvania and the University of California, Berkeley. The work began with the US government wanting to see if there were ways to improve forecast accuracy beyond the so-called “wisdom of the crowd”.  I had signed up for the research project and started emulating the behaviours of the best forecasters.’ After about 20 questions, there was a click, said Hatch, and he got his “aha” moment. He became a superforecaster.

Good Judgement now has a network of superforecasters working on economic, social and investment issues, including transport. Hatch holds a doctorate in politics from Oxford, a masters in Russian and international policy studies from Middlebury Institute of International Studies at Monterey, and a bachelors in history from the University of Utah. 

The epidemiologists hope that such ‘ensemble modeling' will generate better outputs for global health organisations. So while COVID-19 has shaken everything up, best-methods forecasting is able to quantify a lot of that additional uncertainty

Speaking to TransportXtra last week about increased levels of uncertainty across all aspects of life since the pandemic took hold, Hatch was cautiously confident. ‘There’s more uncertainty across the board. But Superforecasters are fine with that, and have quickly adjusted their thinking. Overall, we did not know beforehand whether their skills—honed for forecasting geopolitics and geoeconomics—would transfer to forecasting a pandemic.

'It turns out that the Superforecasters were quickly able to develop early, accurate forecasts on COVID-19. Time Magazine actually called them ‘eerily accurate’ in a piece back in June, and they have consistently outperformed epidemiologists and biostatisticians from around the world in predicting the course of the pandemic. We are working now with several epidemiologists on ways to combine forecasts from their models with the consistently accurate forecasts from our humans.

'The epidemiologists hope that such ‘ensemble modeling will generate better outputs for global health organisations. So while COVID-19 has shaken everything up, best-methods forecasting is able to quantify a lot of that additional uncertainty.'

The superforecaster and the modeller

Tom van Vuren, Chairman of Modelling World and an experienced modeller who has had an eye on superforefcaster for many years, is looking forward to meeting up with Hatch at this year’s event.

He says: ‘I wrote in 2016 about superforecasting, and am super-excited to have a real superforecaster present at Modelling World this year. My 2016 thinking was influenced by the Freakonomics Radio Podcast: 'How to be less terrible at predicting the future'. Although the predictions discussed in the podcast are much more related to politics, economics or sports than to transport policy or infrastructure demand forecasting, our own profession can also benefit.

People are bad at forecasting because there is no danger in getting it wrong. That is sometimes true for us modellers as well. Even if our forecasts look wrong, we present ourselves as being powerless, as the future growth assumptions are prescribed by third parties including Government (even though our models are correct...).  But we can do better – we were not asked to produce (just) a robust model, we were also expected to provide a decent forecast. Challenging the inputs should be part of what we do – who is better placed than us? A bad forecaster is systematically over-confident and often dogmatic in his/her view of the world, unwilling to change his/her mind in response to new evidence. Good forecasters are humble about their judgments, are curious and are willing to challenge conventional wisdom. 

The average forecast derived by a group of forecasters tends to be more accurate than most of the individual forecasts from which the average was derived; we are more likely to provide better forecasts if we are willing to learn from other experts. That’s what Modelling World is all about so get ready to hear van Vuren and Hatch in conversation at Modelling World!

Sustainable Transport Officer
Derbyshire County Council
County Hall, Matlock
Grade 9 £30,054 - £31,963 per annum
Team Leader (Network Co-ordination) RBKC618467
The Royal Borough of Kensington And Chelsea

£49,854 - £56,565 per annum
Specialist Transport Manager
Derbyshire County Council
County Hall, Matlock
Grade 13 £46,363 - £50,006 per annum
View all Vacancies
 
Search
 
 
 

TransportXtra is part of Landor LINKS

© 2024 TransportXtra | Landor LINKS Ltd | All Rights Reserved

Subscriptions, Magazines & Online Access Enquires
[Frequently Asked Questions]
Email: subs.ltt@landor.co.uk | Tel: +44 (0) 20 7091 7959

Shop & Accounts Enquires
Email: accounts@landor.co.uk | Tel: +44 (0) 20 7091 7855

Advertising Sales & Recruitment Enquires
Email: daniel@landor.co.uk | Tel: +44 (0) 20 7091 7861

Events & Conference Enquires
Email: conferences@landor.co.uk | Tel: +44 (0) 20 7091 7865

Press Releases & Editorial Enquires
Email: info@transportxtra.com | Tel: +44 (0) 20 7091 7875

Privacy Policy | Terms and Conditions | Advertise

Web design london by Brainiac Media 2020