Braveheart 2.0 | What is superforecasting?

😬 Stick to cars and space, Elon

🎧 You can listen to this week’s two main stories on Spotify or Apple Podcasts.

❤ Please click that little heart at the very top of the page - it’s like paying us, but for free!

📧 If you reached us from the web and would like to subscribe, you can do so here 👇

Good Morning! We have the pleasure of welcoming 27 new friends this week. We might stop announcing new subscribers one day, but we'll never stop being thrilled that you chose to sign up!

Those who've been with us a little longer will know that we're raising money (update: we've almost closed our round 🤯). We'll be using that funding to grow our team, so we've been thinking about creative ways to hire folks...

Unless the application is for 'ivory trader' and the correct answer is ‘kill it’, we just can't imagine what a ‘good’ answer is?

It also reminds us of a perhaps apocryphal story about a Goldman Sachs job interview during which the applicant was asked "what kind of fruit would you be?". Apparently the correct answer is “a grape”, because they're good alone but grow together. Spew.

This week:

  1. Scottish independence: Braveheart 2.0?

  2. Superforecasting: what you can learn from the CIA.

Housekeeping: last week's edition was our most liked yet! We received a very positive response to our 'Analysing an Analyst' series, so we'll bring you another soon. Thanks for the feedback.


⚔ Scottish independence: Braveheart 2.0?

By Helen

This week, I turned to pop culture for research inspo on Scotland’s potential secession from the United Kingdom (UK) – a play that’s back on the cards following last Thursday's election victory of the pro-independence Scottish National Party (SNP).

What began with a ‘quick scroll’ through Netflix morphed into ten hours of binge-watching Outlander and a tartan-clad me yelling ‘freedom’ from my couch. (Look, I know he’s a bit mad, but young Mel Gibson was 🔥 – watch Gallipoli and thank me later).

The takeaway from this thorough investigation is that one should always back the Scots to defeat the English. The success of this latest independence movement could spell the end of the 314-year-old English-Scottish union, and rock the broader UK.

History brush-up: a marriage of convenience

Scotland boasts fine cultural exports such as deep-fried Mars bars, Billy Connolly, and haggis. But perhaps it is most legendary for being a fierce and independent kingdom.

Did you know:

  1. The Scots booted out the mighty Roman Empire in 103 AD? We have the Justice League, yet no one thought of dusting off Rusty and Mel for a Maximus Decimus Meridius v William Wallace crossover extravaganza?

  2. The Scots also fought a series of victorious wars against the English to maintain Scottish independence in the 13th and 14th centuries?

  3. Scottish comedian Janey Godley single-handedly rebuffed a US invasion of her homeland in 2018?

They won’t tell you that last one in the history class 😂. Okay, back to the olden days.

All was separate and peachy in the British Isles until 1603, when the death of English Queen Elizabeth I transferred the English crown to Scottish King James VI (the OG Queen Lizzie’s cousin – we swear all the European monarchs are related).

As the King of Scotland and the King of England, James began what would be a 100-year political process of reuniting the two countries. The Kingdom of Great Britain was officially created by the Acts of Union in 1707.

Back then, the Scots were not in great shape. The union with England delivered welcome economic support and political clout, particularly as England lined its coffers with the spoils from the 'Empire(Niall Ferguson is always a good read).

Rising Scottish nationalism and Brexit

Not every Scot welcomed the English union in 1707. Calls for Scottish independence continued into the 20th century, reaching a peak in the 1960s and 70s after oil was discovered in the North Sea. Because Scotland was not independent, it didn’t control royalties:

It’s Scotland’s Oil!

- SNP political slogan circa 1974

Two following referendums established the system of ‘Home Rule’ for the Scottish Parliament in 1999, which essentially granted the Scots more autonomy from the UK over some areas of governance:

Alas, this ‘devolved’ system wasn’t enough to quell secession calls. So, the Scots held a referendum in 2014 for total independence, which ended with a decisive loss for the pro-independence camp (and a smug then-UK PM David Cameron, who thought this issue had been settled once and for all, lol).

The Brexit result in 2016 rekindled Scotland’s independence movement. 62% of Scots voted to remain in the EU; but the Brexit vote passed, leaving Scots miffed that their voices went unheard.

Nicola Sturgeon, Scotland’s popular first minister and head of the pro-independence SNP, pounced on this swell of sentiment after Brexit (>50% in favour of independence), and here we are today.

The ‘For and Against’ of Scottish Independence

It's all for nothing if you don't have freedom.
- Sir William Wallace (Braveheart)

Aside from honouring the wishes of Sir Wallace, those fighting for independence argue:

  • Independence grants self-determination over political affairs like foreign policy and defence. Plus, Scotland gains representation in international organisations;

  • Scots can exploit their own natural resources, such as the North Sea oil and gas reserves (which account for 64% of the EU’s oil reserves, and a whopping 96% of the UK's oil output);

  • Independence would make life easier for Scottish exporters (e.g. fish and shellfish industries) which relied on free-trade with the EU and contributed ~US$20b to the economy annually; and

  • Scotland could continue building its proud national identity, buttressed by what most countrymen believe to be a ‘superior’ legal and educational system compared to those of their UK neighbours.

Those against independence argue:

  • Scotland’s population (5.5 million) can better compete in a globalised world under the diplomatic, military, and economic influence and security of ‘Team UK’ (64 million);

  • There’s no real long-term plan for generating tax revenue after the UK’s union handouts stop and Scotland's natural resources run dry;

  • Scotland would invite the headache of deciding which currency to use: the pound, euro, or a new Scottish currency?

  • Independence would hamper Scottish trade with England, its biggest trading partner (60% of Scottish exports go to the UK) and restrict freedom of movement for people from both sides.

But here’s the really big question: what will happen to Balmoral Castle? For God’s sake, won’t anyone think of the Queen’s holidays?

Zoom out: what’s next?

The fight for Scottish independence has just begun. The SNP still needs to push the legislation for #indyref2 through Scottish Parliament, hope that the Scots actually vote for it, and then get the legislation approved by the UK Parliament.

If Scotland were to become independent, the UK would lose about a third of its territory and about 8% of its population and GDP.

If those hurdles are cleared, Scotland faces a long road ahead. First up is a time-consuming and costly (~3x more than Brexit) renegotiated trade deal with the EU. If that all clears, there’s the tedious admin tasks of state-creation, like:

  • bolstering defence capabilities;

  • striking independent trade and security deals with other countries; and

  • building a physical border between Scotland and the UK (passport checks, not Hadrian’s Wall 2.0)

The big question is: might a successful Scottish independence bid spark secession movements in Northern Ireland and Wales? Or even inspire separatists further afield (e.g. Catalonia)?

Scottish secession would be the biggest geopolitical challenge the UK has faced in generations. And never forget that with Boris at the helm, anything is possible.

⛈ Superforecasting: what you can learn from the CIA

By John

[T]the average expert was found to be only slightly more accurate than a dart-throwing chimpanzee. Many experts would have done better if they had made random guesses.

- Professor Philip Tetlock in 2014

What is superforecasting?

Superforecasters are people who have proven to be even better than subject matter experts (see above) at predicting the future. So perhaps think of superforecasting as crowdsourcing - if that crowd excluded the village idiots.

Tetlock’s book Superforecasting explains how to superforecast:

  1. Develop a forecasting competition. Ask employees to publicly assign probabilities to predictions about the future, with reasons for their assessments.

  2. Keep score. According to the Good Judgement Project, measuring the accuracy of forecasts over time is the secret to understanding what works and what doesn't.

  3. Identify those people who have proven to be the best forecasters. You'll want a group big enough to have 'cognitive diversity'.

  4. Give your team of superforecasters specific and verifiable questions about the future. No 'social media may become less influential over time' nonsense.

I wanted to see how it worked, so I signed up to become a forecaster for the Center for Security and Emerging Technology (CSET) at Georgetown’s Walsh School of Foreign Service.

After a few questions about myself and a few more about my expertise, I was given a list of questions to answer (so you can ignore what I said about superforecasting being crowdsourcing sans idiots).

I chose:

Does superforecasting work?

The truth is we don’t really know. Superforecasting works well in experiments, but there is limited evidence as to how useful it is in practice.

And yet, public organisations are turning to superforecasting to grapple with rising global uncertainty. The CIA, Downing Street, the Czech Government, and likely plenty of others have experimented with superforecasting.

(In fact, a few years ago a former Australian Ambassador hosted a forecasting competition among embassy staff. Presumably they excelled - it’s always tomorrow in Australia.)

Ray Dalio, the founder of infamous hedge fund Bridgewater, arguably adapted the idea behind superforecasting to develop his trademark ‘Radical Transparency’ principle.

Some benefits of superforecasting:

  • superforecasts draw on a broad range of expertise and opinions

  • the process can remove the authority bias and involve all levels of an organisation

  • the possibility of quantifying good judgment to develop verifiably better decision-making processes

Some of the drawbacks:

Based on my experience in government, I am sceptical whether superforecasting could actually work across any big bureaucracy:

  • employees are not likely to be enthusiastic about programs that highlight how poor they are at their job, causing serious cultural problems

  • superforecasting exercises must be conducted by experienced managers to avoid cognitive biases

  • the process risks becoming a way for decision makers to pass the buck (“the superforecast told me do it”)

None of that means that superforecasting is a bad idea; just that even the best ideas often can’t withstand the hot mess that is humans in large groups (aka organisational culture).

How you can use superforecasting in your organisation

1. Understand what the best forecasters have in common

It is the personality of superforecasters rather than any specific knowledge that allows them to be so accurate:

  • they are brutally honest about their success (“I got lucky”), and untroubled by their failures (“I can learn and get better”)

  • They are cautious, humble, and open-minded when researching an issue

  • They’re analytical and good with numbers without necessarily being statisticians

  • They have a collaborative, ‘win-win’ approach to sharing information and ideas

No one in your organisation fits that bill?

Luckily forecasters are made, not born - decision makers can improve their judgment from 36% better than random to 66% better with training and practice in forecasting.

2. Ensure decision makers understand how to use the superforecasts

According to Tetlock, predictions made more than five years out are no better than chance. Add to that the fact that decision makers often don’t understand probability…

… and you’ve got a recipe for forecasts being misinterpreted and misused.

I’m sad to say my former colleagues aren’t much better:

When political experts described an event as being absolutely certain, it failed to transpire an astonishing 25 percent of the time.

- Nate Silver

(Yes I know he ruined polls forever, but I promise that's a great article about predicting the weather.)

So while superforecasts might add 10% or 15% accuracy to predictions, they still can’t tell you how to prepare for that predicted future.

That’s one of the reasons we founded International Intrigue: the world is getting more volatile and decision makers of the future will need to understand how to respond.

3. Outsource your forecasts

Too small to build your own internal superforecasting team? There are consulting firms that can either lend you a team of superforecasters or construct one for you.

You heard it here first: Superforecasting-as-a-Service companies are going to be huge.

Superforecasting is another tool, not an answer

“Show me the numbers” can be a crutch for poor managers. The fact is, the decisions that senior managers get paid to make rarely come down to a spreadsheet, no matter what Chad from finance says.

On the other hand, I have learned to be suspicious of any decision maker who routinely refers to their ‘good judgment’ or ‘gut feel’. Public institutions too often trust senior executives based solely on their years of experience.

The sweet spot that [organisations] should focus on is forecasts for which some data, logic, and analysis can be used but seasoned judgment and careful questioning also play key roles. - Harvard Business Review, 2016

My forecast? There is an 85% chance you’ll hear the phrase ‘superforecast’ in your organisation before 2023, and a 100% chance that if I’m wrong I’ll deny it.

Further reading: check out this summary of 'Superforecasting', and this excellent series on forecasting more generally. If you’re interested in the nature of uncertainty and predicting the future, I highly recommend reading the amazing 'Fooled by Randomness' and 'The Black Swan' by Nassim Taleb. Warning: they're not beach reads.

➕ Extra intrigue

  • The Colombian government's Covid-19 measures and recent tax hike (to mitigate the country's steepest GDP drop in 50 years) have sparked nationwide protests. Colombia’s long legacy of civil conflict underlies the protests, compounded by reports of police committing human rights abuses against protestors.

  • The hacks are back. Hackers from cybercrime group 'DarkSide' (apt name) shut down the US Colonial Pipeline for 3 days. The pipeline runs from Texas to New York and carries almost half of the East Coast's fuel supplies, fuelling fears over imminent petrol price spikes. It's considered amongst the worst cyber-attacks on critical US infrastructure to date.

  • Big tech gets responsible? Facebook is testing a new tool that generates a pop-up message prompting users to read the thing they're about to share before they share. Twitter has had this feature for a while, and we’ll admit to totally ignoring the warning and sharing anyway 😬.

  • Take your eyes off China for one minute, and it's off building entire villages in other countries. This time, it's in northern Bhutan. In an effort to 'fortify' China's Tibetan borders, Chinese officials have planted settlers, security personnel, and military infrastructure within territory that's internationally and historically understood to be Bhutanese.

🔎 Intriguing recommendations

💁🏻‍♀️ Helen: If you're wondering how to blow the crypto-cash you've earned, consider bidding on this bottle of wine that's spent 14 months aboard the International Space Station. It's selling at a bargain of ~US$1 million a pop after its extra-terrestrial journey. And for those of us not lucky enough to spend time in space, remember that our Earthly days are precious, and we should stop spending them doing things we hate.

👴  John: Firstly, sincere apologies about the link to a Belgian trash collector last week. Riveting as that content was, here’s the right link. I’ve thoroughly checked this week’s recommendation which is… an excellent explainer on how to pick a perfect bottle of wine. It is noticeably silent on space wine, so I best go and test the accuracy of the explainer one more time 🍷.

What’s the best job application question you’ve ever been asked (preferably not fruit related)?

Leave a comment

❤ Click the little heart at the top or bottom of this page if you liked this article. We use this highly sophisticated metric to deliver more of the content you like most.

👋 Say hi by tweeting us @intintrigue, replying to this email, or dropping a comment below.

📢 And please do share our work. We’ve even prepared a tweet for you to share if you’re so minded. Thank you!

Until next week!