Absolute triumph for the quantitative risk analysis – part 3/3

Alex Sidorenko

5 years ago

Read this till the end. This is one of the best stories of risk management application in real life by some of the best risk managers I have ever met.

I started writing yet another article trying to convince risk managers to grow their quant competencies, to integrate risk analysis into decision-making processes and to use ranges instead of single point planning… but then I thought… why bother… why not show how risk analysis helps make better risk based decisions instead?

After all, this is what Nassim Taleb teaches us. Skin in the game.

So I sent a message to the Russian risk management community asking who wants to join me to build a risk model for a typical life decision? 13 people responded, including some of the best risk managers in the country, and we set out to work.

We decided to solve an age-old problem – win the lottery!

With the help from Vose Software free ModelRisk we set out to make history (not really, been done before, still fun though). Here is some context:

lotteries are an excellent field for risk analysis since the probabilities and range of consequences are known
in Russia, as in most countries of the world, lotteries are strictly regulated
There is a rule when a large amount accumulates, several times a year it is divided among all the winners. This is called roll-down.
if no one takes the jackpot before or during the roll-down, then the whole super prize is divided between all other winners
so the probability of winning is the same as usual, but the winnings for each combination can be significantly higher if no one wins the jackpot.

And so we set out to test our risk management skills in a game of chance.

8.06.2019

Whatsup group created. Started collecting data from past games. Some of the best risk managers in the country joined the team, 15 in total: head of risk of a sovereign fund, head of risk of one of the largest mining company, head of corporate finance from an O&G company, risk manager from a huge O&G company, head of risk of one of the largest telecoms, infosec professionals from Monolith and many others.

9.06.2019

Placing small bets to do some empirical testing.

10.06.2019

First draft model is ready…

11.06.2019

Created red team and blue team to simultaneously model potential strategies using 2 different approaches: bottom up and top down. Second model is created…

12.06.2019

Testing if the lottery is fair, just in case we can game the system without much math. Yes, some numbers are more frequent than others and there appears to be some correlation between different ball sets but not sufficient to make a betting strategy out of it. The conclusion – the lottery appears to be fair, so we will need to model various strategies.

13.06.2019

Constantly updating red and blue models as we investigate and find more information about prize calculation, payment, tax implications and so on. The team is now genuinely excited. Running numerous simulations using free ModelRisk.

14.06.2019

Did nothing, because all have to do actual work.

15.06.2019

After running multiple simulations we selected a low risk good return strategy. Dozen more simulations later here are the preliminary results:

used very conservative assumptions
probability of loss 9.8%, worse case scenario we lose 60% of the money invested
probability of winning 90.2%, 80% of the time winning would be between 50% and 100% of the amount invested, after taxes (this means there is a high possibility to double invested cash at relatively low risk)
potential upside significantly higher than downside

Red and blue team models produced comparable results.

16.06.2019

Started fundraising.

If we manage to collect more than the required budget, we decided to make 2 bets: one risk management bet (risk management strategy) and one speculative bet with much higher upside and as a result greater downside (risky strategy).

Full budget collected within just a few hours, actually collected almost double the necessary amount and, as agreed, separated 50% of the funds in to the second investment pool. Called it risky strategy. Separate team set out to develop the strategy. While I was an active investor in the risk management strategy, I decided to play a role of a passive investor in the risky strategy and only invested 16% into the risky, compared to risk management strategy.

17.06.2019

Continued to develop the model, improving estimates every time. Soon we felt the financial risks were understood by the team members and we need to take care of other matters before the big day.

First, took care of legal and taxation risks. Drafted a legal agreement clearly stating the risks associated with the strategy, the distribution of funds and the responsibilities of team members. Each member signed. Agreed to have an independent treasurer.

Then started to deal with operational risks. Apparently transferring large sums of money, making large transactions and placing big bets is not plain vanilla and required multiple approvals, phone calls and even a Skype interview :)) 5 team members in parallel were going trough the approvals in case we needed multiple accounts to execute the strategy.

Probably the biggest risk was the ability of the lottery website to allow us to buy the tickets at the speed and volume necessary for our low risk strategy. This turned out to be a huge issue and we found an absolute ingenuous solution. The information security team at Monolith did something amazing to solve the problem, and I mean it, amazing. I have never seen anything like this. It’s a secret unfortunately, because you guessed it, we are going to use it again.

Ironically, the strategy that the lottery company recommending for large bets is actually much riskier than the one we selected. How do we know that? Because we ran thousands of simulations and compared the results.

18.06.2019

The lottery company changed the game rules slightly. Ironically this made our 90% confidence interval and the probability of loss a bit better. So thank you, I guess.

More testing and final preparation. The list of lottery tickets waiting to be executed.

In the true sense of skin in the game, team members who worked on the actual model put up at least double the money of other team members.

19.06.2019

8am. We were just about to make risk management history. A lot of money to be invested based on the model that we developed and had full trust in. I felt genuinely excited. Can proper risk management lead to better decisions. I am sure other team members did too.

By about lunch time, the strategy was executed, we bought all the tickets we were planning and now had to wait for the 10pm game. Don’t know about the others, I couldn’t do any work all day, I couldn’t even sit still, let alone think clearly. Endorphins, dopamine and serotonin and more.

At 9:30pm we did a team broadcast, showing the lottery game as well as our accounts to monitor the winnings, both for excitement purposed and as full disclosure.

While they were announcing the winning numbers, two of the team members actually managed to plug them into the model and calculate the expected winning. We had the approximation before the lottery company knew themselves.

Here are just some of the happy faces :))

You guessed it, we won. Our actual return was close to 189% on the money invested after taxes (or 89% profit, remember our estimate was 50% to 100% profit, so well within our model). We almost doubled our initial investments. Not bad for risk management. Good luck solving this puzzle with a heatmap :))

20.06.2019

More excitement, model back-testing and lessons learned and, this was most difficult but not surprising, explaining to all non-quant risk management friends that no, this was not luck, it was great decision making. In fact our final result was close to P50, so in all honestly we were actually unlucky, both because we didn’t get some of the high ticket combinations and, more importantly because 5 other people did, significantly reducing our prize pool.

Let me repeat that, we were unlucky and still almost doubled our money.

21.06.2019

Job well done!

…one week later

As any good risk managers would, once the dust settled we did a back-test. Interestingly, the first finding from our back-testing was that it appeared that the lottery company made a $1.5 million blunder in calculating the winnings and they still owed us some money. I immediately wrote to the tech support asking for clarification.

A week more later, we finally figured out why there was a discrepancy. The bad news was, no, the lottery company did not owe us money, because they used a very obscure but nevertheless accurate logic to calculate winnings, the good news was that our model became much better because we did back testing. Risk management rocks!

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

P.S. By the way, this is how the MIT students, the Goldman Sacks employee and the family of retired mathematicians did it and earned good money.