SealdayHow to get better at making predictions
You get better at predictions the way you get better at anything else: by doing it, getting feedback and adjusting. Write each prediction down with a date and a confidence, check it when the date comes, and look for patterns in your misses. A handful of habits does most of the work.
Coming soon to iPhone
What makes someone good at predicting?
Research on forecasting suggests that the best forecasters are not psychics or experts in one field. The Wikipedia article on superforecasting, based on Philip Tetlock's work, describes people who keep careful records, update on evidence and break questions into smaller ones.
You can borrow the habits without joining a tournament. For daily life the key is the loop of write, wait, check, learn. The more often you run it, the quicker your instincts improve.
What are the six habits?
- Write it down. An unrecorded prediction cannot be checked, and memory flatters you.
- Make it specific. A subject, a claim and a date: "I will still live here on 1 January".
- Start with the base rate. Ask how often this kind of thing happens before you think about your case. The base rate is the starting point.
- State your confidence. Sure, Maybe or Wild guess is a start. Percentages are better.
- Check on a fixed date. Do not check early and do not skip.
- Review the misses. Ask whether you were too sure, too shy or just unlucky.
How do I use base rates?
People overweight the details of their situation. If you ask "will I finish this book by March?", start with how often you have finished books in the past, not with how much you like this one. A related idea is reference class forecasting, which tells you to compare your case to a group of similar cases.
Then adjust for what is unusual. A friend who has run three races is more likely to run a fourth than a stranger is to run a first. The adjustment should be small unless the evidence is strong.
How do I score myself with a simple formula?
The Brier score is a standard way to grade probability forecasts. For one prediction, subtract the outcome (1 if it happened, 0 if not) from your forecast probability and square the result. Lower is better. The Wikipedia article on the Brier score explains the full definition.
| Forecast | Outcome | Calculation | Score |
|---|---|---|---|
| 90% it happens | Happened | (0.9 - 1) squared | 0.01 |
| 90% it happens | Did not happen | (0.9 - 0) squared | 0.81 |
| 50% it happens | Either way | (0.5 - 1) squared | 0.25 |
| 10% it happens | Did not happen | (0.1 - 0) squared | 0.01 |
Do I need percentages?
Not to begin with. Sealday asks you to tag guesses Sure, Maybe or Wild guess, and the tags show up on the scorecard. If you want to go further, write a percentage inside the guess text, as in "I'll be living in Lisbon (30%)". Sealday does not compute a Brier score, so do the arithmetic yourself in a note on open day.
Marks on open day are Called it, Kind of or Missed, with Kind of counting as a half in Table leaderboards. Compare your tags with your marks: if your Sure guesses miss often, lower your confidence next time.
What does a one-year practice plan look like?
- December: seal ten guesses for the next year, with tags.
- Each quarter: write three short guesses in a note and check them on the date.
- Next December: grade, add notes and count hits by tag.
- Adjust: lower your Sure tag if it missed, raise your Wild tag if it hit.
- Repeat for three years and compare the sets.
What mistakes do beginners make?
- Predicting what they want, not what they expect.
- Writing guesses that cannot be marked, such as "things will improve".
- Checking too early and changing their minds.
- Marking a miss as "close" every time.
- Throwing away the record after the first year.
What does a graded example look like?
After a few rounds the lessons form a pattern, and the pattern is your personal calibration. Most people find they are overconfident about work and housing, and underconfident about small habits and taste. Your pattern will differ, and that is the point.
| Prediction | Tag | Result | Lesson |
|---|---|---|---|
| I will still be in this flat | Sure | Missed. I moved in July | I was too sure about housing |
| The song on repeat will be a slow one | Maybe | Called it | I know my taste |
| I will run a 10 km race | Wild guess | Kind of. I ran 8 km | Goals need dates |
| The group chat's word of the year is a food | Wild guess | Missed | Fun, no lesson |
Where does a capsule fit in?
A sealed capsule is the review step you cannot skip, because the date arrives whether you feel like it or not and the guesses cannot be edited. It also lets friends play, which makes the loop social: in a Table everyone answers the same questions and the group scores them together.
Read what hindsight bias is for the memory problem the habit solves, and how to score your predictions for rules on partial credit.
Frequently asked questions
How can I improve my prediction skills?
Write predictions down with a date and confidence, check them on schedule and study your misses.
What is a base rate?
How often something happens in general. It is the right starting point before you adjust for your own case.
What is a Brier score?
The squared difference between your forecast probability and the outcome, averaged over predictions. Lower is better.
Does Sealday calculate a Brier score?
No. It records Called it, Kind of or Missed and shows a score such as 3 of 5. You can do the Brier arithmetic yourself.
How many predictions do I need to see a pattern?
Twenty to thirty is enough to notice whether your Sure tags are reliable. Two yearly capsules of ten get you there.
Should I predict with percentages or words?
Start with words such as Sure, Maybe and Wild guess. Add percentages once you have graded a few rounds, because they show how well your confidence matches results.