Grade Your Own Goals Like a Superforecaster: A 4-Week Brier Score Log

Here is a number to carry around: 0.25. It is the Brier score you earn by answering “50 percent” to every question, the score of someone who has no idea about anything. Most people who start scoring their own goals, myself included, land somewhere near it or above it (worse) in the first month. That is useful news. It means your confidence has been decorative, and now you can measure by how much.

A Brier score is the average squared gap between the probability you gave and what happened. Forecasters use it to grade predictions. You can use it to grade the sentence “I’m definitely getting up at 6:30 this week.”

Where do Brier scores come from?

Glenn Brier, a meteorologist, proposed the measure in a 1950 paper in Monthly Weather Review, “Verification of forecasts expressed in terms of probability.” Weather forecasters were already saying “70 percent chance of rain,” and he wanted one number that rewarded them for being both right and appropriately sure.

The measure got a second career through Philip Tetlock, a psychologist at the University of Pennsylvania (earlier at Berkeley). His book Expert Political Judgment (Princeton University Press, 2005) reported a roughly twenty-year project in which hundreds of professional commentators and analysts made forecasts about politics and economics, which he then scored. The headline that escaped into the culture was that experts often did little better than simple baselines, and the “dart-throwing chimp” image stuck, though Tetlock has said that was a journalist’s flourish. The finding underneath it was more interesting. The experts who did worst were the confident ones with one big theory (Tetlock borrowed Isaiah Berlin’s “hedgehogs”). The better ones, the “foxes,” mixed several ideas and changed their minds.

Tetlock then co-led the Good Judgment Project, a team entered in a forecasting tournament run by the US intelligence research agency IARPA beginning in 2011. Thousands of volunteers forecast geopolitical events, and the top performers, a small slice of the pool, became known as superforecasters. Tetlock and the journalist Dan Gardner described what those people did in Superforecasting: The Art and Science of Prediction (2015). The habits were unglamorous — break a big question into smaller ones, start from how often similar things happen, update in small steps when evidence arrives, and score yourself afterward.

That last habit is the one we are going to steal. It is also the one almost nobody applies to their own life.

What is a Brier score, with actual numbers?

For a yes/no event, the formula has three parts:

1. Write your forecast as a decimal (60 percent is 0.6).
2. Write the outcome as 1 if it happened and 0 if it did not.
3. Square the difference. Average over all your forecasts.

Suppose you say 60 percent that you will hit a goal.

• It happens: (0.6 − 1)² = 0.4² = 0.16
• It does not: (0.6 − 0)² = 0.6² = 0.36

Now say 90 percent.

• It happens: (0.9 − 1)² = 0.1² = 0.01
• It does not: (0.9 − 0)² = 0.9² = 0.81

Squaring makes the scale harsh on confident misses. Being 90 percent sure and wrong costs 0.81, while being 60 percent sure and wrong costs 0.36. Confidence is a bet, and the Brier score bills you for it.

Three anchor points help:

Score	What it means
0.00	Perfect: every forecast was 100 percent or 0 percent and correct
0.25	Say 50 percent to everything; the score of knowing nothing
1.00	Perfect confidence, always wrong

(Brier’s original 1950 version summed over both outcomes and ran from 0 to 2. This post uses the 0-to-1 form common for a single yes/no question.)

Why bother scoring goals this way?

Because “I’m going to do it this week” is a forecast in a resolution’s costume. Behind that sentence sits a number, usually 100 percent, which is absurd for anyone with a track record of missing it. Writing the real number forces two useful things.

First, you find out how good you are at predicting yourself. Research on probability judgment going back decades (a review by Lichtenstein, Fischhoff and Phillips, 1982, is the classic one) found that people’s stated confidence tends to run ahead of their hit rate, especially on hard questions. The people in those studies who said “90 percent” were right noticeably less often than 90 percent of the time. There is no reason to think your wake-up forecasts are exempt.

Second, a probability changes what you do with a plan. If you write 55 percent next to a goal, the question “what would move this to 80?” arrives on its own. Maybe the answer is moving the alarm, telling a friend, or cutting the goal from five mornings to four. If you had written “definitely,” none of those thoughts would show up. Before scoring anything, you could also run a pre-mortem on the goal to generate the reasons it might fail, which gives you better inputs for the percentage.

How do you turn a goal into a forecast?

The conversion has two rules: it has to resolve cleanly, and it has to come with a number.

Resolve cleanly. “Be more of a morning person” cannot be scored. “I will be out of bed by 6:30 on at least 5 of the next 7 mornings” can, because on Sunday night you will know. Put the cutoff inside the sentence, and pick one that is a little inconvenient, since anything easy gets 95 percent and teaches you nothing.

Attach a probability. Pick a number between 5 and 95. Here is a useful trick from the forecasting world: ask yourself how many times out of ten you would expect to succeed if you lived this exact week ten times. Seven out of ten is 70 percent.

Then lean on your history before your hopes. Superforecasters begin with the “outside view,” meaning how often things like this actually go. For you, that is how many of the last four weeks you did the thing. If the answer is one, 85 percent needs a better reason than “I feel motivated.” The planning fallacy research describes why the inside view runs optimistic.

A few examples of the conversion:

• Instead of “go to bed earlier”: “Lights out by 11:00 on at least 5 of 7 nights. 65 percent.”
• Instead of “exercise more”: “Three workouts of 20 minutes or longer this week. 70 percent.”
• Instead of “cut screen time”: “No phone in bed on at least 6 of 7 nights. 90 percent.”

Write them in a notes app, a spreadsheet or a paper notebook, with the date. The only requirement is that the number is recorded before the week begins, because the mind is a skilled editor of its own memories.

What does a 4-week log look like?

Here is a log of one person with three weekly goals. The goals are wake by 6:30 on at least 5 of 7 mornings (A), three workouts of 20 minutes or longer (B), and no phone in bed on at least 6 of 7 nights (C). The probability is what they wrote on Sunday night. The outcome is 1 or 0. The last number in each cell is the squared error.

Week	A: wake by 6:30	B: three workouts	C: no phone in bed	Week’s Brier score
1	0.8, missed (4 of 7): 0.64	0.7, hit: 0.09	0.9, missed: 0.81	(0.64 + 0.09 + 0.81) / 3 = 0.513
2	0.8, hit: 0.04	0.7, missed: 0.49	0.9, hit: 0.01	(0.04 + 0.49 + 0.01) / 3 = 0.180
3	0.7, missed: 0.49	0.6, missed: 0.36	0.9, hit: 0.01	(0.49 + 0.36 + 0.01) / 3 = 0.287
4	0.7, hit: 0.09	0.6, hit: 0.16	0.8, hit: 0.04	(0.09 + 0.16 + 0.04) / 3 = 0.097

Across all twelve forecasts, the squared errors add up to 3.23, so the overall Brier score is 3.23 / 12 = 0.269. That is slightly worse than the 0.25 you would get from saying 50 percent to everything. I ran the arithmetic in a short script to be sure, because a post about scoring should not contain a scoring error.

Week 1 is the painful one. Two 80-plus percent forecasts missed, and the 0.81 for the phone goal alone outweighs everything else in the week. Week 4 looks great at 0.097, but it came after the person lowered their numbers to 70, 60 and 80, and they also happened to have a good week. One good week is not skill, and a dozen forecasts is a noisy sample.

What does the calibration check reveal?

The weekly score tells you how you did. The more useful analysis comes after four weeks, when you sort forecasts by how confident you were.

From the same log:

Confidence group	Number of forecasts	Average stated probability	How many came true
80 percent and above	6	85%	4 of 6 (67%)
Below 80 percent	6	67%	3 of 6 (50%)
All forecasts	12	76%	7 of 12 (58%)

If this person’s confidence matched reality, the 85 percent group would have come true about five times out of six, and the overall 76 percent average would have produced roughly nine hits out of twelve. They got seven. That gap, 76 percent stated against 58 percent achieved, is what overconfidence about goals looks like on paper. Forecasters call this calibration: of the things you rate 80 percent likely, do 80 percent happen?

Twelve forecasts prove nothing about a human being, and a different four weeks could have come out kinder. But the direction fits what the probability-judgment research predicts, and it replaces “I think I’m usually pretty reliable” with a number you can argue with.

After a month or more, look at goals one by one. A decent overall score can hide a perfect record on easy habits and a disaster on one hard goal, and the goal that keeps scoring badly is the one to rebuild. If wake-time forecasts keep missing by one morning, the target may be unrealistic for a week that includes a late shift.

How do you use the score to change your behavior?

A Brier score is a diagnostic, not a motivator, which I did not expect. Seeing 0.81 next to your name stings, but stings fade. What matters is what you do on Sunday night.

Superforecasters treated a miss as information: they asked what they had overlooked and changed how they forecast, instead of resolving to try harder. The personal version is a two-minute weekly review:

1. Which forecast cost me the most? (The highest squared error.)
2. Was it a prediction error (I was too sure) or an execution error (I was right about the odds and got unlucky)?
3. What number will I write next week for the same goal?

Often the biggest miss is a high-confidence forecast where the plan depended on willpower at the moment of choice. For those, change the odds instead of the number. Add a commitment device with a price attached. For wake-up goals, DontSnooze does this by giving you 30 seconds to record a video proving you are up, with a random camera-roll photo sent to your chosen friends if you do not. That may justify writing 85 instead of 60, but only if your log confirms it. If it does not, the number goes back down. The score does not care which tool you used.

Common mistakes when you start

• Forecasting effort instead of outcomes. “I will try to wake at 6:30” cannot be graded.
• Writing only 80 to 100 percent. Force a few 50s and 60s in, or you learn nothing about which goals are shaky.
• Moving the goalposts at week’s end. Four of seven is a miss against a goal of five. Change next week’s target, not this week’s grade.
• Playing it safe. Forecasting only easy goals at 95 percent scores well and teaches nothing. Include one that makes you nervous.

Quick start for this Sunday

You need a pen, a list of three goals, and ten minutes. Write each goal as a sentence that resolves to yes or no, put a probability next to it, and stick the page somewhere you will see it. Next Sunday, mark each one 1 or 0, square the gaps, and average them. Compare the average probability you wrote with the fraction that came true.

The first month will probably tell you that your confidence runs ahead of your follow-through, and the useful part is the size of the gap. It turns “I’m bad at this” into “I’m about 15 points too optimistic on early mornings,” which is a thing you can fix by writing a smaller number. If you have more goals than you can score, an 80/20 audit can pick the three worth forecasting.