Marguerite Asks a Biostatistician Whether Her 30-Day Experiment Proves Anything

This is a constructed conversation built on the published literature, not a transcript.

At lunchtime Marguerite sent her friend Tomasz, a biostatistician, a photo of her kitchen windowsill: a mug, a notebook, a thirty-row tally of wake-ups. A one-person experiment can tell you whether a sleep change works only when the effect is large, quick and reversible, and only if you switch the change on and off more than once. Moving a wake time mostly fails that test.

Marguerite: I moved my wake time to 6:30 for thirty days and I feel great. Proof?

Tomasz: Evidence, sure. Proof, no. Which month was it?

Marguerite: March into April.

Tomasz: Days lengthening, and you probably started it when you felt worst. People begin experiments at a low point, and low points drift back up on their own. That’s regression to the mean, and it flatters every before-and-after.

Tomasz: Plus novelty. New routines feel good for a bit. Medicine fixed this for single patients. In 1986 Guyatt, Sackett and colleagues at McMaster published “Determining optimal therapy” in the NEJM: one patient, active and comparison periods in pairs, order randomised. Kravitz and Duan’s 2014 AHRQ user’s guide is the modern manual.

Marguerite: So I’d do ABAB? Early week, normal week, early week, normal week?

Tomasz: Better to randomise the blocks, so you can’t pick the good weeks. And allow washout, the days before the old condition stops affecting you.

Marguerite: Fine. Washout for a wake time is what, a day?

Tomasz: That’s where you’ve picked a nasty one. General jet-lag and circadian guidance suggests the body clock shifts an hour a day at most, and usually less when moving earlier. Shift your wake time two hours and the first week is partly transition. Your A leaks into your B, which is called carryover, and it is the reason drug trials like conditions that switch cleanly.

Marguerite: Whereas coffee switches off in an afternoon.

Tomasz: Right. Take Seth Roberts, who ran self-experiments for years and published “Self-experimentation as a source of new ideas” in Behavioral and Brain Sciences in 2004 (volume 27, issue 2). Roberts himself framed self-experiments as a source of hypotheses, not proof: one subject, no blinding, and he knew what he hoped for.

Marguerite: You’re deflating me.

Tomasz: Gently! If your effect is big and fast, you can see it. If it needs three weeks to appear, you can’t tell it from the season.

Marguerite: So what do I do?

Tomasz: Three rules. One: decide in advance what number counts as success, say, minutes to fall asleep. Two: switch at least twice, in randomised blocks, never one big before-and-after. Three: pick changes that wear off fast, and treat slow ones, like mine and yours, as “worth keeping” rather than “proven”.

Marguerite: A month without an alarm is logged in this field log, and I’ll reread how the clock entrains, since mine is slow.

Tomasz: Bring data. I’ll bring biscuits.

Where Marguerite goes next

How many switches does an N-of-1 design need? Guyatt’s group paired each active period with a comparison period and repeated the pair several times, with the order randomised. One switch cannot separate your change from the calendar.

What can a sleep self-experiment still show? Effects that are large, arrive within a day or two and vanish when you stop, such as an evening coffee cutoff or a bedroom temperature change.

Should I skip the experiment, then? No. Treat a month of before-and-after data as a good reason to keep the change, not as proof it caused the improvement.

The written plan Tomasz describes, with the success number fixed before day one, takes about fifteen minutes to set down. A small one-week swap you could run is one alarm versus five.