A Paper Titled 'Alarm Clock-Based Reminder' Never Tested an Alarm Clock

Search for evidence that alarms help people take their medicine and one result looks perfect: Alarm Clock-Based Reminder for Improving Low Adherence on Option B Plus Antiretroviral Therapy Among HIV Positive Pregnant and Lactating Mothers in Northern Ethiopia. Its abstract ends by urging providers and policymakers to step up their efforts for HIV-positive mothers “using a clock as a reminder,” and the discussion says alarm clocks “helped to improve adherence.”

Nobody in the study was given a clock. It was a cross-sectional survey of 350 mothers in the Eastern zone of Tigray, run in January and February 2017 and published in HIV/AIDS (Auckland) in 2020 (Gebretsadik, Gebretnsae, Ftwi and Tesfahunegn). Interviewers asked about adherence and about how each mother reminded herself to take her pills. Mothers who said they used a clock reported good adherence more often (adjusted odds ratio 2.51, 95% CI 1.30 to 4.86). That is an association between two things people said, measured at one moment.

Who the clock users were compared with

The comparison group is the detail the abstract leaves out. Table 3 sorts mothers into two reminder types: a clock, used by 293 mothers (208 with good adherence, 85 with poor), and “others (friends and parents),” used by 57 (28 good, 29 poor). Roughly 71% of clock users were adherent against roughly 49% of the mothers who relied on other people. So the question the data answer is whether mothers who use a clock differ from the minority who depend on someone else to remind them. A mother who needs a relative to prompt her may already be in a harder situation, with less independence or less stable days. Other things in the same model moved with adherence too: formal education (AOR 2.78), travel time to the clinic (2.03), and male partner involvement (2.91). The variables that entered the model include nothing about household income or how regular her days are.

The authors offer a causal story: a fixed clock “is always with the mother,” while friends and partners “do not always live with” the mothers. It is a reasonable idea. The survey did not test it, and the same data fit one in which organized mothers are the ones who keep clocks.

What the paper does show

Two-thirds of the mothers (67.3%) reported good adherence, which the authors call low against the national target, and forgetfulness led the reasons for missed doses among those who did not (45% of 115). If forgetting is the commonest cause, reminders are an obvious thing to try. The paper stops at an association where a trial was needed.

Reading the numbers

What does an adjusted odds ratio of 2.51 mean here?

After the model accounted for the twelve other variables entered, the odds of reporting good adherence were 2.51 times higher for mothers who used a clock than for mothers reminded by friends or parents. The 95% confidence interval ran from 1.30 to 4.86, compatible with a modest difference and with one nearly five times larger. It describes how two groups of mothers differed and says nothing about what a clock would do for a mother who did not have one.

How was good adherence measured?

By self-report, using four questions adapted from work in South Africa for resource-limited settings. A mother counted as adherent if she gave the ‘good’ answer to all of them. The authors list recall bias as a limitation. Among the 115 mothers with poor adherence, the leading reasons were forgetfulness (45%) and lack of care (15.3%).

What would a fair test of an alarm clock look like?

Randomize mothers, or clinics, to receive an alarm clock or phone reminder versus usual care, and measure adherence with something harder to flatter than self-report, such as pill counts or viral load, and compare them with mothers who received usual care. The comparison is what turns a pattern into evidence.

What a trial looks like

The Kenyan WelTel trial (Lester and colleagues, The Lancet, 2010) randomized 538 adults starting antiretroviral therapy to weekly text messages from a clinic nurse or to standard care. Adherence above 95% was reported by 168 of 273 patients in the message group and 132 of 265 in the control group, and viral suppression was found in 156 of 273 against 128 of 265. Adherence there was self-reported and participants knew which group they were in, so even this trial has soft spots, but the viral-load assays were run by investigators who did not know the allocation. The messages also asked patients to reply within 48 hours, so they were contact with a person as well as a ping. Randomization and a control group are what let the authors say the messages did something.

Reminders have also been tested against a comparison group outside health, as in the New York City summons redesign and its text messages, where the effect on missed hearings could be measured. For pills, electronic pill-cap data suggest that the time slot a dose falls into is linked to how often it is missed, which is a separate lever from any reminder. Rogers and Milkman’s 2016 experiments assigned people to a written sign, an object cue or nothing, and the results are in the stuffed-alien study.

“People who use X do better” is the easiest sentence to write about any tool, and the people who choose a tool are rarely a random sample. That is as true of alarm apps as of alarm clocks. Read the methods before the title; words such as “improving” and “based” describe what the authors hope to show.