Science

The Blank Field Became Mandatory On Monday, And Three Hundred Trials Cannot Fill It In Honestly

The field is worth having only if it is written before the result is known. Two hundred and ninety-four of the three hundred entries now on file were written after it.

The registry rule took effect on Monday. Every trial filed from that date must state its minimum clinically important difference: the number below which the authors agree in advance that a positive result would not matter. The field has existed since 2023 and was optional for three years, and in that time it was left blank on nearly every form that carried it, including the one behind the two of us. It is now compulsory. The registry is asking for the number three years after most of the trials it is asking about have already reported.

This is the correct rule. Both of us have said so in print, one of us at length, and neither of us is withdrawing it. The number is the only part of a trial that has to be committed before the result is known in order to mean anything, and a field that is optional is a field that gets filled in after the effect size arrives. What the rule did not anticipate is what happens to a register of three hundred trials when the requirement arrives late.

Three Hundred Entries, Two Hundred And Ninety-Four Of Them Late

The registry opened its first three hundred entries to public inspection on Tuesday. Every one of them now carries a number, because the form will not accept a blank. Two hundred and ninety-four of them are stamped with a submission time after the trial's result was posted. The number in those two hundred and ninety-four entries was therefore chosen by people who already knew what their effect size was, which is exactly the condition the field was designed to prevent, and there is no way to tell from the register which of them chose honestly.

"A number that is required after the result is a comment on the result. We now hold three hundred comments and two hundred and ninety-four of them are about a trial that had already finished." A registry methodological note, published with the inspection file

This is not an accusation and the registry has not made one. It has printed the timestamps next to the numbers, which is the only thing it can do and the thing that makes the file worth reading at all. A late number is not necessarily a dishonest one. It is a number that cannot be distinguished from one, and a register whose central column has that property across ninety-eight percent of its entries is a register measuring something other than what it says on the front.

The Two Of Us Disagree About The Threshold, On The Trial That Started This

The form in front of us has two fields and one of them is now mandatory. The trial is the one that produced the 0.4 millimetres of mercury finding, and the two of us have been asked to supply the number its preregistration left blank. We disagree about it, in writing, and the disagreement is now going into a box instead of into two headlines, which is the entire point of the rule and is also why it is going to take a week.

One of us says the minimum clinically important difference for systolic pressure is two millimetres of mercury, because two is the resolution of the cuff any clinician would actually use and a difference the instrument cannot report is not a difference anybody can act on. That number is defensible, it is standard, and it makes the trial a null result, which it was always going to be in a clinic regardless of its p value.

The other of us says the number cannot be set at all for a population study of eleven thousand four hundred people, because the question the trial asked was whether televised political coverage moves adult blood pressure, and for that question the minimum important difference is a distribution and not a point. A population effect of 0.4 is not nothing to a cardiologist at population scale; it is nothing to one patient in a room, and both of those sentences are the same sentence. Neither of us will sign the other's number.

"We have been asked to agree in advance on a threshold for a trial whose result is three years old. Whichever number goes in, we will both have signed it after the fact, and the registry will record that we signed it before." Anita Farnsworth, on the second day of trying to fill it in

What The Registry Is Actually Measuring

The rule has been in force five days. In that time the registry has collected three hundred numbers and published two hundred and ninety-four timestamps, and the one property those timestamps have in common is that none of them is a commitment. The compliance rate is one hundred percent and the useful rate is two percent, and both of those figures are printed on the same page by the same office in the same document.

This will resolve itself. Every trial registered from Monday forward gets the number written at registration, and in three years those entries will be the ones the file was built for, which is the ordinary way a good rule looks bad in its first week. The honest thing to say about this week is that the register is now measuring registration discipline rather than trial design, and that it is doing so accurately. The two of us will file a number. It will be one of ours and it will not be the other's, and one of us will be recorded as having agreed to something she did not.

Desk note on the byline: the disagreement above is real, is in both files, and was not created for the article. The order in which the two positions appear is alphabetical by argument and not by seniority, weight, or who is right, which is a question this desk has stopped putting to the two of us. The registry has been offered the note that the number is disputed by its own authors and has accepted it into the record.