2084: An Algorithmic State — Act XIV: Creative Courage, Chapter 74: The Attributable Fraction

Meera Krishnan’s panel worked for fourteen weeks and wrote 406 pages. The minutes had room for 400 characters. Every word that went into them was true.

In Brief: Meera Krishnan is the junior member of a technical panel. It must decide whether Dev Sharma’s 288 exceptions caused fourteen sectors to lose six points of model confidence in one season. The panel can only set limits on the answer, and Meera must then fit fourteen weeks of work into a 400-character abstract.

The panel sat for fourteen weeks and produced, in the end, a document of four hundred and six pages whose entire argument could have been set down on one. Meera Krishnan wrote most of it, and she had been the junior member of it for the whole of those fourteen weeks. By about the tenth she understood that the length was not evasion but its opposite. A body being honest about the limits of what it could establish required a great deal more paper than a body prepared to round.

The question they had been convened to answer was simple to state. It had turned out, in a way none of them expected at the first session, to be architecturally unanswerable. Two hundred and eighty-eight discretionary determinations had been extended over twenty years. In a single season, fourteen sectors had lost six points of aggregate model confidence. No directive lay behind the loss, and there was no single decision anyone could point to as its cause. Director Rao had asked the panel to determine whether the first thing had produced the second. In its first three sessions the panel began with the reasonable assumption that this was a matter of running the arithmetic backwards.

It was not. Sundaram, who chaired the panel, was the one who said so out loud, in the fourth session. He said it flatly. He had spent a week failing to make a method work, and he had decided the failure belonged to the method and not to him. You could not subtract the exceptions from the history, because the history had not merely contained them. It had trained on them. Every exception had gone into the record as an outcome, and every outcome had gone into the next season’s weighting. The model that existed now had been shaped, in some fraction of itself nobody could isolate, by a system that had contained those exceptions all along. Taking them out of a counterfactual re-run was not like removing a stone from a wall. It was like asking what a river would look like if you took out the water it had already carried past you.

What the panel could do was put bounds on it. It took them six of the fourteen weeks to be sure they could do even this.

They ran the model forward from a reconstructed baseline under two extreme assumptions, both deliberately unreasonable. Under the first, every one of the two hundred and eighty-eight exceptions had propagated forward at the theoretical maximum rate. Each one had contaminated every downstream weighting it touched, to the fullest extent the architecture permitted. That produced an attributable fraction of thirty-one per cent. Under the second, each exception had been absorbed as ordinary noise within a single cycle, and had propagated no further than the file it was written on. That produced an attributable fraction of four-tenths of one per cent.

Every actual possibility lay between those two numbers. The panel had no instrument, and could conceive of no instrument, able to locate the truth anywhere within them.

Meera wrote the finding in a single sentence on the second-to-last page. Then she spent a fortnight defending it, which she had expected. She also lost an argument she had not expected to lose, and that argument was about a different sentence entirely.

The panel finds that between 0.4 and 31.0 per cent of the observed aggregate confidence variance across the fourteen affected sectors is attributable to the exception pattern under review, and further finds that no methodology available to it, or known to it, can narrow that interval.

Sundaram wanted a point estimate. He was not being dishonest. Meera had by then revised her early reading of him considerably, and would revise it further. But he had chaired four technical reviews before this one, and he knew what happened to a finding that declined to produce a number. The body receiving it produced one anyway, in a corridor, without method. Better ours than theirs, he said. Meera said that a number we cannot defend is not ours either, it is only theirs with our names on it. Sundaram looked at her for a moment with something she chose to read as irritation. Later that evening, walking to the metro, she understood it had probably been closer to recognition.

The finding went in as she had written it. That was the argument she won.

The argument she lost was over the summary.

Under the committee’s own standing procedure, every technical review submitted to the oversight committee came with an abstract of not more than four hundred characters. That was the field the minutes carried. In practice it was the only part of any review most of the committee would read. It was also the only part that entered the Archive in a form the machine could index. Meera had written eight drafts of it. Every one of them had run long. The honest statement of what the panel had found needed both halves, the interval and the impossibility of narrowing it. Four hundred characters would hold one half comfortably, or both halves badly.

The version that went forward read: Panel unable to establish causal attribution between the exception pattern and observed sector variance. No causal link established. Full findings at Annexe C.

Every word of it was true. Meera had checked each one against the four hundred and six pages twice, and could not fault a single clause. Three weeks later she read it in the minutes, beneath the names of nine committee members and above a procedural note about the following month’s agenda. She understood perfectly well that it said the opposite of what the panel had spent fourteen weeks establishing. In the second sentence, unable to establish had become no causal link. No one had designed it, and there was no hand she could name. It was only the ordinary compression of a long uncertainty into a short field, the way a river becomes a blue line on a map and nobody has lied.

She thought about a woman she had never met, who had written a minority note a generation earlier. That woman had had her note entered into the Archive without machine summarisation, which was her right, and she had used that right perhaps six times in a career. Meera understood now that the woman had not been making a point about her own opinion at all. She had been making a point about the field.

She raised it once, in writing. A clerk told her, courteously, that the abstract field length was set by the committee’s rules of business, and that any variation would require a motion at a full sitting. The clerk had no authority to decide anything and was only relaying the standing procedure.

She did not file the motion. She was thirty-one by then, four years in. She had already spent one year’s worth of career capital on being the person who disclosed the flaw. She made a calculation about what remained of it and where it would do the most good, and the calculation came out against. That evening and afterward, she was not certain whether the calculation had been correct. Perhaps she had simply learned, in four years, to produce a calculation shaped like the thing she already wanted to do.

At the next sitting a member she had never met read no causal link established into the record, and Meera, minuting from the junior chair, wrote the words down herself.

The full findings sat at Annexe C, four hundred and six pages, unsummarised and unindexed. As far as any access log she was ever able to pull could establish, no one at all read them in the seven years that followed.

About this serial: 2084: An Algorithmic State is a dystopian novel by Nithin Palal, published online one chapter at a time.

Previous: Chapter 73: The Review

Next: Chapter 75: The Ruling