AI-Proof: What Machines Can Never Be — Volume II: Application, Part IV: Conscious Creation, Chapter 18: The AI-Native Organisation

Part IV turns to what people make with machines. After the products of Chapter 17, this chapter asks about the organisations that make them: who decides, who answers and who is asked when the work is shared with machines.

← Previous: Chapter 17: Intelligence-Native Products · Series hub · Next: Chapter 19: Rethinking Roles and Structure →

Rajan’s new cameras rejected good cloth, and his best supervisor let them. The cameras did what they were trained to do. Nobody had said who could argue with them. An organisation can give a machine its work, but not its answering.

In Brief: For readers whose workplace is bringing in AI and who wonder where that leaves them. Chapter 18 is about organisations that hand their work to machines, and who answers when they do. A textile exporter in Coimbatore orders every system at once and watches his floor fall silent; an operations head in Hyderabad asks her teams to redesign their own work and is asked who will answer for it; a firm in Tallinn works without managers; a town in New Jersey closes its streets to navigation apps; a studio in Medellín builds a room for being wrong. Kauṭilya on the king’s council, Dennis Thompson on the problem of many hands, Frederick Taylor’s planning office and Amy Edmondson’s psychological safety help answer what an organisation owes the people who live inside its machines’ decisions, and who owes it.

An organisation can give a machine its work but not its answering: every decision a machine makes still needs someone who chose to let it decide, can take it back and answers for it, and a way for the people it falls on to be heard.

Opening: The Bin of Good Cloth

The bin at the end of the folding tables was meant for faulty cloth. In May 2025 it began to fill with good cloth.

The cloth came from Rajan Subramanian’s speciality line, finished by hand and still ordered by his oldest customers. Kamala, who is sixty-eight, finishes it at the far end of his floor in Coimbatore. Its surface is meant to be a little uneven. That is how a buyer can tell that a hand finished it. The new inspection cameras had learned what good cloth looks like from images of standard weaves, which are even. To the cameras, the speciality cloth looked faulty, and they rejected it, bolt after bolt.

Muthu, Rajan’s floor supervisor for nearly twenty years, watched the bolts go into the bin. He could feel a fault in running cloth before most men could see it. He said nothing. One of the young men he was teaching to set up a loom asked him why he let good cloth go. Muthu shrugged.

“The machine says it is defective. Who am I to argue with the machine?”

In November 2024 Rajan had refused almost all of this. His nephew Karthik had shown him the software, and taken the laptop home again. Through the winter Rajan used one module that checked suppliers, and listened on Sundays to what Karthik’s machines said about his trade. Then, early in 2025, three competitors announced large AI programmes: two mills in Tiruppur and one in Erode. Trade journals printed photographs of cameras over looms. Buyers began to ask what Rajan was doing with AI. Vendors called. Karthik sent articles.

On a Tuesday in March 2025, Rajan phoned Karthik and said: everything. Cameras to inspect every bolt. Forecasts of demand for every line. A scheduler to plan every loom. Letters to buyers drafted by machine. All of them at once, running by July.

Karthik was pleased, and so were the vendors. Nobody asked the 140 people on the floor. It did not occur to Rajan to ask them. He had run the business since 1997 by deciding, and the floor had carried out his decisions. The plan, Rajan’s July written into the vendors’ timetable, left no weeks in which anyone could have asked. And nobody, in any of the contracts, had written down which decisions the machines would now make, or who could overrule them.

The systems arrived in April, and they worked. The cameras caught broken threads and stains that tired eyes missed, especially on the night shift. The scheduler planned the looms more tightly than the floor ever had, and without anyone deciding it, it began to move loom time away from the speciality line. In June the forecast saw a buyer’s reorder from Tokyo coming weeks before Rajan did. Output per loom-hour rose.

The floor went quiet. In May the three inspectors whose work the cameras now did were moved to packing, at the same wage. By the end of June all three had left for mills in Tiruppur. The young men learning from Muthu stopped practising half of what he taught them: seeing a fault before it ran for a hundred metres. The camera now saw it for them. And Muthu let the bin fill.

This chapter asks: When an organisation hands its work to machines, who must answer for what they decide, and what is owed to those who live inside the decisions?

The print edition answered with a gap: “The gap between the metric and the meaning is the subject of this chapter.” When it explained the gap, it reached for consciousness. The floor’s errors were “a failure of consciousness”, and the good organisation was one in which consciousness was “distributed”. This edition does not need that word. Each failure on Rajan’s floor had a cause that someone could name, and the causes were of two kinds. The cameras and the scheduler were failures of specification: one was trained on other cloth, the other was given one measure. Muthu’s silence had another cause. The floor had learned who may argue with a machine, and that is a matter of authority and habit. What was missing, at each point where the work passed between a machine and the floor, was a person who could say no and would stand behind saying it.

The Prologue gave this problem a name borrowed from the political philosopher Dennis Thompson: the problem of many hands. In 2017 he put it plainly. “Because many different individuals in an organization contribute in many different ways to the decisions and policies, it is difficult even in principle to identify who is responsible for the results.” And more briefly: “The profusion of agents obscures the location of agency.”¹ A machine is not one of the hands that can answer. It can give reasons for its output, and as of 2026 often good ones. But when good cloth is lost, the money and the remedy must come from people, or from a company acting through them. The machine also brings more hands with it: the people who built it, sold it, recommended it and set it up. A camera vendor, a model’s makers, a nephew who advised, an owner who ordered and a supervisor who stayed silent all helped to fill the bin. None of them, alone, filled it.

The chapter keeps its title, and the title needs a word of argument. A native is born somewhere. No organisation is born with machines. Rajan’s was converted by one phone call, and most are converted more slowly. Here the title names a test, not a type. Was answering built in from the first day a machine decided anything, or added later, after something good had gone into the bin? Each of the five sections below is a place where it can leak away.

18.1 Smaller Teams, and What the Layers Held

To call smaller teams higher leverage, more output from each person, is to promise an advantage. When the layers of an organisation go, where does their work go?

Kadri runs a data-analysis firm in Tallinn, in Estonia. She founded it in 2023. It does made-to-order analysis for mid-sized European companies, and it has fourteen people. She reckons that a firm doing the same work a few years earlier would have needed fifty or sixty. That is her estimate, and the machines are her reason. Three of the fourteen judge the machines’ analysis and sign the recommendations that go to clients. Two work with the clients. Four design the frameworks that make an analysis mean something to a business. Five build and look after the AI systems. There are no middle managers.

The machines do the gathering, the sorting and much of the first analysis, faster than any team of juniors did. Kadri’s clients get their results in days. That gain is real, and much of it is hers: a firm of fourteen pays fourteen salaries.

Kadri hires with one question: “Tell me what you would do in this role that an AI system could not.” It sounds hard-headed. It is also the move this book refused in its Prologue, defining a person by what machines cannot yet do. Every reply to it carries a date. A candidate who replies well in 2025 may have nothing to say in 2027, with no change in herself at all. A question that would still mean something in ten years is different: what in this role would you answer for, and what would you refuse to sign?

Kadri’s own story about the firm is more honest than her question. In its second year, 2024, two analysts left. They had come from large consultancies, where the work was structured, approval passed through many hands, and the layers above them absorbed whatever was unclear. At Kadri’s firm the unclear was theirs. One left for a larger firm within three months. The other asked to be managed more closely. Kadri told him that there was nobody to manage him; the firm had been built on the idea that each person would make their own calls. He thought about it for two weeks and resigned.

Kadri does not tell this as a story of the weak leaving. She tells it as a fact about design. Some good professionals do their best work inside a structure. For them, the loss of layers is not freedom but vertigo, the dizziness of a high place with nothing to hold.

Machines take much of what the middle layer used to do: the gathering, shaping and passing of information up and down. That part is real. The two analysts show what else the layers held. They absorbed ambiguity: someone above decided what an unclear brief meant. They spread responsibility across many people, so that a mistake was rarely one person’s. And they trained people. Kadri hires only the experienced. Where will a firm’s future seniors come from, if it hires no juniors? For a firm of fourteen, the honest reply is: from other firms, which must still be hiring them.

Fewer layers is not the same as less management. Someone still has to settle disagreements, share out the work, coach the newest person and decide when an unclear case needs a second pair of eyes. At Kadri’s firm those jobs did not vanish. They moved to the fourteen, unnamed and unpaid for. And clarity has a cost of its own. Kadri’s three signers know whose recommendation it is, which is good while each has the time, and a colleague, to check it. A signer with neither gains only a clear view of whom the blame will fall on.

An ancient Indian treatise on statecraft argued about the size of a council long before anyone argued about the size of a team. The Arthaśāstra, the manual of government attributed to Kauṭilya, records earlier schools that fixed the king’s council of ministers, the mantripariṣad, at twelve, sixteen or twenty members. Kauṭilya rejects all three numbers. In R. Shamasastry’s translation, his own view is that the council “shall consist of as many members as the needs of his dominion require”. For the small circle in which the king deliberates, he gives reasons. One minister alone acts as he pleases. Two may join against the king, or endanger him by quarrelling. “But with three or four ministers he will not come to any serious grief, but will arrive at satisfactory results.”² The text is about a king’s advisers, not a firm’s staff, and I borrow only its refusal of a fixed number. Size should follow the work. The fashion for small teams, like the older fashion for large ones, is a number chosen before the work is known.

What the layers held does not vanish when they go. It moves to each person’s desk, and often into their evenings. But a firm of fourteen is a choice for a firm of fourteen. Rajan has about 140 people, and a pharmaceutical company may have thousands. For them the question is not how many layers to keep. It is how each handoff between machine and person is designed. And who stands behind it.

18.2 Hybrid Human-AI Workflows

The order of work between people and machines is always someone’s design. In a private hospital in Mumbai, the radiologist Sanjay reads ten images himself each morning before he looks at the machine’s flags. Nobody asked him to. The hospital chose the order of work: the machine reads first, the doctor confirms. Sanjay changed one corner of it, on his own time, so that his eye would stay able to check the machine. Someone decides who does what, in what order, with what authority, and often that someone is far from the work.

In 1911 the American engineer Frederick Winslow Taylor set out the rule of his Principles of Scientific Management: “In the past the man has been first; in the future the system must be first.”³ His planning office designed each worker’s movements, and the worker carried them out. Many machine rollouts are Taylor’s method with a new tool. The work is redesigned above and carried out below. Rajan’s phone call in March was one of them.

Plot twist

Taylor is remembered for putting the system first. The sentences that follow in the same introduction are quoted less often. “This in no sense, however, implies that great men are not needed,” he wrote. “On the contrary, the first object of any good system must be that of developing first-class men”. In his second chapter he listed four new duties of management. The second was to “scientifically select and then train, teach, and develop the workman”.³

Taylor’s training was narrow. It taught the method the planning office had chosen, not the judgement Muthu was teaching at the looms. But his system made the training of workers the planners’ duty. Rajan’s rollout did not. Nobody on his floor was given the task of teaching the young men what the cameras now saw for them. In this one respect, many rollouts fall short of Taylor’s method.

Meera Reddy had spent two years learning the price of that method. She is Vice-President of Operations at NovaChem Pharmaceuticals’ site in Genome Valley, outside Hyderabad. Chapter 16 told how, from March 2023 to March 2025, she rebuilt the company’s machines into its work after a first plan failed. A chemist, Padma, had told her that the first system was “not ours”. The rebuild made the tools fit the work. By March 2025 Meera could see that fit had not given anyone a say. The teams complied and offered nothing.

So in April 2025 she did not draw the new workflows herself. She gave each team three questions and four months. At each step, who is responsible, the person or the machine? Which cases go to a person, and who decides which? When a person corrects the machine, where does the correction go? Then she asked each team to redesign its own work. What she wanted was a relay: at every point, the work in one pair of hands, and everyone knowing whose.

Shalini, the customer service director, has been at NovaChem for eight years. At the first meeting she asked a question that Meera had not prepared for. “If my team designs this, who answers when it goes wrong: my team, or you?”

Meera said that they would work it out as the designs came in. That was honest. It settled nothing.

By June, Shalini’s team had built a routing rule. The machine listened to the first part of each call and estimated how upset the caller was. Agitated callers went straight to the most experienced agents, not to the general queue. The agents knew from years of calls how much an angry first minute could cost if it was left to wait. The design did what they wanted. Angry callers waited less, and fewer calls ended in complaints about being passed around. That call went the machine’s way. The psychologist Lisa Feldman Barrett and her colleagues found that a face is an unreliable sign of what a person feels.⁴ The machine’s reading of a voice deserved the same caution. But the larger problem was not the reading. It was what the rule rewarded.

In July 2025 a hospital pharmacist in Warangal called about a late delivery of a cancer drug. A patient needed it within two days. She was calm; she had made such calls many times. The machine scored her as calm, and she waited in the general queue behind a dozen callers who were angrier and less urgent. The agent who took her call handled it well, and the drug arrived in time. Nothing went wrong that anyone could count. But nobody had decided that calm callers with urgent needs should wait. The rule had decided it, and the rule was the team’s.

Who must explain that, and put it right? Shalini’s team designed the rule and knew what it measured. Meera approved it. The vendor’s model claimed to read agitation, and the vendor sold it on that claim. And the India managing director had set a target for calls handled per hour, which made speed for the angriest callers look like the whole job. Thompson’s view is that each answers for the part they knowingly and freely gave: “to assign responsibility for an organizational outcome fairly and effectively, we have to identify individuals who knowingly and freely contribute to it.”¹ That divides the burden among them. It does not abolish it.

Asking the teams had given them a voice in their own work, the thing Chapter 16 found missing. It had not told anyone who would answer. Participation can even blur that. A design made by everyone can end with nobody’s name on it. And the team nearest the work can become the place where the blame lands, for a model and a target it did not control. The researcher Madeleine Clare Elish calls that place a moral crumple zone.⁵

From August 2025 Meera added one rule to every redesigned workflow. Each handoff got one named person. Call these handoffs seams: the points where work passes from a machine to a person, or from one person to another. For the routing rule, the named person was Shalini. She had the authority to change the rule without asking upstairs, and an hour a week to listen to the calls it had sent to the back of the queue. Her team added a second rule that did not depend on tone: any caller who names a patient, a medicine and a date goes to a trained agent at once. That was better, not complete. An urgent caller may not know the date, or may be calling for someone else. So once a month Shalini spent her weekly hour on a sample of the calls the rule had left in the queue. She listened for the urgent ones it still missed. Chapter 7 gave four conditions under which a position is full. A seam is a position like any other, and it empties the same ways: a name without the time or authority to act, or authority with no name attached.

A seam can also be well held and still fail, because its target is set somewhere else. Shalini could change the routing rule. She could not change the calls-per-hour target, and it was the target, more than any rule, that made the angriest callers the whole job. Chapter 17 argued that before anyone asks whether harm will be found and put right, someone must say who chose what counts as success, and for whom. A seam comes after that choice too, and here the managing director had made it. So the named person stands behind the seam, and the people above stand behind the targets, the staff and the hours the seam works within. Someone must also look at all the seams together and ask whether, taken whole, they produce an outcome the organisation could defend to the people it serves. At NovaChem’s site that was Meera’s work. A name at every handoff did not do it for her.

18.3 Decision Velocity: Fast Decisions, Slow Harm

It is easy to ask for speed with awareness. A real case says something better.

Leonia is a borough of about 9,000 people in New Jersey, near the George Washington Bridge into New York. By 2017 navigation apps were sending drivers who were heading for the bridge through its side streets whenever the main roads were jammed. Each route made sense for its driver: a few minutes saved. Together, at rush hour, they filled streets built for the people who lived on them. The town’s police chief described the problem simply: “We are a small town in a very busy area with a very small police force.” From 22 January 2018 the borough closed its side streets to non-residents during the morning and evening rush, with a fine of $200 after two weeks of warnings. Once the closures took effect, the apps stopped sending drivers that way. In August 2018 a state judge struck the ordinance down, because the state’s transport department had not approved closures that touched a state road.⁶

The apps were not slow to learn. Told that the streets were closed, they obeyed at once. What they had never been told, until a town passed a law, was that people lived on those streets and counted.

The drivers had a case too. The streets were public, open to a driver from the next town as much as to the people who lived there. The judge did not rule that the residents did not count. His ruling was about who may approve such closures. Neither side owned the streets. What nobody had done was weigh what thousands of reasonable routes did to the people who lived along them.

Notice the shape of the harm. No single decision caused it. Each route could be defended, and the harm lived in the sum, which nobody decided. Thompson’s problem was many hands contributing to one result. This is its twin: one system making many decisions, each too small to notice on its own. And the people who bore the sum were not the apps’ customers. They had no seat anywhere in the decision. Their only voice was the law, and the law moved in months while the routing changed by the minute.

Notice too what a quick fix cannot do. Changing the routes tomorrow does not give the town back today’s traffic. So the people outside need more than a door to knock on. Someone inside must be named to watch the sum, street by street, before enough people are hurt to complain.

Speed is a real good, and waiting has costs. But waiting is not empty. It can buy time to check, to ask the people affected, and to see the sum of many small decisions before it sets. So the pace of a decision should match three things: how much is at stake, how far the decision spreads, and how far it can be undone. A route offered to one driver is a small act. The same route offered to ten thousand drivers at once is a different act. One route can be undone by the driver. A pattern of ten thousand routes through a town can be undone only by the people who set the system, and only if someone tells them.

So a fast organisation needs two slow things. The first is a person who sorts decisions into those that may run fast and those that may not. That sorting is itself a decision, and it needs a name. The second is a way for the people outside to be heard before they must pass a law. As of 2026, a machine can watch such patterns more closely than any weekly human glance. What a person adds at that seam is not sharper eyes. It is a name that will explain and put right. It is also a way in for the people outside.

18.4 Experimental Culture: Permission to Be Wrong Consciously

The title stays, and so does its hardest word. Here “consciously” means knowingly. You know that you are experimenting. You know what result would show you wrong. You know what the error will cost, and who will pay. An experiment that nobody planned to learn from is only activity.

Medellín has long been Colombia’s textile city. Coltejer, one of the country’s oldest and largest textile firms, was founded there in 1907.⁷ In a converted mill in the city, Luciana runs a studio of nine people that designs software interfaces for shops and small firms. In March 2025 she started what she calls the wrong room. Every Thursday afternoon the team meets there to show ideas that are probably wrong. The condition of entry is real uncertainty. If you are sure an idea is good, take it to the ordinary meeting. If you think it might be good and do not know, and the not-knowing is itself interesting, bring it here.

The wrong room nearly died in its third month. In June 2025 Daniela, a junior designer, showed a way to browse a shop’s catalogue that broke the usual grid of products. She had built it on her own evenings. She said she was nervous, showed it, explained her reasoning and waited. The room was quiet for a few seconds. Then the senior developer spoke without looking up from his laptop. He was the most technically gifted person in the studio, and his opinions carried weight because he was usually right. “That is technically naïve.”

He moved on to the next item. Daniela did not speak again that afternoon, and did not come back to the wrong room for six weeks. Two others quietly stopped coming.

The Nyāya logicians of classical India named kinds of debate. Vāda is discussion between people who both want the truth. Vitaṇḍā attacks the other side’s position without defending any position of its own.⁸ “Technically naïve” was close to what they called vitaṇḍā, shrunk to two words: a verdict that defended no position anyone could test. It may even have been right. Nobody could tell, because it gave them nothing to check.

In 1999 the American organisational researcher Amy Edmondson defined what the room had lost. She called it team psychological safety: “a shared belief held by members of a team that the team is safe for interpersonal risk taking”.⁹ A few years earlier she had studied medication errors in hospital teams. The teams with the most open leaders recorded more errors, not fewer. She found that the teams differed “not just in the frequency of errors, but also in the likelihood that errors will be detected and learned from by group members”.¹⁰ A team that punishes error may not have fewer errors. It may only have fewer reports of them.

Receipts, please

The hospital study behind that last sentence is famous, and small. Edmondson worked inside a larger study of errors in giving medicines, run by two physicians. She looked at eight hospital units.¹⁰ Units whose nurse managers gave stronger direction and coaching, and whose members rated their relationships better, had significantly higher rates of detected errors.

Three limits matter. The errors counted were those detected, so the study could not show how many errors each unit really made. Eight units is a small number for a finding about teams. And the reading that open units reported more, rather than erred more, was Edmondson’s explanation of a puzzle. She supported it with a blinded researcher’s descriptions of how each unit worked, not with an experiment.

My reading. The study supports the word “may” in the sentence above, and nothing stronger. It is a reason to read a low count of errors with care. It is not proof that a quiet room is hiding mistakes.

In August 2025 Luciana made one rule. In the wrong room, the reply to any idea is a question, never a verdict. What would happen if…? How would we test it? What are you least sure of? The rule did not make anyone kinder. It made the room’s talk checkable. A question asks for something the idea’s author can go and find out, and that moves the room from vitaṇḍā towards vāda. The rule is for ideas whose worth nobody yet knows. An idea that is unsafe or unlawful, or would harm the people who use the shop, gets a boundary, not a question. Luciana said so the day she made the rule. The senior developer did not apologise in front of anyone. He started asking questions, some of them sharp. Daniela came back that month.

The studio’s AI tools let her build four versions of the design in two weeks, and test each against simulated users: programs that imitate how people move through a screen. In 2025 the tools could do in an afternoon what a month of building had done a few years before. The simulated users found the new layout fast.

Early in 2026 the team put the best version in front of twelve customers of the shops it was for. Most of them got lost in it. The simulated users had been built from people who already shop online often, and many of these customers did not. A simulation is good evidence for the questions it can answer, such as which of four layouts is faster for people like those it was built from. It could not say whether these shops’ own customers would find their way. The idea failed with the people it was for. The room wrote down why, and what it had learned about its own simulations.

Feeling safe is not the same as having good evidence. The wrong room needed both: a place where uncertainty could be said aloud, and a way to find out which claims the evidence supported.

The print edition ended this story with Daniela’s idea becoming “a significant share of the studio’s revenue”. That ending proved nothing, and this one proves nothing either. Judging a decision by its outcome is what psychologists call outcome bias.¹¹ A room for being wrong is not shown to work by one idea winning or losing. It works if wrong ideas are found early, knowingly and cheaply, and if the people who bring them come back.

And “permission” names something easy to miss. Permission is given by someone with authority. In the wrong room, it meant that Luciana would face her clients, and pay, for the hours spent on ideas that failed. After August the room’s experiments were done in studio hours, not on Daniela’s evenings. That was Luciana’s decision, at the studio’s cost. In an organisation, permission to be wrong is someone’s agreement to answer for the cost of the error. Without that, it is only encouragement.

So the section’s old title holds, read this way: experiment deliberately, and know when you are doing it. Machines have made being wrong cheaper. An idea can be built and tested in days. They have not made it free, and they have not changed who pays.

18.5 Change Management: Leading Humans Through a Machine Transition

Rajan did not notice the silence by instinct, as the print edition had it (“His body said it was worse. He trusted his body.”). What he had in June 2025 was evidence. The cameras’ own weekly report showed the speciality line rejected at three times the rate the inspectors had rejected it before April. Three letters of resignation lay on his desk. Two of the young men at the looms had stopped staying after the shift. And Muthu, who had argued with him for nineteen years, had stopped arguing. The figures showed him where to look. They could not tell him why.

Some of Muthu’s silence may have been grief. There were ordinary reasons too. Nobody had told Muthu that he could overrule a camera. Nobody would have thanked him for it. And on most of the cloth the camera was right more often than a tired eye, so a supervisor who argued with it would look stubborn. In the arrangement Rajan had made, silence was the safe choice. Muthu’s own sentence, “The machine says it is defective”, made the camera the doer, as language lets us say that the axe cuts.

That spring had tested a smaller promise too. In January Rajan had told a young owner in Erode that he would come back in three months. The page in his diary came round in April, the month the systems arrived. He went in late May, a month late. The loading bay was cleaner. He gave the unit a small overflow order, due in December. Whether he was being fair, or making up for January, he did not know.

One Saturday afternoon in August, after the shift, Rajan called everyone together. Not the supervisors: everyone. Everyone on the floor sat on crates and upturned baskets, because there were not enough chairs. He stood in front of them and said something he had never said to his floor.

“I made a mistake.”

He told them what he meant. The machines were good at much of what they did. The mistake was that he had decided for the floor without asking it. He had treated a change in their work as his decision about them. He did not tell them that the machines were not there to replace anyone. The print edition had him say that, and then, a few minutes later, refuse to promise it. Then he asked them to speak.

For a long while the questions were cautious and practical. Then Parvathi stood up. She had worked at the folding tables for thirteen years, beside the three inspectors.

“You say you made a mistake. The cameras took the inspectors’ work. They were moved to packing. They were good workers. They did not deserve to be moved. Now they are gone. How do we know the same thing will not happen to us?”

Rajan could have said that their jobs were safe. He did not, because it might not be true, and everyone in the room would have known it.

“I do not know,” he said. “I do not know what these machines will be able to do in two years. Nobody does.”

About fifty kilometres away, in a garment factory in Tiruppur, two inspectors named Selvi and Sumathi had been moved to packing the same year, at the same wage. They kept their pay and lost their craft. Rajan’s three did something else. The economist Albert Hirschman named two ways people respond when an organisation fails them. Exit is leaving; voice is telling someone who can change it what is wrong.¹² On Rajan’s floor they became possible in order. Exit came first, for the three, because some mills in Tiruppur, without cameras yet, still paid for a good eye. For Muthu, after nearly twenty years on one floor, exit was costly, and voice had not been safe. Parvathi could stand up that afternoon because Rajan had spoken first, and had said that he was wrong.

The philosopher Hannah Arendt called the promise the remedy for an uncertain future.¹³ A promise does not make the future safe. It makes one part of it something others can rely on. Rajan could not promise jobs. He could promise what was in his power, and he did. Nobody would again be moved off their work without being asked first and told why. And before any new system came onto the floor, the people who would work beside it would see it first, and say what they thought it would get wrong. Both promises looked forward. The first was what the three inspectors had been owed in May. Nothing he said that afternoon brought them back, or the good cloth that had already gone out cheaply as rejects.

Then they talked for two hours, and the floor named the small daily things the systems got wrong. The cameras rejected the speciality cloth because they had learned only from even weaves. The scheduler ignored the order in which Muthu’s men change a loom from one cloth to another, which keeps each changeover short. It sent looms from fine yarn to coarse and back in a day. And its plan for October gave full shifts to the week of Deepavali, as Diwali is called in Tamil Nadu. The festival fell that year on 20 October, when half the floor would be travelling home.¹⁴ Its date is set by the moon, and in 2024 it had fallen on 31 October. A plan that learned from last year’s months put it in the wrong week.

These three were failures of specification. The cameras were trained on other cloth; the scheduler was given one measure and last year’s calendar. Each could be fixed once someone with the right to change the system heard about it, and each was. Nobody had tried the cameras on the speciality cloth before they went in; the vendor retrained them in October, and Rajan and the vendor split the cost. The changeover order became a fixed rule in the scheduler. The week of Deepavali was planned thin. The silence that had kept these faults unheard for four months was a different kind of failure. Nobody had specified it. The floor had learned it from an arrangement in which arguing with a machine had no route and no reward. An arrangement like that is changed by whoever holds authority, not by retraining a camera.

Kauṭilya would have recognised the meeting. In the Arthaśāstra’s chapter on councils, the king “shall despise none, but hear the opinions of all. A wise man shall make use of even a child’s sensible utterance.”² The king listens widely because listening makes his decisions better. But he still decides. When an emergency brings his whole council together, the text says: “He shall do whatever the majority ... of the members suggest or whatever course of action leading to success ... they point out.” The king acts on their advice, and the acting is his. And Kauṭilya gives him a measure of his own good: “In the happiness of his subjects lies his happiness; in their welfare his welfare; whatever pleases himself he shall not consider as good, but whatever pleases his subjects he shall consider as good.”¹⁵ Kauṭilya wrote for kings whose subjects had no vote, and I borrow only his duty to listen and his measure. By that measure, a rollout that raised output while the floor went silent was not yet good for Rajan either.

Thompson would read the same afternoon differently. “I made a mistake” was honest, and it cost Rajan something in front of his whole floor. But in 1980 Thompson wrote about leaders who take all the blame. “With regular incantations of ‘I accept full responsibility,’ an official strengthens his or her own political standing,” he warned, and “the ritual often quells public debate about a controversial decision or policy, effectively blocking further inquiry into the genuine moral responsibility of all of the officials involved”.¹⁶ Rajan was no official, and his words were not a ritual. But they put the whole mistake on him, and so they let the other hands go unexamined. The vendor had sold cameras trained on cloth unlike his without asking what he wove. Karthik had advised everything at once. Muthu had let good cloth go. By Thompson’s test, which asks what each person gave knowingly and freely, Muthu’s share was real and small. He knew the cloth was good. Nobody had given him the right to overrule a camera.

Here the two lamps disagree. Kauṭilya places answering in the one who rules, who listens as a duty of prudence and acts on what he hears. Thompson says that in an organisation no single person can honestly hold it all. It has to be shared out in proportion to what each person knew and could change, and it has to be designed in advance. Kauṭilya’s own text softens the contrast, since his king acts on the majority’s advice. But it does not move the burden. For the future, Thompson proposes what he calls “prospective design responsibility”: “we designate specific individuals or groups of individuals as overseers, who can be held accountable for monitoring the structure of the organization and making or recommending changes in it as necessary.”¹ Rajan’s floor needed both lamps. It needed an owner who listened and decided, and it needed responsibility designed into its seams.

That is what Rajan did next, at the smallest scale. He named Muthu as the person to whom any camera reject could be appealed. Muthu would look at every rejected bolt from the speciality line, and at any other bolt a worker asked him to look at. He would decide whether it went to the buyer or the bin, and write one line of reason for each decision in a log. Rajan said that an overrule with a good reason would never be held against him, even if it turned out to be wrong. The log took Muthu half an hour at the end of each shift. Rajan took that half hour out of Muthu’s other paperwork, not out of his evenings. Here, at one seam, were the conditions of a full position, written into a supervisor’s day. They can lapse while the name stays. The half hour may be swallowed by other work, or the overrules that turn out wrong may be counted against him. Then Muthu will still be the named appeal, and the seat will be empty.

And every Friday afternoon, inside paid shift hours, the people who worked most closely with the systems sat with Rajan to say what the machines were doing well and badly. The floor called it, in plain Tamil, the Friday meeting. Karthik, who had left in July for a venture firm in Singapore, joined by video for the first few weeks, then only by his Sunday calls. Each meeting began with what had happened to the points raised the week before: which had been done, which refused, and why.

Be exact about what the meeting could change. Through Rajan, it could change how a system was used: the cameras’ training, the order of changeovers, the plan for a festival week. It could not change whether the systems had been bought, which one phone call had settled in March. It could not change where their gains went. And it could not decide. Rajan kept the decision, as Kauṭilya’s king did; the difference was that he now heard the reasons before he made it. A voice is not a vote, and a hearing is not a say in the decision. Voice is a claim to be heard before a decision, and to be told why after it. That is less than a share in the deciding, and it should not be called more.

Some laws mark the same line. Germany’s law on works councils¹², bodies that employees can elect at their workplace, gives both. The employer must tell the works council in good time about plans for working procedures, including the use of artificial intelligence, and must consult it. That is a hearing. For technical devices designed to monitor the employees’ behaviour or performance, the council has a right of co-determination. The employer cannot decide alone. If the two sides cannot agree, a conciliation committee decides. That is a say in the decision. Nothing in Rajan’s two promises gave his floor anything like the second.

A month later, over the production reports, Muthu said to Rajan without being asked: “The machine is not wrong about the numbers, sir.” A supervisor who had said nothing to Rajan for four months was arguing again, and this time for the machine.

Output stayed up, and the gains were Rajan’s. That was not something the cameras decided. The extra output could have gone into wages, shorter shifts, lower prices for his buyers or the young men’s training. Rajan kept it, and nobody on the floor was paid more. He had a claim to some of it. He paid for the systems, and he would have carried the loss if they had failed. That gives him a claim to a return on what he risked. It gives him no claim to the part of the gain that came from costs others carried: the inspectors’ places on his floor, and the practice the young men lost. He carried some costs: his half of the cameras’ retraining, the Friday hours, Muthu’s half hour, two promises he must now keep. Beside the gain they were small. The Friday meeting worked, in the sense that the floor was heard and the systems were fixed. That does not make the arrangement fair, and the chapter offers the output as proof of neither. Those who choose a change and take its gains should carry most of its cost. On Rajan’s floor, so far, they carry a little of it.

Some of the cost is still unpaid. The four young men learning from Muthu now learn to set up a loom, but not to see a fault in running cloth, because the camera sees it. That is the loss this book calls a missing rung. Nobody has decided whether they should still learn it, or who would pay for the learning. Kamala’s debt is open too: nobody else on the floor can finish the cloth as she does, and nobody has decided who will learn.

Ending: Four Bolts, One Name

By November 2025 Muthu’s log had three months in it. On the speciality cloth, most of his overrules had held when the buyers saw the cloth. On standard cloth, the camera had been right more often than he was, and by October he had stopped overruling it there. A kept record can become a rung for someone learning. Whether this one will, for the young men, is not settled.

Rajan’s Hamburg buyer, Jürgen Weiss, had still not had an answer from his lawyers about Clause 11 of his new contract. In November 2024 Rajan had asked him to put a person’s name back into that clause. He did not want a machine’s inspection report to be final on his cloth. On his own floor, for four months, the camera’s report had been final. Now it was not.

At the end of a shift late in the month, the bin held four bolts. Beside each, in Muthu’s hand, was a reason. Two of the reasons would turn out to be wrong. All four had his name on them.

Practice

Find the seams. Choose one piece of work in your organisation that passes between a machine and people. It might be a reject, a score, a route, a schedule or a reply to a customer. Follow it from start to finish, and list every point where it passes from a machine to a person or from one person to another. At each seam, write four things. Who chose to let the machine do this step? Who can stop it or take it back? Who answers if it goes wrong? Who does it fall on, and how can they be heard? A seam with no name, or with “the system”, is an empty position. So is a seam whose named person has no time or authority to refuse. Then write who set the target the seam works towards, and whether anyone looks at all the seams together. Choose one empty seam and write whose hours it would cost to fill it, and who should pay for them. If you can raise it safely with the person who could fill it, do; if you cannot, keep the page and write what stops you. This will not make your job safer, and it promises nothing about pay. It shows you where answering has gone thin.

Words to keep

• exit and voice: Hirschman’s two answers to an organisation that is failing you: leaving it, or telling someone who can change it what is wrong.

• mantripariṣad: in the Arthaśāstra, the king’s council of ministers.

• moral crumple zone: the person in an automated system who takes the blame when it fails, though they had little control over it (Elish).

• overseer: in Thompson’s sense, a person named in advance to watch how an organisation’s work is arranged, and to change it or ask for change when it fails.

• problem of many hands: when many people contribute to an outcome in different ways, so that it is hard to say who is responsible for it.

• prospective design responsibility: Thompson’s term for responsibility for how an organisation will work, assigned before anything goes wrong.

• seam: in this book, a point where work passes from a machine to a person, or from one person to another.

• team psychological safety: Edmondson’s definition of a team’s shared belief that it is safe to take risks with one another, such as admitting an error.

• vāda: in Nyāya, honest discussion between people who both want the truth.

• vitaṇḍā: in Nyāya, attacking another’s position without defending any of your own.

Sources

1. Dennis F. Thompson, “Designing Responsibility: The Problem of Many Hands in Complex Organizations,” in Jeroen van den Hoven, Seumas Miller and Thomas Pogge (eds.), Designing in Ethics (Cambridge: Cambridge University Press, 2017), pp. 32–56 (author’s manuscript, Harvard DASH). The phrase “the problem of many hands” is from his article of 1980; see the Prologue.

2. Kauṭilya, Arthaśāstra I.15, “The Business of Council Meeting,” trans. R. Shamasastry, Kautilya’s Arthashastra (Bangalore: Government Press, 1915), pp. 37–39 (Frances Pritchett’s transcription, Columbia University; archive.org scan). Shamasastry spells the council mantriparishad. The earlier schools’ numbers (twelve, sixteen, twenty) and the reasons against one or two ministers are in the same chapter. In the sentence on the majority, Shamasastry’s bracketed Sanskrit (bhūyiṣṭhāḥ, kāryasiddhikaraṃ vā) is left out.

3. Frederick Winslow Taylor, The Principles of Scientific Management (New York: Harper & Brothers, 1911), introduction; see Chapter 2. For “Plot twist” in 18.2: the introduction continues, “This in no sense, however, implies that great men are not needed. On the contrary, the first object of any good system must be that of developing first-class men”; chapter II, “The Principles of Scientific Management”, groups the new duties of management “under four heads”, the second being “They scientifically select and then train, teach, and develop the workman, whereas in the past he chose his own work and trained himself as best he could” (Project Gutenberg ebook 6435, checked 4 November 2026).

4. Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M. Martinez and Seth D. Pollak, “Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements,” Psychological Science in the Public Interest 20, no. 1 (2019): 1–68; see Chapter 13.

5. Madeleine Clare Elish, “Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction,” Engaging Science, Technology, and Society 5 (2019): 40–60; see Chapter 1.

6. “New Jersey town closes streets to non-residents during rush hour,” ABC7 New York, January 2018 (closures from Monday 22 January 2018, 6 to 10 a.m. and 4 to 9 p.m.; fine $200; the police chief’s words); “Judge tosses Leonia’s ban on nonresident drivers,” FOX 5 New York, 31 August 2018 (ruling of 30 August 2018 by a New Jersey Superior Court judge; the Department of Transportation’s approval was needed for closures affecting State Route 93). The apps named in the reports were Waze and Google Maps.

7. “Coltejer,” Encyclopedia of Latin American History and Culture (Encyclopedia.com): founded in Medellín in 1907; “one of the oldest and largest textile firms in Colombia”.

8. Nyāya Sūtra 1.2.1–3, the definitions of vāda, jalpa and vitaṇḍā; see Bimal Krishna Matilal, The Character of Logic in India, ed. Jonardon Ganeri and Heeraman Tiwari (Albany: State University of New York Press, 1998), chapter 2, and Chapter 12. The plain glosses in the text are mine.

9. Amy C. Edmondson, “Psychological Safety and Learning Behavior in Work Teams,” Administrative Science Quarterly 44, no. 2 (1999): 350–383.

10. Amy C. Edmondson, “Learning from Mistakes Is Easier Said Than Done: Group and Organizational Influences on the Detection and Correction of Human Error,” Journal of Applied Behavioral Science 32, no. 1 (1996): 5–28 (abstract). On the more open teams recording more errors, see “Secret Errors Kill,” Harvard Magazine, March–April 2001. For the receipts in 18.4: AHRQ PSNet’s summary of the 1996 study (psnet.ahrq.gov): eight units; team members surveyed; “an individual researcher also blindly observed nursing teams”; units with stronger nurse manager direction and coaching, perceived unit performance and quality of unit relationships “had significantly higher rates of medication errors”; detected rates depend “both on the actual error rates and the rate of reporting”. Harvard Magazine reports that Edmondson joined a study of errors in drug administration at two physicians’ invitation, and analysed “detected error rates” in “eight hospital units” (both checked 4 November 2026).

11. Jonathan Baron and John C. Hershey, “Outcome bias in decision evaluation,” Journal of Personality and Social Psychology 54, no. 4 (1988): 569–579; see Chapter 11.

12. Albert O. Hirschman, Exit, Voice, and Loyalty: Responses to Decline in Firms, Organizations, and States (Cambridge, MA: Harvard University Press, 1970), chapter 3; see Chapter 13. On a hearing and a say in law: Germany’s Works Constitution Act (Betriebsverfassungsgesetz), official English translation including amendments to 19 July 2024 (gesetze-im-internet.de): s. 1(1) (works councils are elected in establishments that normally have five or more permanent employees with voting rights); s. 90(1) no. 3 and s. 90(2) (the employer informs the works council “in due time” of plans concerning “working procedures and operations including the use of artificial intelligence”, and consults it “in good time”); s. 87(1) no. 6 (“a right of co-determination” in “the introduction and use of technical devices designed to monitor the behaviour or performance of the employees”); s. 87(2) (failing agreement, “the conciliation committee makes a decision”).

13. Hannah Arendt, The Human Condition (Chicago: University of Chicago Press, 1958), §33, “Irreversibility and the Power to Forgive”: “The remedy for unpredictability, for the chaotic uncertainty of the future, is contained in the faculty to make and keep promises” (p. 237; page as cited to the 1998 edition by Marguerite La Caze, “Hannah Arendt and Self-Forgiveness,” Parrhesia 24 (2015), the printed page itself not seen); see Chapter 4.

14. Diwali (Deepavali) as a national holiday in India: 31 October 2024; 20 October 2025 (timeanddate.com).

15. Kauṭilya, Arthaśāstra I.19, “The Duties of a King,” trans. Shamasastry (1915).

16. Dennis F. Thompson, “Moral Responsibility of Public Officials: The Problem of Many Hands,” American Political Science Review 74, no. 4 (1980): 905–916, p. 907, as quoted in Thompson (2017), note 3, which also omits words before “the ritual”; the words left out after “standing” are “by reassuring the public that someone is in charge and by projecting an image of a courageous leader who does not pass the buck”.

Named people in this chapter are composites, and their conversations and circumstances are constructed; the studies, laws and figures cited are real and sourced.

Revised from the print edition (2026)

• What the print edition claimed: that “The gap between the metric and the meaning is the subject of this chapter”; that the small firm’s design rested on “the premise that every person was capable of conscious, independent judgement”, and that it should “hire only people whose work requires consciousness”, with “Tell me what you would do in this role that an AI system could not” as its hiring filter, a question the print edition admired; that the boundary between the tool and the person “is the boundary between intelligence and consciousness”; that a radiologist’s machine in Seoul gave “a confidence score of 91.3 per cent”; that in Meera’s redesign “The consciousness was distributed throughout the organisation”; that an invented Rotterdam routing system “lacked the thing that makes a data point meaningful: the experience of being affected by it”, cured by a weekly human glance at the routes; that “experimentation without consciousness is not innovation. It is noise”; that three data faults on Rajan’s floor were each “a failure of consciousness”; that a lab technician would judge “whether the anomalies matter, which the AI cannot do”; that Rajan noticed because “His body said it was worse. He trusted his body”; that Muthu’s silence was “grief dressed as compliance”; that he told his workers “The machines are not here to replace you” and then refused to promise it; and that the stories ended in success: productivity “resumed climbing”, Muthu “started offering opinions”, Parvathi trained two juniors, and the wrong room’s idea became “a significant share of the studio’s revenue”. It also had Rajan careful “for nearly a year” with a scheduler, twenty-seven years in business, a board, Karthik at the council in person, a “handloom” camera error, a “dyer” on his floor and a Diwali forecast error reported before Diwali; put Meera on the eighth floor under “the CEO”; and used Veritas Analytics, a former McKinsey manager, a Rotterdam routing firm, Dr Park in Seoul, and the names Reshma, Valentina and Camila.

• What I now hold: an organisation answers through people, and machines add hands that cannot answer: the people who build, sell, recommend and set them up (Thompson’s many hands). The question is not whether consciousness is distributed but whether each seam, each point where work passes between a machine and a person, has someone who chose it, can take it back and answers, and whether the people it falls on can be heard. Smaller teams move ambiguity and answering onto fewer people; size should follow the work (Kauṭilya I.15). A workflow designed by its users gives them voice but does not settle who answers; participation can even blur it, so each handoff needs a named person with the time and authority to act. Some harm belongs to no single decision (Leonia); a fast organisation needs a named person to sort decisions and a way for outsiders to be heard. Permission to be wrong is someone’s agreement to answer for the cost of the error (Edmondson; vāda against vitaṇḍā). A leader who takes all the blame can be honest and still leave the other hands unexamined (Thompson against Kauṭilya); a promise must be within the promiser’s power (Arendt, recalled); voice is a claim to be heard, not a vote. The cameras’ and the scheduler’s errors were failures of specification, fixed once someone with the right to change the systems heard about them; the floor’s silence was a failure of authority and habit, which only whoever holds authority can change. Fewer layers is not less management, and a single clear signer without time or a colleague only concentrates blame. A seam can be well held inside a wrong target: the people above stand behind the targets, staff and hours, and someone must judge the seams together (Chapter 17 asks who chooses what counts as success). Waiting can add wisdom: pace should match the stakes, the spread and how far a decision can be undone; changing a pattern does not give back the harm it already did, so someone inside must watch the sum. The wrong room’s rule is for uncertain ideas; unsafe or unlawful ideas get a boundary; feeling safe is not good evidence, and a simulation is evidence only for the questions it can answer. Muthu’s appeal seat can empty while keeping its name. The Friday meeting can change how a system is used, not whether it was bought or where its gains go; the floor is told what happened to what it said; a hearing is not a say in the decision. Rajan’s promises cover the future and repair nothing already lost. The gains were a choice Rajan made, not a result of the cameras, and the meeting’s small costs beside them do not make the arrangement fair. A machine can give reasons, but the money and the remedy come from people. Rajan’s claim to a return on what he risked is no claim to the part of the gain that came from costs others carried, and asking first and telling why is what the three inspectors had been owed. Muthu’s share of the bin was real and small. The drivers through Leonia had a right to the streets; nobody had weighed the sum.

• What changed my mind: the Prologue’s rule against defining people by what machines cannot yet do; Chapter 7’s empty position; Chapter 11’s outcome bias and its rule that judgement is not the override; Chapter 13’s account of who should carry the cost of a change; Thompson’s 2017 chapter on designing responsibility and his 1980 warning about the ritual of accepting full responsibility; Edmondson’s finding that open teams record more errors; the Leonia case; the audit’s canon and invented-case findings; two reviewers’ readings: that silence is not a specification failure, that a seam can sit inside a wrong target, that waiting can be wise, that stopping a harm does not repair it, that voice differs from a say in the decision, and that the gains were a choice.

• What survives: the bin of good cloth and Muthu’s “Who am I to argue with the machine?”; the Tuesday in March and “everything”; Kadri’s fourteen and the analyst who asked to be managed; “The workflow is a relay, not a committee” (now in plain words); the wrong room, “That is technically naïve”, and the rule that every idea gets a question, not a verdict; the floor on crates and baskets; “I made a mistake”; Parvathi’s question and “They did not deserve to be moved”; “I do not know”; the Friday meeting.

• Section titles in the print edition: 18.1 was “Smaller Teams, Higher Leverage”; 18.3 was “Decision Velocity: Speed with Awareness”.

Next in the series: Chapter 19: Rethinking Roles and Structure.