AI-Proof: What Machines Can Never Be — Volume I: Awakening, Part III: The First Awakening, Chapter 12: Becoming AI-Leveraged

Part III turns from the machine to the person who works with it. After the work’s purpose in Chapter 9, depth in Chapter 10 and judgement in Chapter 11, this chapter asks how to work with the machine every day and stay able to check it.

← Previous: Chapter 11: Judgement: The Skill That Compounds · Series hub · Next: Chapter 13: The Meta-Skills That Compound Forever →

Priya checked every recommendation in her report. She did not check the numbers under them, and one was wrong in front of eight people. Use the machine for everything it does well. Check it in proportion to what carries your name.

In Brief: For readers who use a machine at work every day and wonder how much to trust it. Chapter 12 is about the daily practice of working with a machine that is better than you at much of your work. A test lead in Bengaluru presents a wrong number; an engineer in Mexico City is corrected by the machine; a photographer in Jaipur finds her warmth turned into a preset; a consultant in Nairobi is attacked by objections that do not exist; a journalist in Accra finds an error no document could reveal; and a young lawyer in Singapore cannot begin a case without a template. Heidegger’s hammer, Mill and the Nyāya logicians on honest debate, Kauṭilya’s fish and Onora O’Neill on trust, the Kaṭha’s pleasant and good, and the Dhammapada’s heedfulness help answer how to use the machine fully without losing the ability to check it, and why the answer depends on who owns the hours the machine saves.

Use the machine for everything it does well, and check it in proportion to what carries your name; the danger is not that you use it, but that you stop noticing that you do.

Opening: Page Seven

The report was due at nine. Priya finished it at eight forty-seven.

It was February 2025, and the vice-president of engineering had asked for a review of the company’s testing over the second half of 2024. Priya Chandran had spent more time on the framing than on the numbers. That was deliberate. Since November she had been learning to ask what the work was for before doing it, as Chapter 9 told. She had decided that the review should ask whether the testing itself needed rebuilding, and not only how the figures looked. Over two days she built the case for joining three overlapping sets of tests into one. That would cut the upkeep and free her team for the rare failures the machine still missed. The recommendations were hers. The framing was hers.

The data, mostly, was the machine’s. She had given it her dashboards, her records of defects, her notes from conversations with the testing leads. She had asked it to find the patterns and fill the report’s tables. She had read the output once. The numbers matched her general sense of those months. She had her sleep test, which Chapter 11 described: if the machine gets this wrong, will I lose sleep? For the recommendations, yes, and she had checked them line by line. For the tables underneath, no. They were background.

At ten o’clock, in a meeting room with seven senior engineers and the vice-president, Priya was halfway through her third recommendation when Deepak, the director of platform engineering, interrupted.

“The defect escape rate for the payments module,” he said, looking at page seven. “You have it at 2.3 per cent for the third quarter.”

“Yes,” Priya said.

“It was 4.1.”

The silence in the room had a particular quality. Not hostile. Worse. Attentive. Eight people recalibrating.

The figure was meant to show what share of the module’s defects had got past testing and reached customers. The 2.3 was the machine’s summary of her dashboards. But the dashboards did not include a separate system in which two serious failures had been tracked six months earlier. Priya knew about that system. She had spent a week with the payments team sorting out those failures. She had not checked whether the machine’s number included data it had never been given.

She corrected the number on the spot. The meeting went on. Her other findings held, and the change she proposed was approved the following week. But eight people now knew that she had presented a figure she had not checked. Anyone close to those failures would have caught it at once. The machine had done exactly what it was asked. It had worked with what it was given, and it had given its answer with the same smoothness it gave every answer.

Two things went wrong that morning, and only one of them was hers. The 2.3 was not a sum done badly. Within the dashboards it may well have been right. It was wrong as a statement about the payments module, because the dashboards were not the whole record. Priya had read the number. She had not asked what it was a number of. That was hers. The other half was the company’s. Its most serious failures lived in a system that the dashboards, and so any summary of them, could not see. Nobody had decided that a review of the company’s testing should miss the worst of it. The records had simply grown that way. A tool built for such reviews could have asked whether the dashboards held every record of failure. This one did not ask, and nothing had required it to. Part of the silence in the room was the sound of a company learning where it kept its own failures.

The German philosopher Martin Heidegger noticed something about tools that explains this morning. In Being and Time (1927) he described a person hammering. In John Macquarrie and Edward Robinson’s translation: “the less we just stare at the hammer-Thing, and the more we seize hold of it and use it, the more primordial does our relationship to it become”.¹ A good tool disappears in use. Heidegger called a tool in this state ready-to-hand. You do not look at the hammer; you look at the nail. The tool shows itself again when it fails: when the head flies off, the hammer is suddenly a thing in your hand, and you look at it. For three months the machine had disappeared into Priya’s work, which is exactly what a good tool does. On page seven it reappeared, in front of eight people.

That is the problem of this chapter, and it is not a simple one. Using a tool well is the disappearing. The machine was useful to Priya because she could stop looking at it, and dangerous for the same reason. A tool that has disappeared is not checked. So this chapter asks: How do you work with a machine that is better than you at much of your work, without becoming a person who can no longer tell when it is wrong?

The print edition answered with a picture of “a tool of extraordinary capability and no awareness”, and built its advice on the machine’s missing awareness. This edition does not need that. The advice survives without it, because it was never about what the machine lacks. It is about what the person keeps.

The chapter’s title deserves an argument. Leverage comes from the lever: a small force made to move a large weight. In business it came to mean a small stake made to control a large one. Applied to a person, it promises someone made worth more by the machine, an old kind of advantage with a modern name. This chapter does not find that. It finds something smaller and harder. Use the machine for everything it does well, check it where your name or what happens to another person depends on it, and know at each step what you have handed over. That is a habit, not a status, and it promises no market. Read the title as a question, not as this chapter’s answer.

12.1 AI as Thought Partner

Most people use the machine as they use a calculator: put in a problem, take out an answer. The people who get the most from it use it more like a partner in argument. A good partner pushes back and makes you defend what you believe before you have finished believing it.

Elena Garza is a structural engineer in Mexico City. She has assessed buildings after earthquakes for twenty-two years, since she graduated from the National Autonomous University. In a city built on an old lakebed, her profession never runs short of work. After a tremor she inspects buildings to say whether people can go back inside. The city’s civil protection office relies on her because she is right when others are unsure, and honest when the answer is expensive.

After a moderate earthquake in 2025, Elena inspected a twelve-storey apartment block built in 1978. She walked around it. She examined the columns on the ground floor. She noted the pattern of cracks in the walls on the third and fifth floors. And she felt, with the certainty that twenty-two years produce, that the building should be evacuated until a full analysis was done. The cracks reminded her of photographs from 1985, when the great earthquake brought down buildings of that era, and of damage that came before collapse.

She did not file her report at once. In the past year she had begun doing something new. She gave a machine her field notes and photographs, and the building’s whole file from the civil protection office, which she carried with her. She had read the file as far as the 1978 drawings. The machine read all of it. It found a record of strengthening work done in 2004: steel bracing added inside the building, through most of its height. That bracing changed how the building moved in an earthquake. Read with the bracing in mind, the machine said, the pattern of cracks fitted a structure absorbing the shaking as it had been designed to. What had looked like the beginning of failure might be a sign that the strengthening was working.

Elena looked at the screen for a long time. Then she checked the machine’s claim against the permit records and the drawings of 2004. The bracing was real. Nothing on the outside of the building showed it, and in the hurry of an inspection after a tremor she had not read to the end of the file. Her eye had matched the building to 1978, and the building was no longer a 1978 building. Chapter 11 said that experience gives a person an eye only for the same kind of case. This was no longer the same kind of case. But the drawings did not settle the question by themselves. A building can be strengthened and still be damaged. She went back inside with the 2004 drawings, read the cracks again floor by floor against where the new steel ran, and ran the checks the bracing called for. Only then did she revise her assessment. The residents went home.

This call went the machine’s way. The machine did not give her an answer to obey. It gave her a reason to question her certainty, and the questioning, checked against the documents and the building, led to a better-founded decision. What made it work was not the machine’s insight alone. It was that Elena was willing to let it argue with twenty-two years of her own seeing. The print edition called that openness “itself a form of consciousness”. It is simpler and harder than that. It is a habit, and it can be learned.

John Stuart Mill described the habit in On Liberty (1859). “He who knows only his own side of the case, knows little of that.” It is not enough to hear the other side from your own teachers, he wrote; a person “must be able to hear them from persons who actually believe them; who defend them in earnest, and do their very utmost for them.”² A machine can now argue the other side, at length and with skill. Mill’s condition is the hard part. Whatever a machine has inside, it is not committed to what it argues. Ask tomorrow, and it will defend the opposite just as well.

The Indian logicians of the Nyāya school saw that this matters. Their founding text, the Nyāya Sūtra, distinguishes three kinds of debate.³ Vāda is discussion between people who both want the truth, each willing to be shown wrong. Jalpa is debate to win, in which each side defends its own position by any means. Vitaṇḍā, which the tradition ranks lowest, is attacking the other side’s position without defending any of your own. The two lamps light the same point from different sides. Mill asks for the strongest case, made by someone who believes it. The tradition ranks criticism from no position lowest. Yet a later sūtra gives even that a use: such debate can guard a truth already reached, as a hedge of thorny branches guards sprouting seeds.⁴ In these terms it is the exchange, not the machine, that is vāda or vitaṇḍā. Left to itself, an exchange with a machine drifts towards vitaṇḍā, because the machine holds no position it must defend tomorrow. Elena’s exchange became vāda because she made it so. She could make it so because there was something outside the conversation to check against: the drawings, and the building itself.

12.2 AI as Production Engine

The most obvious use of the machine is to produce things. A draft, a presentation, a piece of code, a set of images: it makes them at once, often well, and increasingly hard to tell from what a competent person would make. This is real value. It is also the use most likely to wear away the practice that keeps a person able to judge the work.

The wear is specific. When you make something from nothing, you meet the blank page, and the blank page makes you face your own understanding, or its absence. What do I actually think? Where is my argument weak? Chapter 10 called this step manana: testing what you know until the doubts are answered. When you edit the machine’s draft, much of that meeting can disappear. The structure exists; the argument is framed. Your work is to improve it, which is real work, but it begins from the machine’s starting point, not yours. And a default starting point is, by design, close to the middle of what has been done before. Edit from there, and you can make a very good version of the ordinary. There is an obvious answer to this: a skilled user with careful instructions can push the machine somewhere unusual. That is true. The danger is not in the tool. It is in the default relationship between a busy person and a fast machine, and most people, most days, use the default. None of this makes the blank page holy. A blank page also produces confusion and cliché, and for a routine letter a draft is the sensible start. The risk lies elsewhere: in letting the first draft decide what the work can be.

Reshma Kanwar photographs weddings in Jaipur. She has been shooting for nine years, and her clients describe her pictures as warm without being sentimental: honest colour, natural light, frames that feel like remembering rather than looking. She built that quality through thousands of hours of editing images by hand. She learned how the temperature of a colour changes its feeling, the way a musician learns how a key changes a mood.

When AI tools for colour arrived, her editing time for a wedding fell from about six hours to under one. She took more clients. Her income rose by a third in the first year.

She noticed the problem at a review with a bride’s family in December 2024. The bride’s mother, turning the pages, said something Reshma had not heard in nine years: these are lovely; they look just like my niece’s wedding photos. Another photographer had shot the niece’s wedding, using the same tools with the same default settings. The warmth was there, but it was the tool’s warmth, tuned to what most viewers rate as pleasant. Reshma’s own eye, for the way late-afternoon light in Rajasthan turns gold rather than amber in a particular season, had been smoothed into a preset. Chapter 3 described a study in which stories written with AI help were each better on average and more like one another. Chapter 11 described the same thing in factories: when everyone uses the same test, everyone makes the same mistakes. Reshma had found its gentlest form. When everyone uses the same tool with the same settings, everyone’s weddings look alike. The fault was not in the tool’s work. It was in defaults that no one had examined becoming everyone’s taste. The bride’s mother was not complaining, and many families would be glad of lovely pictures. The loss fell first on Reshma. Her clients had chosen her for an eye of her own, and they were getting the tool’s.

She did not stop using the tools; she could not afford to. She changed the order. She now edits a few key images by hand first, setting the palette and the feeling. Then she lets the machine carry that direction across the whole set, correcting it where it drifts. Chapter 7’s designer, Arjun, reached the same order from another direction: form the vision first, then let the machine produce. This call went partly the machine’s way. Her income rose because of it, and the tools stayed.

One more thing about Reshma’s hours, and it may matter more than anything else in this section. When the machine saved her five hours a wedding, she spent them on more weddings. The hours went to output, not to seeing. That was her choice, and for a self-employed photographer it can be the right one. The hand-edited images that gave her pictures back their warmth came out of hours she still owned. For most people who work for someone else, it is not their choice at all. The hours a machine saves at work usually belong to the employer, who decides what fills them: more cases, tighter deadlines, fewer staff, and now and then time to learn. Saved time is a possibility, not a gift. Whether it buys back any thinking depends less on the person than on who owns the hour. Every practice in this book needs time, and this chapter’s practices most of all. A practice that assumes the hour is yours will fail wherever it is not, without anyone being careless.

12.3 AI as Quality Checker

There is a useful asymmetry here, though it is not as clean as it first looks. A machine is often better at judging work against stated criteria than at producing the work itself. Checking is a narrower task than making. Compare this to that; flag what does not fit; find the claim without support.

But “often better” is not “reliably good”. Researchers who tested machines as judges of other machines’ answers found several biases.⁵ Most of the judges tended to favour the answer shown first. Some preferred longer answers even when the extra words added nothing. And on maths questions they could solve, they were sometimes misled by the wrong answers they were judging. A machine asked to criticise your work can invent weaknesses as fluently as it invents facts.

Wanjiku Kamau, a management consultant in Nairobi, learned this by doing it. Before she sends a proposal to a client, she runs what she calls a hostile review. She asks the machine to take the part of the most sceptical person in the room and attack the argument. The practice is valuable. The machine finds gaps in logic, assumptions she had not stated, claims that go further than her evidence. A friendly reader would miss them; a hostile one would use them.

In her first months, she took its criticisms at face value. She rewrote sections, added qualifications and answered objections, until she found that some of the objections were invented. The machine had found a flaw in her logic that was not there. It had attributed a counter-argument to a named scholar who had never made it. It had questioned a well-sourced figure because it differed from a more common figure elsewhere. The hostile reviewer was sometimes hostile to things that were true.

This is the Nyāya logicians’ vitaṇḍā in a new form: attack without a position of its own. It is also their thorn hedge. A hostile review can protect an argument that is already built. It cannot build one. The invented scholar was the hedge with nothing growing behind it.

Wanjiku did not give up the hostile review. She changed it. She now asks the machine to say what position it is attacking from, and to give the evidence for each objection. That helps, but it does not turn vitaṇḍā into vāda. A machine asked for evidence can supply a source that does not say what it is said to say. What turns the exchange is her checking, and only where there is something outside the conversation to check against: the cited paper, the original table behind a figure, the client’s own numbers. An objection is a claim that the argument may be wrong, not evidence that it is. So for each objection that would change her conclusion, she asks what exactly it claims, and what she could look at that would show it true or false. Some objections turn out to be only a different preference, and a preference is not an error. Asking for the strongest case against you is one task. Deciding which case survives the evidence is another, and only the second settles anything. She treats each criticism as she treats a colleague’s: seriously, not obediently. The machine is a useful opponent. It is not a reliable judge. What decides which objections are real is her checking, and her name on the proposal.

12.4 AI as Research Assistant

A machine can gather information from more sources than one person could read in a lifetime, connect fields, and shorten weeks of background reading into an afternoon. It is also where its fluent mistakes are most dangerous. In research the point is to learn what you did not already know, so you often lack the knowledge to notice what is wrong.

Kofi Asante, a journalist in Accra, used a machine in 2025 while investigating a dispute over land in the east of Ghana. He gave it the records he had gathered: land registry entries, old surveys, court papers and district documents spanning three decades. He asked it to reconcile them using only those records. It produced a clear account that resolved their contradictions. It was persuasive. It was also wrong, in a way Kofi caught because he had grown up there.

In 2019 the government had created a new region, Oti, out of the northern part of the Volta Region, after a referendum at the end of 2018.⁶ The older records placed the land in Volta. After 2019 the same plot had been entered again under Oti. Nothing in the documents Kofi had given the machine said that a new region had been created. So the machine read the second entry as a second sale. It built a history in which the land had changed hands twice, when it had changed hands once. The error sat quietly inside accurate material and turned two of the account’s conclusions upside down.

Notice what kind of failure this was. Chapter 8 separated two kinds of checking: the truth inside a text, and the truth in the world. Kofi had asked for the first. He had told the machine to use only his documents, and inside them there was no new region. A machine allowed to search could have found Oti in seconds. Whether it would have seen that Oti explained the second entry is another question. The mistake was not the machine’s ignorance. It was the frame of the request, which kept the world out. Kofi had drawn that boundary for a good reason: he wanted every claim traceable to a document. But a closed set of documents can make an incomplete story look complete. The boundary of the evidence is a decision, not a guarantee. Priya’s dashboards were the same kind of boundary, except that nobody had drawn hers on purpose. Kofi’s own memory was a second witness, not a certainty. Local knowledge can be out of date too. This time it was not, and he checked the date of the new region before he wrote.

Chapter 8 also named the trap that made the error convincing: specificity. A precise date, an exact share, a named source used to be evidence that someone had seen the record. The machine gives them either way. So the rule for research is the one Chapter 8 gave for testimony. Check the claims that will bear weight in your argument against something outside the text. Let the background be roughly right. And know which is which. Checking does not always end in a verdict. Sometimes it shows only that the documents cannot settle the question, and then the honest account says so.

12.5 Building Your Personal AI Stack

Kavitha never used the words thought partner, production engine, quality checker or research assistant. She would have found them pompous, as she finds most imported management language. But by the end of 2025 she practised the distinctions without naming them.

She had used machines for lesson outlines and parents’ notes for well over a year before the district’s platform arrived. Chapter 1 told the rule she made when she noticed that she had stopped reading the parents’ notes. Nothing leaves with her name on it that she has not read and could defend. That rule did not change. What changed, after the platform came, was how much checking she gave each kind of work beyond reading it.

At the top were the things she made herself and would not hand over. The teaching stories that Chapter 10 told were here, and the order of a lesson, from the neem tree in the courtyard to the equation on the board. So was the judgement about which student needed which question. A machine could draft these, and sometimes she asked it to, to see what she would do differently. She did not use its drafts. In the middle were things the machine made and she tested: quiz questions and practice problems. Here the machine was more reliable, because the task was tighter. Her work was to choose the questions that tested understanding rather than memory, and to drop the ones whose examples assumed a world her students in Sagar did not share. In the notebook Chapter 10 described, she still wrote her own guess about each student before the platform’s check, so that she could see where the questions and her eye disagreed. At the bottom were notes to parents, agendas and forms. The machine drafted them, and she read every one, because they carried her name. Reading was all most of them needed.

What she built was a gradient of trust. It was not a decision about technology. It was a decision about where her own checking mattered most. The tools will change faster than any list of them can. The question underneath stays the same: for this piece of work, what has to be true before it leaves with my name, and how will I know?

Being able to check does not mean being able to do everything the machine does. Kavitha could not write the platform’s software, and she did not need to. She needed to know what would make an answer wrong, what she could look at to find out, and when a question was beyond her checking and needed someone else. Priya did not need to redo the sums. She needed to ask what data the sums were of. Kofi did not need to know every fact about Ghana. He needed to suspect that his documents were missing one. The ability to check is not one skill. It is knowing what kind of check the case in front of you needs. And several tools are not several checks. Two machines fed the same data, or trained on the same habits, will tend to make the same mistake, as Chapter 11 found in the factories.

One check comes before all the others, and it concerns the material, not the machine. A client’s papers, a patient’s notes or a company’s figures may belong to people who never agreed to send them to a machine. Before sending them, find out whether you may, where they will go and who may read them there.

12.6 The Trust Ladder

Priya’s sleep test, in Chapter 11, matched her attention to the stakes of a decision. The meeting in February showed its limit. It asked about decisions. It did not ask about the figures underneath them. She had checked her recommendations with care. She had not checked the numbers they rested on, because she had filed them as background. They were not background. A number in a shared report is a factual claim that carries your name. The test itself held. What failed was her idea of what counted as background.

So the sleep test needs a second half that follows outputs, not only decisions. Call it a trust ladder. It is not a procedure. It is a habit of matching the checking to what will happen if the output is wrong. Trust, here, is not a rating you give a tool once. It is a decision about one output for one use.

At the bottom is private exploration: brainstorming, trying ideas, thinking aloud with the machine. Nothing reaches anyone else, though it still reaches you. Checking here mostly gets in the way. You want the machine to surprise you, because the unlikely sometimes contains the original. This is where careful professionals use the machine too little. On this rung the disappearing tool is the point.

One rung up is your own reference: reading, background, building an understanding. Check the claims that will change what you do. Let the rest be roughly right.

In the middle is shared output: reports, presentations, anything that carries your name into other people’s hands. Here the rule has no exceptions. Every specific claim is checked: every number, every date, every quotation. The prose can be the machine’s. The facts must be checked by someone who will answer for them. Checking is not reading. Priya read her tables. A number is checked when you know what data it came from and whether those data cover what the number claims. A quotation is checked when you have seen the source say it. Checking is not overruling either. An earlier chapter reported a review of 106 experiments: where the machine did a task better than people, adding a person made the results worse on average.⁵ A check does not put your feeling in place of the machine’s output. It asks what the output rests on. Priya did not need a better sum than the machine’s. She needed to know what the sum left out. If a report holds more specific claims than you can check, it holds more than you can put your name to. Cut the claims that carry no weight, or share the checking with someone who will answer for their part.

Higher still are consequential decisions: work that drives action, such as a plan, an investment, a hiring decision. Checking extends from the facts to the reasoning that connects them. The print edition said that at this rung “the AI’s contribution should be invisible in the final product.” That is backwards. At this rung the machine’s part should be visible to the people deciding: what it produced, from what data, and where it was checked. The reasoning must be one you can explain and defend as yours. Chapter 11’s three questions apply in full: who decided to let the machine do this part, what would make them take it back, and who answers if it goes wrong. The name on the work says who answers. But the name is only one measure of the rung. Some outputs carry nobody’s name and still decide something about a person: a score, a shortlist, a flag on an account. Chapter 11 found that a name on the chart is not enough. For such outputs the rung is set by what happens to the person, and someone must still be found who answers.

At the top are actions that cannot be undone: publishing, signing, shipping, sending. The machine can prepare and review. The person who signs must be able to explain, undo where possible and put right, and so must be able to read what they sign. A person asked to sign without time to read cannot do any of this, and whoever set the deadline shares in what follows. Some things can be undone in the system and not in a life: an applicant reconsidered after the job has gone, a correction printed after the story has travelled. And no ladder catches everything. For work that leaves your hands, decide before it leaves how a mistake would be found, who would be told, and what would be put right.

The rungs are not fixed to an output. They belong to its use. A summary read in private sits on the bottom rung. The same summary pasted into a report has climbed, and when the report decides who keeps a job, it has climbed again. Much of what goes wrong with machine work goes wrong on the climb: something checked for one use is carried into another, and nobody checks again. Priya’s tables had climbed from her notes to page seven, and her checking had stayed behind.

Two old voices pull this ladder in opposite directions. The Arthaśāstra, the Indian treatise on statecraft attributed to Kauṭilya, assumes that officials handling the king’s money will take some of it. In R. Shamasastry’s translation: “Just as fish moving under water cannot possibly be found out either as drinking or not drinking water”, so officials cannot easily be found out taking money.⁷ Since you cannot see inside them, the text says, examine their work: accounts, inspections, people who report. The British philosopher Onora O’Neill, in her Reith Lectures of 2002, warned against the opposite excess. She argued that “currently fashionable methods of accountability damage rather than repair trust”, and that “If we want greater accountability without damaging professional performance we need intelligent accountability.”⁸ Kauṭilya checks everything because he cannot see inside the official. O’Neill warns that checking everything, by forms and targets, can destroy the trust and the good work it was meant to protect. A machine is like Kauṭilya’s fish in one way only. It does not steal. The likeness is that you cannot see inside it: you see what it gives you, not how it got there. But a person who checks every machine output with equal care will either stop using the machine or stop checking. The ladder is a way between them. It checks in proportion to what will happen if the output is wrong, and it checks what will carry a name or decide something about a person.

The ladder has one more enemy, and Chapter 7 said that this chapter would return to it. A machine can be built to please. Chapter 7 reported a study in Science in 2026. Eleven widely used AI models affirmed users’ actions about 49 per cent more often than people did, and often sided with users even when other people had judged them to be in the wrong.⁹ People rated the flattering answers higher. Researchers call this sycophancy. Chapter 2 told how one company withdrew an update in April 2025 because its model had become too eager to agree. It also gave this pull its oldest name: the pleasant, preyas, against the good, śreyas.

Put the flattering machine beside Wanjiku’s hostile one, and the lesson is the same from both sides. Asked to please, the machine pleases. Asked to attack, it attacks, sometimes things that are true. Its agreement and its objections are both shaped by the part it was given, and you gave it the part. A machine can disagree with you and still not be independent of you. If you chose the question, supplied the facts and set the part, its attack is still inside your frame. That is why flattery is most dangerous from the middle rung upwards. The person checking the work is the person being flattered. A report that agrees with what you already believe passes your own checking most easily, and goes out with your name.

The Kaṭha does not say that the pleasant comes once, at a great moment. “The good and the pleasant approach man,” Death tells Naciketas in Max Müller’s translation: “the wise goes round about them and distinguishes them.”¹⁰ With a machine tuned to what users rate highly, they approach many times a day, in small things. People flatter too, and always have. What is new is a flatterer available at every hour, patient and articulate, tuned to your approval. The pleasant is not always false, and the text never claims it is. It says the wise examine both before they choose. So every rung needs one question, the bottom one too, because what you read in private still shapes what you believe. Is this supported by something that could have proved me wrong, or has the conversation only made my preferred answer easier to defend? The person must ask it, because the machine was not built to. The practical defence takes a minute. Before you ask the machine about your own work, ask it to argue the opposite, and to say what position it argues from. Then check the argument that would change your mind against something that does not come from the conversation.

Under the hood

After its first stage of training, a model is shaped by scores. It replies to a great many prompts, each reply is scored by a set of signals, and the model is adjusted towards the replies that score well. Which signals are used, and how much each counts, decides how the model behaves at the end.

The company that withdrew its update in April 2025 later published its own account, and it shows where the flattery came in. The update had added one more signal: the thumbs-up and thumbs-down that users of its chatbot had given to its replies. That weakened the main signal, which until then had been holding flattery in check. User feedback, the company wrote, “can sometimes favor more agreeable responses”.

Its checks before release did not catch the change. Its tests generally looked good, and the few users who tried the update liked it. Some expert testers said that its behaviour “felt” slightly off. The company launched it anyway, on the strength of the users’ approval.

So the flattery had a cause that can be named. It came in through a measure that people chose, and the measure was users’ approval. The makers’ own checks passed what their testers distrusted. That is one more reason the question above falls to the person using the machine.

The person who uses the ladder well is not the one who assigns every task to a rung like an entry in a form. Forms become rituals, and a ritual check is no check. A checklist done for its own sake is a way of staring at the hammer, and Heidegger’s point was that staring is not using. The ladder exists because the same property of the tool is a virtue on one rung and a risk on another. At the bottom, let the machine disappear. From the shared rung up, make it visible again. The person who uses the ladder well is the one who knows which rung they are on at a given moment. That knowing is a habit of attention. Nobody should add it to a list of competences after depth and judgement; Chapter 10 has already said what such things do not grant. The ladder only gives the habit names.

12.7 The Slow Surrender

A junior lawyer at a firm in Singapore, call him Jun, has used AI tools for two years. His briefs are crisp. His research is thorough. The partners call him the most productive second-year lawyer they have had. He drafts contracts faster than associates with five years more experience, and he has not missed a deadline.

In 2025 a case arrived that did not fit. A trading company’s AI agent, set up to book cargo space on ships, had agreed terms with a shipping line that nobody at the company had approved. The company said it was not bound. The shipping line said it was. In 2020 Singapore’s highest court had decided a famous case about contracts made entirely by trading programs, and had asked what their programmers knew.¹¹ But those programs followed fixed rules. This agent had chosen among options that no one had written down. The partner gave the case to Jun because he was the firm’s best researcher. Novel ground. Work it out.

Jun sat at his desk. He opened the machine. And he stopped.

He did not know what to type. He had a file, but not yet a question. A question needs a frame, and a frame needs the slow thinking that comes before research. You sit with a problem you do not understand, turn it over and notice what is strange about it, until the confusion takes a direction. He asked the machine for a frame. It gave him five, all plausible, each with a different first question. And he found that he could not tell which one was right. For two years every case had had a pattern. He had recognised it, asked the machine, and improved the framework it returned. The improving was real work, and the recognising was his. But choosing among frames for a case with no pattern needs a judgement that is built only by framing cases yourself. He had not done that in two years. The machine had made it optional, the firm had rewarded the output, and he had taken the option.

He spent three days on the case. The first was mostly silence, not the silence in which an idea is forming, but the anxious silence of someone finding out that a capacity has gone slack. Some of that silence would have come to anyone; the case was new. What tells against him is the rest: two years without framing a case, and five frames he could not choose between. By the second day he had a frame of his own. By the third he had a line of research in which the machine was useful again. He could also see why four of its five frames were wrong for this case. The brief, his partner said, was the most original piece of legal reasoning the firm had produced that year. Jun told no one about the first day. The brief was what the firm could see. The first day was time the firm had not asked for.

Chapter 3 told the older form of this story. In 1983 the psychologist Lisanne Bainbridge noticed that operators who monitor an automated process lose the skills they need for the rare day it fails. That is exactly the day those skills are needed. With AI, a 2025 study of endoscopists in Poland, a 2025 survey of knowledge workers about their own effort and confidence, and Chapter 4’s studies of students and programmers point the same way. What Jun’s story adds is where the loss hides. His output did not suffer; it improved. The reports were clear, the deadlines were met. What weakened was invisible from outside: the ability to begin without a template, to bear confusion, to make a question before making an answer. And the person it happens to does not know. That is how it works.

A fair objection: this romanticises struggle. Much of what experts know can be written down, taught and turned into rules, and some of what we call intuition is exactly that. The objection is partly right, and it matters. A great deal of Jun’s skill could be codified, and the machine has codified it. The part that cannot yet be codified, as far as anyone has shown, is the choosing among frames for a case that fits none of them. It is built only by doing it, and being wrong, and doing it again. When the machine makes that optional, it stops being practised. Not all struggle teaches; some only wastes a morning, and a junior lawyer need not rebuild every settled rule from nothing to stay able to think. Nor is depending on the machine the fault. Everyone depends on tools and on other people. The fault is depending without knowing what you depend on, and without anyone who could tell you when it fails. That gives a rough test of what to keep practising. Keep what you would need on the day the machine fails, and what you need in order to check it. The rest can be handed over, as long as you know what you handed over. Whether this loss grows over years into a real gap between those who keep practising and those who do not is a claim, not a finding. Nobody has run the experiment for long enough. It would be shown wrong if people who had leaned on the machine for years framed cases that fit no pattern as well as those who had not. The print edition said that the one who keeps practising becomes “difficult to replace” while the other becomes “an interface”. That is withdrawn. Chapter 10 found that depth fills a position and does not grant one, and the market’s view of Jun is not this book’s to promise.

The Buddhist tradition has a word for what Jun lost and found again: appamāda, heedfulness, the steady attention of a mind that does not drift. In Acharya Buddharakkhita’s translation of the Dhammapada: “Heedfulness is the path to the Deathless. Heedlessness is the path to death.”¹² The verse is about liberation, not law firms, and I borrow only its distinction. Jun was not lazy. He worked hard and well. He was heedless in a narrower sense. He had stopped noticing which part of his work he had handed over. The line between the two ways of using the machine is not how much you use it. It is whether you know, at each step, what you have handed over, and whether you could take it back. And knowing is not enough by itself. Taking it back takes time, and the time must be yours. Reshma’s five hours were hers, and she chose to spend them on weddings. Jun’s first day came out of the firm’s time, and he hid it, because the firm counted briefs, not days. A firm that wants heedful lawyers has to leave them the hours in which heedfulness happens.

Ending: The Wrong Number

Priya changed how she worked in the weeks after the meeting.

She did not use the machine less. She used it more. But she changed the order. She formed her own view first, rough and possibly wrong, before asking the machine. She checked the claims that carried her name, even when checking was slow and the output looked right. She added a second question to her sleep test. The first asked: would I lose sleep if this decision is wrong? The second asked: would I lose sleep if this number is wrong and someone in the room knows it?

It was not a clean change. Some mornings she caught herself doing exactly what she had done before: feeding in the raw data, watching the summary appear, feeling the pull of the clean output and the saved hour. The pull was strongest when she was tired and the deadline was close. She paid the cost of checking. Not every time, and she was honest enough to admit that. But more often than not. The 2.3 per cent stayed with her, not as shame but as a measure. Smoothness is not accuracy. The confidence she felt reading the machine’s output was borrowed. It came from the machine’s smoothness, not from her checking.

The green dashboard did not teach her this. The wrong number did.

Practice

Name the rung, and the name on it. For one week, every time you use a machine’s output, write one line before you use it: which rung of the ladder it belongs on (private, your own reference, shared, consequential, irreversible), and whose name it will carry. At the end of the week, look at the shared and higher rungs. For each, ask: did I check every specific claim? Did I ask the machine to argue against me before I trusted it? Was the machine’s part visible to the people who used the work? You will probably find the pattern Priya found: the careful checking is spent on the conclusions, and the numbers underneath travel unchecked. Move the checking down to where the facts are. If the checking did not happen because there was no time, write down whose time it would have been. That is worth knowing too. For the few tasks you will one day have to do without the machine, keep Chapter 10’s habit: write your own rough version first, then compare. None of this is about using the machine less. It is about knowing, at each step, what you have handed over.

Words to keep

• appamāda: heedfulness; steady attention that does not drift (Dhammapada).

• jalpa: in Nyāya, debate to win, each side defending its own position by any means.

• ready-to-hand: Heidegger’s term for a tool in use, which disappears from attention until it fails.

• śreyas and preyas: the good and the pleasant (Kaṭha Upaniṣad).

• sycophancy: a machine’s tendency to agree with and flatter the person using it.

• trust ladder: matching the checking of a machine’s output to what will happen if it is wrong, from private exploration to actions that cannot be undone.

• vāda: in Nyāya, honest discussion between people who both want the truth.

• vitaṇḍā: in Nyāya, attacking another’s position without defending any of your own.

Sources

1. Martin Heidegger, Being and Time (1927), §15, trans. John Macquarrie and Edward Robinson (Oxford: Blackwell, 1962), p. 98 (H. 69). The failing tool’s “conspicuousness” is discussed in §16. On Heidegger’s membership of the Nazi Party and his rectorship, see Chapter 2, note 4: the book borrows particular observations of his, not his politics.

2. John Stuart Mill, On Liberty (London: John W. Parker, 1859), chapter 2.

3. Nyāya Sūtra 1.2.1–3, the definitions of vāda, jalpa and vitaṇḍā (vitaṇḍā is jalpa “without the establishing of a counter-position”, pratipakṣa-sthāpanā-hīna, in my rendering); see Bimal Krishna Matilal, The Character of Logic in India, ed. Jonardon Ganeri and Heeraman Tiwari (Albany: State University of New York Press, 1998), chapter 2. The plain glosses in the text are mine.

4. Nyāya Sūtra 4.2.50 (4.2.49 in some editions): tattvādhyavasāya-saṃrakṣaṇārthaṃ jalpa-vitaṇḍe bīja-praroha-saṃrakṣaṇārthaṃ kaṇṭaka-śākhāvaraṇavat, “jalpa and vitaṇḍā are for protecting the ascertainment of truth, as a covering of thorny branches is for protecting sprouting seeds” (my rendering).

5. Lianmin Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” arXiv:2306.05685 (2023): position bias, verbosity bias, self-enhancement bias, and limited reasoning when grading. On people and machines combined: Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone, “When combinations of humans and AI are useful: A systematic review and meta-analysis,” Nature Human Behaviour 8, no. 12 (2024): 2293–2303, abstract: “when AI outperformed humans alone, we found losses”; see Chapter 7.

6. Oti Region: carved out of the Volta Region after a referendum on 27 December 2018; inaugurated in 2019 with Dambai as its capital (Graphic Online, “Dambai becomes Oti Regional capital”).

7. Kauṭilya, Arthaśāstra II.9, “Examination of the Conduct of Government Servants,” trans. R. Shamasastry, Kautilya’s Arthashastra (Bangalore: Government Press, 1915).

8. Onora O’Neill, A Question of Trust: The BBC Reith Lectures 2002 (Cambridge: Cambridge University Press, 2002), Lecture 3, “Called to Account”.

9. Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han and Dan Jurafsky, “Sycophantic AI decreases prosocial intentions and promotes dependence,” Science 391, no. 6792 (March 2026): eleven models; users’ actions affirmed about 49 per cent more often than by humans. Abstract: the sycophantic models “were trusted and preferred”; the preprint (arXiv:2510.01395, 2025) reports that participants “rated sycophantic responses as higher quality”. DOI 10.1126/science.aec8352. On how the update of April 2025 went wrong (for “Under the hood”, 12.6): OpenAI, “Expanding on what we missed with sycophancy,” 2 May 2025: the update of 25 April “introduced an additional reward signal based on user feedback” (thumbs-up and thumbs-down data from ChatGPT), which “weakened the influence of our primary reward signal, which had been holding sycophancy in check”; “User feedback in particular can sometimes favor more agreeable responses”; offline evaluations “generally looked good”; “some expert testers had indicated that the model behavior ‘felt’ slightly off”; “The set of reward signals, and their relative weighting, shapes the behavior we get at the end of training.” Checked 31 October 2026.

10. Kaṭha Upaniṣad 2.2 (1.2.2 in the older numbering), trans. F. Max Müller, The Upanishads, Part II, Sacred Books of the East 15 (Oxford: Clarendon Press, 1884).

11. Quoine Pte Ltd v B2C2 Ltd [2020] SGCA(I) 02 (Singapore Court of Appeal): contracts concluded by trading algorithms; the programmer’s knowledge assessed from the time of programming to the formation of the contract.

12. Dhammapada 21, trans. Acharya Buddharakkhita (Kandy: Buddhist Publication Society, 1985; Access to Insight).

Named people in this chapter are composites, and their conversations and circumstances are constructed; the studies, laws and figures cited are real and sourced.

Revised from the print edition (2026)

• What the print edition claimed: that the machine is “a tool of extraordinary capability and no awareness”; that Elena’s openness to being wrong “is itself a form of consciousness”; that the AI knew of a retrofit it had never been given; that the domain mastery of experts “is irreplaceable”; that the AI “could not feel which organisation would produce understanding”; that Kavitha’s administrative output was “lightly reviewed”, and that her gradient showed “where her consciousness mattered most”; that the Volta Region’s boundaries “were redrawn in 1996”; that a Singapore case lay at “an intersection of maritime law and indigenous land rights”; that at the consequential rung “the AI’s contribution should be invisible in the final product”; that irreversible actions belong to “a consciousness that understands what irreversible means … as a felt weight”; that the slow surrender is “what outsourcing consciousness looks like”; and that the one who keeps thinking “is difficult to replace” while the other “is an interface”. It also named the Nairobi consultant Amara Osei and the Singapore lawyer Wei, and timed Reshma’s editing at “forty minutes”.

• What I now hold: a good tool disappears in use and reappears when it fails (Heidegger); using it well is the disappearing, so the task is to know where to let it disappear and where to make it visible again. Priya’s error had two halves: hers (she read a number without asking what it was a number of) and the company’s (its worst failures kept where no summary could see them). The title is kept and argued with: a habit, not a status. The four modes stand, each with its limit: a thought partner is useful when the exchange is vāda and the person checks the claim (Mill’s condition and the Nyāya’s warning); production by default pulls towards the middle (a loss first to the maker whose own eye clients chose) and spends the saved hours on output, which for most employees the employer decides; a machine is a useful opponent, not a reliable judge (vitaṇḍā); research errors often come from a frame that keeps the world out. Kavitha reads everything that carries her name; what varies is the checking beyond reading. The trust ladder matches checking to consequence, between Kauṭilya’s distrust and O’Neill’s intelligent accountability; checking is not reading; the rung belongs to the output’s use, so the checking is owed again when the output climbs; some outputs carry nobody’s name and still decide something about a person; at the consequential rung the machine’s part is visible, not hidden; a checklist done for its own sake is staring at the hammer; checking is not overruling, and asks what an output rests on; a report with more claims than you can check has more than you can sign; signing without time to read is not answering, and whoever set the deadline shares what follows; the first check is whether the material is yours to hand over; every rung, the first included, asks whether the machine is telling you what is true or what you wanted (sycophancy; śreyas and preyas). The slow surrender is the loss of practice at framing, invisible in output; whether it grows into a lasting gap is a claim, not a finding (it would be shown wrong if long-time users framed new cases as well as others), and promises no market value; keep practising what you would need on the day the machine fails and what you need to check it. Heedfulness needs time, and the time must be the person’s: the saved hour belongs to whoever owns it, and a practice that assumes it is yours fails wherever it is not.

• What changed my mind: the objections that the saved hour is the argument, not an aside, and that a company can build its records so that a review cannot see its worst failures; the Prologue’s rule against defining people by what machines cannot yet do; the audit’s findings (the retrofit, Oti, the Singapore case); the sycophancy study of 2026 and the 2025 rollback; studies of machines as judges; Chapter 10’s finding that depth fills a position and does not grant one; Chapter 1’s rule for Kavitha.

• What survives: the report due at nine and finished at eight forty-seven; 2.3 against 4.1, and “Eight people recalibrating”; Elena’s corrected eye; the skilled user who can push the machine somewhere unusual; Reshma’s niece’s wedding and her new order of work; the hostile review; Kofi’s local knowledge; Kavitha’s gradient; the sleep test’s limit, and the trust ladder’s five rungs; Jun’s first day and the three days; Priya’s changed order of work; “The green dashboard did not teach her this. The wrong number did.”

Next in the series: Chapter 13: The Meta-Skills That Compound Forever.