Telling Computers What To Do

Looking back from 2026. This post rests on one claim: computers are deterministic, predictable and reliable, so when they surprise you the fault and the fix are both yours. I still believe that about the machine. What changed is that the most common way I tell a computer what to do now is to ask a large language model, and an LLM is not deterministic by construction. The same prompt gives different answers on different days. Even pinned to greedy decoding at temperature zero, production inference still is not reproducible run to run, because the numerics depend on what else happened to be in the batch (Horace He and Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference”, Sept 2025; Simon Willison’s summary). So the layer I now spend most of my day talking to is the one piece of the stack that genuinely does not do what you tell it.

The determinism did not go anywhere though. It moved down a layer, and the job is getting the probabilistic thing to hand its work back to the deterministic thing. Concretely: do not ask the model for the answer. Ask it to write the program that produces the answer, then run the program.

When the models were dumber this was a party trick. Ask one how many R’s are in “strawberry” and it would tell you two, because it does not see letters, it sees tokens like “straw” and “berry”, and counting characters inside a token is not an operation it can do reliably (“Counting Ability of Large Language Models and the Impact of Tokenization”). Ask that same dumb model to write a python script that counts the R’s in a string and it hands you s.count('r') every time. That script is correct and its output is reproducible forever. The model was bad at being a computer and good at telling one what to do.

Current models get “strawberry” right, so it is tempting to think the trick stopped mattering. It moved up the difficulty curve instead. For anything genuinely hard, a nontrivial calculation, parsing a large file, cross-referencing two datasets, any real arithmetic or bookkeeping, asking for the answer directly is asking a probabilistic system to freehand a result it has no way to check. Asking for the script and running it puts a deterministic, inspectable step between the question and the result. That is the finding behind program-aided language models: have the model generate the code and offload the computing to a Python interpreter, and you beat much larger models doing chain-of-thought in their heads by something like 15 percentage points on grade-school math word problems (Gao et al., “PAL: Program-aided Language Models”, 2022). It is good at translating “what I want” into “the steps to compute it” and bad at being an ALU. So stop making it be one.

Which leaves the 2015 argument intact underneath. The finite-state machine is still down there doing precisely what it is told. There is a probabilistic layer on top of it now, turning my sloppy intent into instructions, and that layer gives “something is going on that I don’t understand” a new front door: the model can hand me a script that is subtly wrong, or narrate an answer it invented. The defense has not changed. Do not trust the narration. Make it show you the program, read the program, run the program. The program does what you tell it, because it is a computer. The 2015 argument below is unchanged.

Two of my favorite sayings are, “I tell computers what to do for a living,” and, “Computers do what you tell them.” Most non-technical folks find these sayings funny or quaint, but they mean something very specific and important. When a user says that the computer deleted my file, there are only two things they can mean. He or she told the computer to delete the file, or someone else told the computer to delete the file. Saying that computers do what you tell them, reminds even the most technical people that computers are, with very few exceptions, deterministic, predictable, and reliable. Which means when they don’t do what you expect, it is not only likely to be your own fault, but also absolutely in your power to fix it. Always. Because computers do what you tell them.

Of course, the point isn’t to provide apologetics for why today, after 60+ years of computer science, it is still so very hard to get computers to do what you want. In fact, in many ways it is harder than ever before. This is about a deep truth of how computers work. When a junior engineer says, “It won’t do X,” or, “I can’t make it do X,” what they mean is, “Something is going on that I don’t understand.” But, you CAN understand it if you can figure out what questions to ask, and how to ask them. You don’t have to ask your boss or your mentor or your technical support line - you can ask your computer. Presumably, it knows. As we experience progress, and experience increasingly complex hardware running increasingly complex operating systems running increasingly complex application stacks written in increasingly complex higher-level programming languages, our understanding of the computer becomes more and more opaque, confusing, detached, and mysterious. We must fight that inaccurate view, however, because underneath that complexity still beats the 3-billion-times-per-second frequency of a deterministic, finite-state machine that would have made Alan Turing proud.

Much like wrong answers on an essay question in English class, there are uncomfortably many ways a computer might break your expectations and manifest an apparent failure to do what you think it should. There are a quite finite number of things, however, that could be the underlying cause, and narrowing it down is the first step to fixing it. Figuring out that underlying cause, quickly and methodically, is what separates the good engineers from the great ones, and what contributes to those few who are 10x or 100x engineers.

In the olden times, everything about a computer was laid bare, and your typical computer operator could describe the state of every register and exactly what the computer was doing. You couldn’t do much without this deep level of understanding. Today, things are not like that. I don’t know how a BIOS works. I’m not sure what happens when I press the power button on my laptop and the electronics whir to life to complete their Sisyphean task of delivering me pictures of cats falling into water and looking pretty unhappy about it, but I am grateful that it works. Somehow, all the registers go from random states of not having enough electrons, to an appropriate, deterministic state where the completely non-physical emergent phenomenon we call an operating system can birth itself from the chaos. I also don’t know exactly how my operating system works, but since I use Linux, which is open-source, at least I can look at the code and try to figure it out. “One of these days…” I always say… and that is just one layer. You probably don’t know how your hard disk works. You probably don’t know how your keyboard delivers I/O to your OS. You probably don’t know how your CPU manages processes or memory. You probably don’t know how your kernel’s network stack delivers requests to and from applications like your web browser. You probably don’t know how the video website’s client-side code makes continuous requests to refill its buffer with video as it plays, so the cat falls gracefully into the water and does not stutter.

This is why people personify computers. They are as inscrutable to us as the network of billions of neurons in your brain that consciousness mysteriously arises from. This is why people think “they” do “things” to “us”. Today, a single person trying to understand everything about their computer seems more like a person trying to know the state of every neuron in a living brain (how would that work? Think about it).

I know people who I consider to be 10x engineers. Their talent amazes me. How do they always seem to know? How do they run circles around the rest of us, and just get so much done? From my experience, my guess is that they ask the right questions, and they ask it of the right entity - their computer. They (often) don’t settle on shoving random print statements into code - they lay the entire thing bare with a debugger. They read and understand every line of code between the moment they press “enter”, and the moment the computer “does something wrong”. And I think they do this because they understand that “computers do what you tell them”. Whatever is wrong, it has to be in there somewhere. I mean, short of a bad motherboard or some spotty ram. And good engineers think about that too. I knew one who, after losing an entire week debugging non-deterministic behavior, only to find the ram in his new machine was faulty, swore that he would run memtest86 on every new machine he bought or got from work annually, for all time. For the price of a few hours of burnin when he wouldn’t have been using it anyways, he can save himself countless hours of wondering if he is, in fact, taking crazy pills (a detrimental state to find oneself in for certain).

So what is my advice to those who would tell computers what to do for a living? The next time the computer doesn’t seem to be listening, maybe you should be the one who is listening, and asking the right questions. Have you read the logs? What versions of the software are you using? What has changed since it worked last? Can you examine the source code? Can you examine the state of the memory? Does it fail deterministically? If not, can you detect a pattern in the failure? Can you produce a simplified reproduction of the bug? Maybe you need to invest some serious time to answer some of these questions. Maybe you need to write a debugger or disassembler. Maybe you even need to go learn assembly, or Java bytecode. Your boss should be happy to “pay” you for your time in these endeavours, because the time you spend today writing these tools or learning how these tools work, instead of “jiggling it until it works”, is paid back to you and your employer tenfold, or even a hundred fold, and those are dividends worth having.