How Fast is 'Fast Enough'?

There’s one very significant catch: it feels slow—especially when it starts over-thinking, but even without that it’s not particularly sprightly.
- Simon Willison on Qwen3.8-27B

The weekend was spent running a series of tests on the new Qwen3.8-27B model. First to see if I could get it running, then testing its capabilities, and finally to see if I could get it running fast.

My venerable M2 Max MacBook Pro tops out at around 11-13 tokens per second. About as fast as someone can read. That's not horrific, but if you hand it a lot to read - and I did, on one occasion - it goes away and thinks for long enough that you can make yourself a very deep cuppa before it gets back to you.

For all that, it is accurate and smart and capable. So the question becomes not can you have intelligence on a PC but is that intelligence fast enough to satisfy users?

That led to a number of other tests on other hardware: an RTX 5090, dual RTX3090s, dual RTX5060 Ti. Different configurations give you different speeds, and as you'd expect, spending more buys more speed.

Here's what spending more did not buy: better answers. I ran the same model through the same substantial task twice - once on a rented 5090 at 111 tokens per second, once on my MacBook at 11 - and, having registered in advance how I'd judge the results, found the slow run every bit the equal of the fast one. Same mind, same quality, one-tenth the speed. The extra money bought comfort, nothing else. It reframes the purchase decision: you are not buying intelligence. Anyone can have that with a 'good enough' computer. You're buying relief from waiting.

So the question centres on how long people are willing to wait. With Claude Fable you can enter a prompt and wait a minute or two for a response. That's a different experience from, say, Google Gemini, which does its best to answer within seconds. Getting that kind of response time on home hardware is not impossible - but you'd sacrifice accuracy for speed. Stupid answers come quickly. Smart takes longer. That's as true for machines as for people.

For the work that matters most - unattended, agentic work - there's a quality that matters more than either speed or brilliance: consistency. A machine that does the same careful thing every single night beats a faster one that's occasionally erratic, because nobody is standing there to catch the erratic night. The metric that counts isn't tokens per second. It's how rarely the machine surprises you. Nobody publishes a leaderboard for that. Maybe someone should?

Will people be willing to wait for smarts? That answer is coded into the way the entire interface is designed. The interface - what we now call the 'harness' - probably matters more than the specifics of the model itself, because the harness decides who is waiting. Speed is not really a property of the machine. It's a property of the observer. A response that takes four hours is agonising if you're watching the cursor and invisible if you're asleep.

Which suggests the design rule hiding inside all my weekend benchmarks: pay for speed exactly where a human is waiting, and spend patience everywhere else. The attended minute is expensive; the unattended hour is free. You see the first half of that rule today with software engineers, who will happily pay double to get answers in half the time - because they're watching. But the agent working through the night answers to no one's impatience. Eleven tokens a second across eight dark hours is 316,800 tokens of work nobody waited a single moment for. Businesses understood this for thirty years: the mainframe era ran on overnight batch, and the reports were on the desk by morning.

Computers weren't always fast. A 1981 Timex Sinclair ZX81 would execute your BASIC program line by line, spending seconds working through each statement. A slowpoke - but it got the job done. Speed came gradually, over decades, as Moore's Law delivered faster and cheaper chips. The IBM PC - released a few months after the ZX81 - wasn't particularly quick in its original incarnation either. People simply got used to the pace of the machine.

Perhaps that's where this lands. Sure, you could use a very big, very expensive, very fast AI system running in a data centre somewhere. But for most of us, most of the time, our systems will be modest and they will be slow. It won't be that way forever - but it will be that way right now.

Now that the Home Watershed has arrived, enormous effort will pour into optimising every part of it, because every optimisation opens the door to new applications. It's reasonable to expect the speeds I'm seeing today to double before the end of the year on optimisations alone. Slow is not forever.

One technical note for the optimisers: for document-heavy agentic work, the number to watch isn't generation speed at all - it's how fast the machine reads, since reading is most of an agent's diet.

So: how important is speed? Exactly as important as the number of people watching the spinning ball. Design the work so that nobody's watching - hand it over in the evening, collect it in the morning - and speed stops mattering long before the hardware gets fast. The ZX81 taught its owners the pace of the machine. This machine works while you're not looking, so its pace never shows. That's not a limitation of the Home Watershed. That's its rhythm.