For three years, the story about artificial intelligence has been a story about size. Bigger models. Bigger data centers. Bigger capital expenditure, announced in bigger press releases. The implicit message to everyone running a business was: this is happening somewhere else, at a scale you cannot participate in, and your role is to sit down, shut up, and subscribe.
That story was accurate for a while. It stopped being accurate somewhere in the last eighteen months, but nobody sent out a memo.
Here is what actually shifted. Training a model — the enormously expensive part, the part that requires a building full of GPUs — is no longer where most of the computing happens. Deloitte's 2026 outlook puts inference, the act of actually using a trained model, at roughly two-thirds of all AI compute, up from about a third in 2023. Training is a capital expenditure. Inference is an operating expense, and it runs every hour of every day.
The good news is that change may finally make it practical for ordinary businesses to turn decades of their own operating knowledge into something their entire company can use to improve business performance.
Making Local AI Viable
The move toward inference may sound like an infrastructure detail. But it's actually a big deal, because training and inference want to live in completely different places. That shift in compute, along with the three major developments below, is driving a capability that used to be a pipedream for AI: a viable local AI solution:
- Models got small enough to be useful. In 2023, a model small enough to run on ordinary hardware was a novelty that struggled to follow instructions. By 2026, well-trained smaller models handle classification, summarization, extraction, and structured output at a quality most enterprise use cases accept. Many business tasks are recurring rather than novel and don't need a state-of-the-art model. They are exactly the kind of narrow, high-volume tasks where a small model that knows your business can be more useful than a large general-purpose model.
- The hardware caught up. Multiple companies have been quietly building for on-premise inference — Lenovo launched three inference servers at CES this year, AMD announced a GPU specifically for on-premise enterprise inference, and Dell and HPE both shipped inference hardware the year prior. That is not one company betting on a trend. That is four companies with enormous research budgets independently concluding the demand is real.
- The economics inverted. When AI use was occasional, per-token pricing was cheap and obviously correct. When a model runs continuously against your operations, per-token pricing is a meter that never stops. Inference runs around the clock, and costs accumulate by call and by token — which is precisely why businesses are now revisiting whether that workload belongs on their own hardware rather than a pay as you go model.
That doesn't mean everything is moving out of the cloud. It isn't. But the direction is measurable. Broadcom's 2026 survey of 1,800 IT leaders at large enterprises found public cloud's share as the primary home for production AI inference fell from 56% to 41% in a single year. Security and compliance, cost predictability, and performance were among the reasons.
That's a survey of large companies, and it's published by a vendor who sells the alternative, so take the framing with the appropriate grain of salt. But these are organizations with procurement departments and cloud contracts negotiated at a scale you and I will never see. If the cost curve is showing up there first, smaller businesses should at least be paying attention.
Why This Matters to Your Business
What I think these trends mean over the next five years is this: the companies with a durable advantage will be the ones whose institutional knowledge lives in a system they own.
Every business of any age owns an asset that probably doesn't appear anywhere on its balance sheet, something genuinely proprietary. Its knowledge. A contractor has twenty years detailing what jobs actually cost. A manufacturer has decades of information around which process settings produce scrap. A distributor knows which customers pay and which ones argue. Unfortunately, that knowledge is currently scattered across spreadsheets, old invoices, and in the heads of three people who are all somewhat overworked and none of whom are getting younger.
For the entire history of business, there was no practical way to make that knowledge operational — to put a twenty-year veteran's judgment into a system that the newest hire can draw on. There is now. And the version where it runs on hardware you own is meaningfully different from the version where you upload it and rent access back.
Not primarily for security reasons, though keeping sensitive data inside your own environment eliminates one main source of exposure. The real reason is ownership. If your operating history becomes the training material for a platform you subscribe to, you've handed your only durable advantage to a vendor and agreed to pay monthly for the privilege of using it. Five years in, with your entire process built around their system, you'll discover what your negotiating position actually is.
What the Hype Leaves Out
To be clear, this is not right for every company, and I'd distrust anyone who says it is. But the usual disqualifier — you're too small — is the wrong test, and it's getting more wrong every year.
What actually matters is whether the same judgment repeats often enough to be worth capturing, whether your data is in good enough shape to learn from, and whether the knowledge is valuable enough to justify making it operational. A four-person medical billing practice can clear all three hurdles easily. A sixty-person company with a simple price sheet and no data discipline may clear none of them.
If your data is a mess, no AI of any kind will help you — it will produce fast, confident, wrong answers. Fix that first. That relatively unglamorous work may capture a surprising amount of the available value by itself.
And honestly, the real question isn't cloud or local. It's which work goes where.
Most business tasks are involved without being intellectually challenging. Assembling a forty-line bid from a price book takes real work — pulling the right numbers, applying the right exceptions, catching the omission. But it doesn't take a frontier model. It takes a system that knows your pricing cold and never gets tired at four on a Friday.
So the sensible architecture for most operating businesses ends up split. The high-volume, recurring, sensitive work runs locally, where it never leaves the building and costs nothing each time it's used. The occasional hard reasoning goes to the cloud, where you pay for it when you need it. You end up with a much smaller cloud bill than a subscription model would have produced, because the tasks driving your volume aren't the ones you're paying per token for.
Taking the First Steps
You can work out roughly where you stand in less than an hour with honest answers to four questions.
1. Where does the same judgment repeat?
Find the task in your business where a person makes a similar decision over and over, drawing on the same body of knowledge each time. Pricing a job. Reading a spec and deciding what it'll take. Approving an invoice against what the work should have cost.
Count how many times a month that happens and estimate how long each one takes. Forty a month at ninety minutes is six hours a week, most of it spent looking things up rather than deciding. That's the shape of a real candidate.
2. Is the knowledge stuck in someone's head?
If the answer to question one runs through one or two specific people, and everything queues behind their availability, that's the constraint. It's also the strongest single indicator, because it's the one problem hiring doesn't solve quickly — a new person takes years to learn what the veteran knows.
If five people can already do the task equally well, the payoff is smaller. Real, but smaller.
3. Could you actually hand someone your data?
Pull three years of completed jobs. Look at whether the pricing is consistent, whether the actual costs got recorded, whether the same work is described the same way across jobs.
Be honest. Most companies fail this one, and it's the most common reason these projects disappoint. If your data is inconsistent, that's your project — not AI. Clean it up over the next few months. You'll find margin you didn't know you were losing, and you'll be ready when you do want to move.
4. Do you want to own it?
Separate from the arithmetic. If it matters to you that your operating history remain under your control rather than on someone else's servers under terms they can revise, count that as a legitimate factor in the decision. This is the same instinct that leads people to buy their building rather than lease it on better terms.
Another consideration: with hardware, your next upgrade is an equipment swap you make on your own timeline, not a renegotiation with someone who knows how dependent you've become on them. The equipment you're replacing may also retain resale value that can help fund the upgrade — something a subscription you've been paying for leaves behind.
Two or three clear yeses with data in reasonable shape means it's worth looking at seriously now. Yes on recurrence and bottleneck but messy data means the data is your project — clean it up, then revisit. No clear recurring task means not yet, and that's a fine answer.
However, if you're deciding based on one task, widen the lens before you conclude no. Most companies find several once they start looking, and the case gets stronger with each one because, unlike per-token pricing, the cost doesn't increase every time an employee uses the system.
Note that the bar is also falling on a schedule. Every force described earlier on — smaller capable models, cheaper inference silicon, better tooling — pushes the minimum viable scale down each year. Whatever the floor is today, it's lower than it was when you last looked, and it will be lower again next year. So a “No” this year is a “No” for this year, not a permanent one.
Why Not to Wait
Wherever you land, there's a reason not to wait on the analysis.
The decisions that constrain you aren't the ones about AI. They're the ordinary ones you'll make in the next eighteen months without thinking twice. The three-year renewal you sign without reading what the vendor can now do with your operating history. The cleanup project that keeps sliding because nothing is on fire. The fact that your pricing still lives in one person's head, and she's fifty-eight.
None feels urgent. All become considerably more expensive to fix after the fact. The owner who spends a Saturday morning on the four questions above and concludes “not yet” is still in a better position: he knows what his contracts say, he knows whether his data needs work, and he knows which retirement could take a chunk of the company's institutional knowledge with it.
So take the time. If you'd rather not do it alone, find someone who's been through it and won't flinch at telling you the truth. Then act on what you find.
Nobody is going to send you a notice when the economics start working for a company your size. There's no press release for that, and the vendors you already pay have no particular reason to bring it up. You'll find out because you went looking.
For businesses willing to look at what they already know — and what that knowledge could become — this may be a good time to start the conversation.
The revolution didn't announce itself. It rarely does.