Private AI for small business, explained by a live deployment.
Private AI starts with control over where your data is processed and who can use it. Here is how to assess a local or hybrid system, what it costs, and what our published lender deployment can—and cannot—tell you.
What is private AI?
“Private AI” is a broad label. Specify the arrangement: a model on hardware in your building, a dedicated hosted environment, or a hybrid that sends selected material to an external service. Each needs explicit data flows, access rules, and operating responsibilities.
The useful question is not whether the vendor calls it private. Ask which documents, text, logs, and backups leave your systems, who can access them, and what the contract permits.
Our published MARCUS deployment processes borrower documents locally. Selected tasks can use external reasoning on filtered text. That design is described on our security page; it is not a promise that every client deployment has the same boundary.
How is private AI different from a ChatGPT subscription?
A consumer subscription is a seat on somebody else’s server. That is not a criticism; for a great deal of work it is exactly right, and it costs a fraction of a build. The difference is what happens to the document.
A cloud chatbot processes the material you send on an outside service. Whether it fits depends on the product, agreement, retention settings, permissions, and the information involved. Personal and business accounts should not be treated as interchangeable.
A local deployment can keep document processing on infrastructure you control, but the model still needs testing against the actual job. Audit trails, access controls, and integration behavior must be designed and checked; they do not appear because the server is local.
Who actually needs private AI?
Not everyone, and the honest version of this page says so early. If nothing in your documents is regulated or confidential, private AI is insurance against a risk you do not carry, and a subscription will serve you better for less.
Start with workflows whose records have specific restrictions on third-party processing. These are reasons to investigate the boundary, not proof that every business in an industry needs local hardware.
Lenders and financial institutions. Borrower records require a documented handling decision. Our published case is B:Side Capital, where our founder is CEO; its requirements shaped the local processing and approval controls.
Healthcare practices and clinics. Assess patient-data handling, required agreements, and the administrative workflow before selecting a deployment. Our healthcare page explains the boundary around clinical decisions.
Legal and professional services. Privileged client files carry an obligation that a vendor’s terms of service does not discharge. See AI for professional services.
The test is not company size or revenue. It is whether you can name the document that cannot leave.
Local or hybrid: which arrangement fits the work?
Choose the processing boundary first. Both arrangements need access controls, maintenance, and tests against the documents your team uses.
| Decision | Local processing | Hybrid processing |
|---|---|---|
| Model inputs | Run approved tasks on your hardware. Verify that connected tools, logs, and backups respect the same boundary. | Specify which tasks may use an external model and exactly what information can leave. |
| Operating costs | Include hardware, upkeep, power, backups, and the people responsible for maintenance. | Include local costs plus external usage, vendor subscriptions, and review of those dependencies. |
| Quality check | Test the local model on representative work and exceptions before accepting it. | Compare the permitted routes on the same work. More capable external reasoning may still be unsuitable for some information. |
| When a task fails | Route it to a person if the approved model cannot complete it. | Do not silently switch to an external model. Escalation must stay within the agreed data boundary. |
Bring the intended documents and access requirements to scoping. See the private AI server build and ongoing-cost guide for the decisions behind the quote.
What does a private AI system actually consist of?
Five controls to evaluate, drawn from the MARCUS architecture. Your scope must establish which controls apply and how they will be tested.
1. A defined processing boundary. State which documents stay local and which tasks may use external services. Include backups, logs, and support access in that map.
2. Identifier filtering with known limits. A filter such as Presidio detects supported identifiers before permitted external processing. Detection can miss sensitive information; test it alongside the other controls.
3. Encryption and controlled access. MARCUS encrypts stored documents and work products. Key management, permissions, backups, and recovery still require named owners.
4. A log that can be verified. MARCUS records actions in an append-only, hash-chained log. Verification can reveal changes to recorded entries; the log does not replace retention rules, access controls, or investigation.
5. Approval for consequential actions. MARCUS prepares work for people to approve before consequential sending, filing, posting, or paying. Routine automatic actions in other builds need their own agreed boundaries.
The build that carries controls 1 and 2 is the private AI server and its data privacy filter.
Does private AI actually work in production?
This is the question most pages on this topic cannot answer, so here are figures from a system that is running rather than a diagram of one that could.
MARCUS is a back office built for B:Side Capital, an SBA 504 and CDFI lender where our founder is CEO: 14 agents across 7 departments, drawing on the institution’s process documentation. B:Side reports the results below, reconciled against its audit log, with written approval on file. This is affiliated first-party evidence, not an independent study.
For June–August 2026, B:Side reports an estimated 1,240 staff hours of preparation capacity returned, calculated from initial workflow studies; 93% weekly staff adoption by week six across 45 employees; 0 borrower identifiers reported as sent to outside models; and 100% of consequential actions approved by a person first. Returned capacity is not measured payroll savings.
Read the full scorecard and its measurement limits before using these figures in a buying decision. They describe one operation. Your workflow, adoption, review time, and data need their own test.
What does private AI cost to buy and to run?
There are two costs and they behave differently.
A defined private-AI workflow can be scoped as an AI Implementation Sprint at $18,000–$60,000, with a fixed quote before work begins. Hardware and third-party costs must be identified in the quote. A multi-department build is a separate engagement: the Full Back Office starts at $95,000. If a scoped workflow is not live in your operation within 90 days, we keep building at no charge until it is. An AI Readiness Audit can investigate the business case first.
Owned hardware avoids a provider’s per-token invoice for the local calls it handles. It still costs money to operate: electricity, administration, backups, maintenance, and replacement capacity. Selected external reasoning adds usage charges. The running-cost guide separates tools from support and staff time.
One number worth stating plainly because vendors rarely do: we buy the hardware at cost, in the client’s name, at zero markup. You own the machine.
What are the honest limitations?
Test the model on your actual documents, including difficult examples and cases it should refuse. Local and external models have different cost, capability, and operating tradeoffs; neither is the right answer just because of where it runs.
Private AI is not a compliance certificate. A documented architecture and verified logs can support a review; they do not establish compliance by themselves.
And it is a build, not a purchase. If your first instinct is that this sounds like a lot for a business your size, that instinct is often correct, and the audit is the cheap way to find out.
Where should you start?
Ask three people what they pasted into a chatbot last week, and do not punish the answer. If the answers include something that cannot leave your building, you have found your first workflow, and the rest of this is scoping.
Private AI, asked plainly.
01Is private AI the same as on-premise AI?
No. Private AI describes control over access and data handling; on-premise describes where a system runs. A local, private-cloud, or hybrid deployment can have different boundaries. Define which documents and requests may leave, who can access them, and how that is tested.
02Do we need private AI, or is a ChatGPT subscription enough?
A business subscription may fit individual drafting and research. Restricted records or workflows spanning several systems may require additional controls or a custom build. Decide from the workflow, exact product terms, contracts, and approved data routes; neither a subscription nor a private server is sufficient by itself.
03What does private AI actually cost to run each month?
Budget tools, model usage or local compute, electricity, hardware maintenance, internal review, and any support. Our illustrative $50–$500 tools-and-model allowance excludes build, hardware, staff time, and optional Managed Services from $1,500 a month. Implementation Sprints cost $18,000–$60,000, with scope and price agreed in writing.
04Is a local model good enough, or do we lose capability?
Test the chosen local model against representative documents, required answer quality, latency, and available hardware. Selected tasks may benefit from external reasoning, but only through approved data routes. Automated filtering can miss sensitive information, so it does not make every cloud request appropriate.
05How do we verify where data was processed?
MARCUS records actions in an append-only, hash-chained audit log. The chain makes changes to recorded entries detectable when verified; it does not independently validate the underlying action or every operational metric. Review the approved routes, network and application records, access controls, retention, and verification procedures. A log or a mandatory privacy filter alone does not prove that no sensitive information left the environment.
06Who is accountable when the system gets something wrong?
Name a person who owns the workflow, reviews exceptions, and can stop or overrule it. Define routine actions that may run automatically and consequential actions that require approval. In the reported MARCUS measurement window, all consequential actions were approved by a person first; that is first-party evidence from one deployment.
30 minutes. A straight answer.
Bring one workflow and its data requirements. We will discuss whether private AI deserves further scoping, including when the answer is not yet.