Internal AI (private/on-premise AI) is an AI assistant that runs entirely on your own infrastructure — open-source models, data that never leaves the organization, no public AI API calls. Building it yourself takes 8 steps: (1) planning, (2) hardware, (3) model selection, (4) serving, (5) RAG over internal documents, (6) UI & integration, (7) evaluation & guardrails, (8) operations & scaling. This is Part 1/8 — the map before we dive into each step.
Quick summary
- What it is: a language model running on your own servers; data is processed on-site, never pushed to the cloud.
- Why: data sovereignty, compliance with the Personal Data Protection Law No. 91/2025/QH15 (PDPL), fixed operating cost (not per-token), and no remote "kill switch" on your access.
- What you need: a capable server (Apple Silicon or GPU) + a commercially-licensed open-source model (Qwen, Gemma, GLM…) + a RAG layer for internal documents.
- Roadmap: 8 steps, each one an article in this series.
- Not all-or-nothing: you can start with a small box (AI Box) for one team, then scale up.
What is internal AI, and how does it differ from cloud AI?
When you use ChatGPT, Gemini or Claude over the internet, every question and document you paste in leaves your organization and is processed on the provider's servers. Internal AI flips that: the model is downloaded and run on your own servers, so all prompts, documents and answers stay inside your network.
| Criterion | Cloud AI (public API) | Internal AI (on-premise) |
|---|---|---|
| Where data goes | Leaves the org, to the provider's servers | Stays on your infrastructure |
| Cost | Usage-based (tokens) — variable | One-time hardware + fixed operating cost |
| Access | Can be repriced/cut/restricted remotely | Under your control, no remote kill switch |
| Compliance (PDPL…) | Depends on terms & server location | Easy to prove data stays on-site |
| Most powerful model | Access to the newest frontier models | Open-source models (gap is narrowing) |
There's no absolute right answer. The pragmatic choice is a hybrid: use cloud AI for general, low-sensitivity work; keep core data and processes on internal AI on-site.
Why should enterprises build internal AI?
The four reasons that come up most often when Vietnamese enterprises consider internal AI:
- Data sovereignty & security: customer records, contracts, source code and financials must not go to a third-party service.
- PDPL compliance: the Personal Data Protection Law No. 91/2025/QH15 (PDPL), in force since 01/01/2026, tightens control over consent and cross-border transfers of personal data — easier to prove when data stays on-site. AI systems are covered by separate rules: see Decree 142/2026 on AI.
- Predictable cost: instead of a token bill that grows with usage, you pay a hardware investment plus near-fixed power/operating costs.
- No lock-in: cloud AI access can change by commercial or administrative decision — as in the Fable 5 global takedown. Internal AI can't be switched off remotely.
The 8-step roadmap (series map)
Each step below is a detailed article in this series. Read in order, or jump to what you need:
- Planning (this article): define the need (internal assistant, document Q&A, automation), scope and success criteria.
- Hardware: choose an on-premise machine — Apple Silicon (Mac Studio or Mac mini, compared here) or a GPU box — by number of users.
- Model selection: which open-source model, what size, and does the license allow commercial use (Apache 2.0…).
- Serving: install and optimize speed — Ollama, LM Studio or MLX on Apple Silicon (vLLM only on NVIDIA GPU servers); quantization to save memory.
- RAG: let the AI "read" your internal documents via embeddings + a vector database, answering with citations.
- UI & integration: a chat interface for staff, an API to plug into existing software.
- Evaluation & tuning: measure quality, reduce hallucination, set safety guardrails.
- Operations & scaling: monitor, back up, update, and move up between single-machine packages (AI Box → AI Pro → AI Enterprise).
| Step | Stage | Key content |
|---|---|---|
| 1 | Planning | Define the need, scope and success criteria |
| 2 | Hardware | Choose an on-premise machine — Apple Silicon or a GPU box — by number of users |
| 3 | Model selection | Which open-source model, what size, and whether the license allows commercial use |
| 4 | Serving | Ollama, LM Studio or MLX on Apple Silicon (vLLM on GPU servers); quantization to save memory |
| 5 | RAG | Let the AI "read" internal documents via embeddings + a vector database, answering with citations |
| 6 | UI & integration | A chat interface for staff, an API to plug into existing software |
| 7 | Evaluation & tuning | Measure quality, reduce hallucination, set safety guardrails |
| 8 | Operations & scaling | Monitor, back up, update, move up AI Box → AI Pro → AI Enterprise |
See also the companion posts: Internal AI system architecture diagram, Internal AI security system and Trending Pool — updating world knowledge.
A minimal, popular internal-AI stack today:
- Serving:
Ollama(fast start, one command),LM Studio(desktop app) ormlx-lm(Apple-native);vLLMonly on NVIDIA GPU servers. - Models: the
Qwen,GLM,Gemma,Llamafamilies — pick by commercial license and a size that fits your VRAM/RAM. - RAG: embeddings + a vector DB (
pgvector,Qdrant,Chroma). - UI:
Open WebUIor a custom app calling an OpenAI-compatible API.
Trying a model on a single machine takes only minutes:
# install Ollama, then run an open-source model
ollama run qwen3.5:9b # chat right in the terminal, 100% offlineStart small, scale up
A common mistake is waiting to build something grand. In practice, start from one clear problem (e.g., internal-process Q&A for one department) on one small machine, measure the impact, then expand. Namtech packages this approach into three single-machine tiers, each one M5-chip Mac Studio: AI Box (Mac Studio M5 Max 64GB, one team), AI Pro (Mac Studio M5 Max 128GB, several departments) and AI Enterprise (Mac Studio M5 Ultra 256GB, whole enterprise) — details on the pricing page.
The Namtech view
Namtech deploys private internal AI platforms running 100% on-site on Apple Silicon (Mac Studio, low power draw) with commercially usable open models — Qwen, Gemma (Apache 2.0) and GLM (MIT) — run on MLX, Ollama or LM Studio. This series shares the exact roadmap we use — so your team can do it themselves, or understand exactly what you're buying when you partner with us. Building your own doesn't mean building alone: you can own the architecture and data while a partner shortens the timeline.
Frequently asked questions
Do I need a dedicated AI team to build internal AI?
Not necessarily. With today's open-source tools (Ollama, Open WebUI, pre-packaged models), an IT engineer can stand up a proof of concept in a day. The harder parts — optimization, security, RAG and operations — are exactly what this 8-step series and partners like Namtech cover.
Is internal AI as strong as ChatGPT?
The best open-source models aren't yet at the level of the strongest frontier models, but the gap is narrowing fast, and for most enterprise tasks (document Q&A, drafting, summarizing) they're already good enough — in exchange for on-site data and fixed cost.
How much does it cost to start?
It depends on scale and hardware. You can start with a single machine for a small team, then expand. As a reference, Namtech's AI Box package lists 120–180M₫ for the software (excl. VAT) plus the Mac Studio M5 Max 64GB machine at 111,150,000₫ (incl. VAT) — see the pricing page. The Hardware article covers how to size a configuration by number of users; specific figures should be assessed against your real needs.
Does data really never leave the organization?
Yes — if deployed correctly: the model runs on-premise, outbound connections are blocked for the inference path, and access is controlled. The Security article details the defense layers.
Want internal AI without starting from zero?
Namtech deploys private internal AI platforms — open-source models running 100% on your own infrastructure, data never leaving the organization.
Book a free consultationNote: This is a general guide, last updated 01/10/2026; tools and models change fast — verify the latest versions when you deploy.