On-premise hardware for internal AI: how to choose

Hardware for internal AI is decided by four factors: model size (parameter count), number of concurrent users, context length, and the speed you expect. The pivotal factor is memory — RAM or VRAM must hold the model weights plus context. For most enterprises, Apple Silicon (Mac Studio or Mac mini, unified memory, low power draw) is a tidy starting point; NVIDIA GPUs fit when you need very high throughput or training. This is Part 2/8 in the build-your-own internal AI series.

What decides your hardware needs?

Before debating "which machine to buy", answer four questions — they drive everything else:

• Model size (parameter count): a bigger model (7B, 14B, 32B, 70B…) is smarter but consumes more memory and runs slower. This is the single biggest variable.
• Concurrent users: one person asking occasionally is very different from 30 people typing at once. Many concurrent users need higher throughput (often a GPU or several machines).
• Context length: letting the AI read long documents, long conversations or many RAG passages consumes extra memory for the "context" (KV cache) — the longer the context, the more memory.
• Desired speed: is "fast enough to read along" acceptable, or do you need near-instant replies? Higher speed expectations demand more powerful hardware or a smaller model.

The key point: all four factors reduce to memory and throughput. Once you pick a model size and user count, the rest (machine type, RAM/VRAM capacity) follows fairly naturally. See how the pieces fit together in the internal AI system architecture diagram.

Apple Silicon or GPU?

This is the biggest hardware decision. Both run open-source models; they differ in memory architecture, power draw and the situations they suit.

For most enterprise tasks (internal assistant, document Q&A, drafting, summarizing) and moderate user counts, Namtech chooses Apple Silicon — a Mac Studio M5 Max or M5 Ultra per package: unified memory lets a single machine load a fairly large model, low power draw means it sits in a normal office, and all three packages fit in a single machine. For smaller pilots, a Mac mini M6 or M5 Pro is an option — see the Mac mini vs Mac Studio comparison. When needs lean toward very high throughput or heavy training, an NVIDIA GPU is the more sensible choice.

Sizing by scale

Namtech packages hardware into three tiers, each one M5-chip Mac Studio. Prices are current selling prices including VAT (sold and invoiced by Decorp), taken from the product pages; models are those Namtech uses per package (updated 01/10/2026). The table does not fix a user count per package — the number of users you can serve depends on model size, context length and real load, so Namtech runs a load test on your own documents before committing.

A pragmatic principle: start at the lowest tier that solves your first clear problem, measure the real impact, then expand — rather than over-buying up front. Details of the packages are on the pricing page, and every Mac Studio configuration with its estimated model size is on the Mac Studio page.

The memory rule of thumb

This is the most important part of choosing a configuration. Memory (RAM on Apple Silicon, VRAM on GPU) must hold:

• Model weights: roughly parameter count × bytes per parameter. Bytes per parameter depend on quantization — compressing the weights to use less memory. At the common Q4 level (about half a byte per parameter), a ~7B model needs only about a few GB; larger models (14B, 32B, 70B) need proportionally more. For reference, Namtech's Mac Studio page estimates roughly ~43B parameters for 36GB, ~76B for 64GB and ~115B for 96GB of unified memory.
• Context (KV cache): the memory for context — the longer the context or the more concurrent users, the more this grows.

Add the two together, then leave a safety margin for the operating system and load variation. This is a rule of thumb for a quick estimate, not an exact figure for every model — real numbers vary by architecture and quantization level, so check the specific model page on Hugging Face. Exactly which model size and quantization to pick is covered in the Model selection article.

Storage & networking

Beyond memory and processing, three often-overlooked items directly affect the experience:

• SSD: models can be many GB and must load quickly into memory at startup. The AI Box and AI Pro machines ship with a 512GB SSD and AI Enterprise with 1TB; plan an SSD large enough to hold multiple models plus the vector database for RAG (indexed internal documents).
• Internal network: users call the AI server over the LAN. A stable internal network keeps responses smooth and keeps everything within the on-premise boundary — no data pushed outside.
• UPS (battery backup): so models and services don't shut down abruptly during a power cut, avoiding data corruption and disruption.

How to lock the data flow inside the network and control access is covered in the Internal AI security system article.

# list the models on this machine
ollama list
# rough estimate: 7B at Q4 ~ a few GB; larger models need more
# pull & run a small model to measure real memory use
ollama run qwen3.5:9b

The Namtech view

Namtech deploys private internal AI platforms on Apple Silicon — one Mac Studio M5 Max or M5 Ultra per package: unified memory lets a machine load a fairly large model, low power draw means it sits in a normal office, and you move up a tier as load grows — instead of over-investing up front. Not sure between a Mac mini and a Mac Studio? See the comparison page. The philosophy is start with just enough, scale gradually: pick a configuration for your first clear problem, measure the real impact, then upgrade as needed. The next step is to choose the open-source model that fits the hardware you've picked.

Frequently asked questions

Note: The memory figures in this article are rules of thumb; package configurations and models last updated 01/10/2026; hardware and models change fast — verify the specific configuration against your real needs when you deploy.