Choose a computer for local AI around the operation it must complete: finding a procedure, preparing a sales response, extracting fields from a PDF or assisting software development. The model, available memory and concurrent requests belong to the same decision. For an entrepreneur, a useful configuration is one the team can administer, test and use every day.
What a local AI system actually includes
A complete system includes the computer, downloaded model, inference engine, interface and document storage. A retrieval application also needs file processing and a document index. Every component consumes resources. The memory available immediately after starting an otherwise empty computer differs from what remains during a working day with browsers, documents and business applications open.
A reference budget of around EUR 10,000 can help frame the assessment for an ordinary business. The actual budget follows the selected process and may start with reusing an existing PC. A purchase quotation separates equipment, installation, integration, VAT and recurring services. This makes proposals comparable while keeping the computer's price distinct from the development of a business application.
Four parts of the investment
- Computer
Processor, memory, GPU, storage and power.
- Software
Compatible engine, model, interface and versions.
- Data
Documents, extraction and a local index.
- Operations
Access, backups and a technical owner.
Start by assessing the computer you already have
For an individual pilot, we suggest assessing a computer with 32 GB RAM, an SSD and an operating system still supported by its supplier. A small model, such as a quantised Ministral 3 3B variant, provides a starting point for a demonstration-data trial. Compatible engines support CPU inference; measure response time on that particular computer using representative files and requests.
The pilot record identifies the processor, free memory, exact model, context and time required for a complete task. When a request takes too long, first identify the stage: PDF reading, indexing, model loading or generation. An additional graphics card helps the stages it accelerates; an incorrectly processed PDF requires improvements to extraction and document handling.
An expandable workstation with a dedicated GPU
A concrete specification for requesting a quotation is a Ryzen 9 7900, an AM5 motherboard with a B650 chipset, 64 GB DDR5 in two compatible modules, a 2 TB NVMe SSD and a 16 GB GeForce RTX 5070 Ti. Complete the specification with an 850 W ATX power supply, appropriate CPU cooling and a case accommodating the exact graphics card dimensions. This is a design specification; the integrator checks BIOS support, the memory compatibility list and power connectors before delivery.
AMD documents the processor with 12 cores, 24 threads and a 65 W TDP. NVIDIA specifies 16 GB GDDR7 for the RTX 5070 Ti and 300 W graphics power. These figures describe components; measure the complete workstation's consumption at the wall. For the initial trial, choose an 8B instruct model, a controlled context and one active request, then measure the effect of expanding the workload.
When unified memory deserves consideration
Computers based on the Ryzen AI Max+ 395 provide a different architecture. A compact configuration with 128 GB unified memory and a 2 TB SSD provides a shared memory pool for the processor and integrated graphics. Framework and other manufacturers document such systems for local inference. Check the operating system, engine, backend and version supporting the selected model separately.
Reserve part of total memory for the operating system, interface, documents and context. Bandwidth and engine implementation affect speed, so a large memory capacity does not directly measure response time. This configuration deserves assessment for larger quantised models, such as Qwen3.8-27B, when the task justifies choosing them over a smaller model.
Assess capacity and speed separately
- Fits
Weights + working memory + context + operating system.
- Responds
Time to first token and to the completed result.
- Serves the team
The same trial repeated with concurrent requests.
The model file is only part of the memory budget
Weights occupy the model's own storage. Additional memory is required for engine structures, working space, context caches and potentially image processing. Increasing context or parallelism changes requirements even when the downloaded file remains identical. Record the model tag, quantisation, context and active-request count together in the configuration sheet.
Ollama exposes the CPU/GPU split and allocated context through ollama ps. Start with a moderate context appropriate to the trial, then increase it for documents that require additional space. Loading an entire archive into a conversation uses unnecessary memory when the request concerns a short procedure; local retrieval can select the relevant passages before answer generation.
Storage, power, cooling and backups
A 2 TB SSD provides a starting point for the system, several model variants and pilot files. Archive volume and retained versions determine final capacity. Models can be downloaded again; configuration, approved documents and rebuildable indexes require distinct protection policies. Keep a backup on a separate device and test restoration.
Choose cooling and placement for actual office use. Sustained temperature, noise and ventilation affect comfort and operation. Size a UPS using measured consumption and the time needed for a controlled shutdown. The wattage printed on a power supply describes its capacity, rather than permanent consumption or the system's monthly electricity bill.
Accepting a configuration
- Specify
Model, quantisation, context, files and users.
- Measure
Duration, correctness, memory and wall power.
- Handover
Versions, access, procedures and a restoration test.
Request an acceptance test alongside the components
The quotation request should include a trial using the agreed model, context and documents. Measure a cold start, a second request with the model already loaded, time to first response and completion time. For a team, repeat the trial with concurrent requests. Record software versions and test conditions with the results so that the comparison can be reproduced.
WebHat uses a customised local AI system and provides consultancy on choosing a setup and adapting AI to business workflows. This assessment can support a custom application with AI integration, with data, interface and acceptance criteria agreed before equipment is purchased. For clearly bounded tasks, consider small language models for businesses as part of the assessment.
Questions about local AI configurations
Does the entire EUR 10,000 budget need to be spent?
The figure is a reference point. A pilot on existing equipment establishes requirements, and the investment follows the process and team that will use the system.
Can the memory of two graphics cards be added automatically?
Splitting a model across GPUs requires explicit support in the engine, model and configuration. The technical specification must describe that distribution and include a test of the resulting setup.
What does 128 GB unified memory mean?
The processor and graphics share the same memory pool. Operating-system and application resources come out of that total, while speed is assessed separately from capacity.
What should the completed installation include?
The equipment list, software versions, models, context settings, administrative access, acceptance-test results and backup and restoration procedures.
