A local AI system for a team operates as an internal service: employees use a browser or application while the model runs on the computer selected by the business. Implementation must connect access, documents and available capacity. Five registered accounts and five requests executing concurrently are different requirements that need separate checks before launch.
Start with the team's activities
Identify who will use the system, for which operations and at what times. A sales office may prepare short responses throughout the day while a document batch runs after hours. A technical team may submit longer requests containing source files. These profiles require different resources even when employee counts are identical.
Separate interactive conversations from batch tasks and record acceptable duration for each. A workstation with 64 GB RAM, a 16 GB GPU and an 8B instruct model is a starting pilot profile for short requests. Active-request count and context are trial parameters. Establish them by measurement on the installed configuration.
The internal service architecture
- Employee
Browser or application with an individual account.
- Controlled access
Approved network, HTTPS and authentication.
- Application
Permissions, documents, queue and request records.
- Local engine
The business workstation executes the model.
Build the access path across the business network
The model computer can host both the inference engine and an internal web application. Employee devices display the interface and submit requests; the model remains on the central workstation. Each employee laptop therefore does not require the same graphics card. Network stability, authentication and central-computer availability become service components.
When application and engine run on the same computer, the engine can remain reachable only locally. Expose the interface through a controlled reverse proxy with HTTPS and access from the approved network. Remote employees can connect through a VPN. The firewall allows necessary connections, while the engine port stays outside public access by unknown users.
Accounts and roles belong in the project
Every employee uses an identifiable account. Administrators manage the application and configuration, while users receive functions and collections required for their work. An interface such as Open WebUI can provide user and permission management according to the installed version. The conversational model remains separate from authentication.
Test explicitly as an ordinary user as well as an administrator. A document visible to the administrator may be hidden from another role. When an employee leaves, disable their account and revoke associated sessions or keys. Configure business-identity integration so that access changes also apply to the AI application.
Documents require their own permissions
Organise collections by responsibility: shared procedures, sales documentation, technical materials or restricted documents. The application checks read permission before selecting passages for the model. A prompt instruction asking the model to hide colleagues' documents does not replace filtering enforced in application code and retrieval.
A useful test uses two accounts and the same restricted document. The authorised account must retrieve the information; the other must be denied both the file and indexed results. Repeat after role changes and reindexing. Preserve document permissions when splitting the document into fragments.
Apply access controls before answering
- Identify
The user's account and role.
- Filter
Documents and fragments they may read.
- Generate
The model receives authorised context only.
- Check
The result preserves sources and application rules.
Size active requests and context together
Multiple conversations can use one loaded model where supported by the engine, but active requests need resources for their context and execution. A large token allowance per request can leave less memory for parallel work. Record the model, quantisation, accepted context limit and active-request limit in the service specification.
Run the same trial first with one request, then with two concurrent requests and, where resources permit, an additional level. Measure completion and queue time for each user. Accepted capacity is the level satisfying business criteria in that trial. Account count remains an administrative measure rather than a statement of system throughput.
Queue tasks that can wait
An application-level queue can retain requests until resources become available. Tasks receive an identifier, status and result, and users see whether an operation is waiting, active or complete. Batch jobs need a size limit and a recovery policy that prevents executing the same operation twice.
Interactive work and batch processing can have distinct windows or priorities within the integrated application. For example, archive indexing can run after hours while short conversations remain available during the day. Implement and test that organisation explicitly; a chat interface does not automatically provide every scheduling rule the business requires.
Capacity has three different counts
- Accounts
Who may use the application.
- Waiting requests
Which tasks are retained in the queue.
- Active requests
What the engine executes concurrently.
Operate it as a business application
Assign responsibility for checking free space, processes, backups and errors. Test updates on a copy before activation and keep the previous configuration for rollback. Operational records can include model, duration, status and request identifier. Retain conversation content according to the purpose and policy defined for the system.
WebHat uses a customised local AI system and provides consultancy on team access, capacity and integration. A custom AI application can include business-specific queues and roles, while AI application data protection is designed alongside permissions. Acceptance checks operation from different accounts, concurrent requests, restart and restoration.
Questions about team use
Does every employee need a GPU?
With a central service, the GPU belongs to the workstation executing the model. Employee devices require access to the approved interface and network.
How should concurrent-user capacity be chosen?
Through trials using the actual model, context and tasks. Set active-request limits according to memory, response times and team working rules.
Can colleagues working from home use the system?
Yes, through a configured business-access path such as a VPN, with individual authentication and permissions. Test connectivity from the devices actually used.
What happens when the central workstation is switched off?
The AI service becomes unavailable. The application must report the correct status, retain tasks according to policy and provide the working path defined for that situation.
