A Local AI Assistant for Company Documents

A local document AI assistant helps employees find answers in procedures, approved quotations, manuals and internal documentation. The system retrieves relevant passages, supplies them to a business-hosted model and builds the answer around them. Its practical value depends on tracing information to the correct document and version while respecting each user's access.

Choose a collection the business can manage

Start with a clear domain: sales procedures, documentation for one product or departmental working instructions. Select current documents and assign someone to approve them. A bounded, well-managed collection makes answers verifiable and exposes extraction problems before the entire archive is included.

For each file, retain title, owner, version, effective date and access rights. Where two editions exist, establish which is active and how history is handled. An answer based on a withdrawn procedure can be fluent yet unsuitable for current work. Version control must exist within document management before text generation begins.

Configure every stage locally

A local workflow includes file storage, text extraction, search representations, the index and conversational model. The chat interface connects to these components. Downloading the conversational model handles just one stage. Check the embedding provider, OCR service and any optional reranking component as well.

A pilot can combine an 8B instruct model, a compact embedding model and an SSD-hosted index. Qwen3-Embedding-0.6B is a multilingual candidate also available in Ollama's catalogue. Keep its licence, artefact and version in the technical record. Measure Romanian retrieval quality on company documents separately from conversational-answer quality.

Convert files into useful fragments

PDFs containing selectable text and scanned documents require different paths. For text PDFs, check extracted paragraph order and tables. For scans, add OCR and inspect numbers, diacritics and columns. Docling documents local processing and advance model preparation for offline operation. Validate the OCR engine against the document types actually used.

Split content according to headings, sections and meaningful units. Retain document identifiers and page references for each fragment. A paragraph describing an exception must remain connected to the rule it modifies. Fragment size and overlap are application parameters checked through representative searches rather than universal values copied across every archive.

Indexing and questions need matching representations

An embedding model represents text as vectors for comparison. Documents and questions must use a compatible configuration: the same model, dimension and relevant instructions. Ollama exposes this operation through /api/embed. Changing the model or dimension requires controlled index rebuilding and repeated retrieval tests.

The index also retains filtering metadata: collection, version, permissions and status. A product code can provide an additional filter alongside semantic similarity. Test similar commercial names and document series with dedicated cases. This combines the question's meaning with the archive's concrete rules.

Make the answer verifiable

The application selects permitted fragments and places them in context alongside the question and answer rules. The interface can display the document, page and passage used. Employees can then open the internal source and check a condition before applying it. Generate references from system metadata rather than allowing the model to invent filenames.

When selected passages do not cover the question, the application requests additional information or routes the enquiry to the responsible person. A procedure might describe standard delivery without defining the exception a customer requests. Preserve that distinction between a documented condition and a case needing a human decision.

In a demonstration trial, ask for a product's delivery conditions and check that the answer retains both the rule and its exception. Rephrase the question, then substitute a product absent from the collection. Retain the passages selected for each request. This comparison shows whether a problem concerns retrieval, context use or information missing from the documents.

Enforce access before retrieval and generation

Authenticated users receive only documents and fragments permitted by their role. Apply filtering within data access and retrieval. Test identical questions with two accounts holding different permissions, including passages, previews and downloadable files. A document hidden from the list must also be excluded from the context sent to the model.

Withdrawing a document requires updating its file, index entries and any application-managed cache copies. Define deletion procedures and retention periods per component. Treat backups and restoration so that a withdrawn version cannot become active again without the collection owner's approval.

Measure retrieval and answer quality separately

The acceptance set includes known-answer questions, alternative wording, similar documents and cases with insufficient information. First check whether the correct passage appears among retrieval results. Then check whether the answer preserves its conditions, figures and exceptions. These two assessments identify where the system needs improvement.

WebHat uses a customised local AI system and provides consultancy on document organisation and integrating an assistant into business work. Implementation can form part of a custom application with AI integration. Access and retention rules are designed alongside application data protection, while handover documentation retains models, indexes and acceptance trials.

Questions about document assistants

Must the model be trained on every file?

In a RAG workflow, documents are indexed and passages supplied on request. Collection updates follow import and indexing rather than turning each file into a training session.

Can it search Romanian documents?

Select a multilingual configuration and test Romanian documents. Include diacritics, internal terminology, product codes and alternative phrasing.

What changes when the embedding model changes?

The index representation must remain compatible with question representations. Apply the change with reindexing, tests and a reference version for rollback.

How is offline operation verified?

Prepare local models and services, restrict external connections and test import, retrieval and answering. A simple conversation verifies only part of that workflow.