Customising an Existing AI Model with LoRA and QLoRA

LoRA and QLoRA customise an existing AI model for a defined business task. The target may be enquiry classification, consistent output formatting or business terminology. For an entrepreneur, the project starts with behaviour to improve and a verified example set. Assess the investment through performance on new cases while retaining the base model's identity and conditions.

Choose behaviour worth adapting

Define the task in a verifiable sentence: routing messages to departments, producing a response structure or extracting a particular information type. Establish a baseline using the existing model and clear instructions. That trial identifies recurring errors and helps select examples supporting adaptation.

Frequently changing prices, stock and current procedures are consulted through updatable systems. Adaptation can teach a working style or format, while the application supplies current information through data and documents. These mechanisms can be combined, with their roles defined before preparing training data.

Understand LoRA and QLoRA

LoRA freezes base weights and trains additional adaptation matrices. Fewer parameters change than during adjustment of all weights. The resulting adapter is tied to the model and configuration used. PEFT documentation describes this mechanism and integration into the base model.

In a QLoRA workflow, the base is loaded in quantised form while adapters train through a compatible configuration. Training memory also includes activations, gradients and optimiser state. Inference-file size alone cannot determine the GPU needed for training. Context, batch size and implementation require joint assessment.

Prepare an accessible pilot configuration

An assessment profile can use a 16 GB GPU, 64 GB RAM and a 2 TB SSD for a compact model with controlled training context. Qwen3-8B is an example with weights published under Apache 2.0 that can be assessed in a compatible adaptation workflow. Final selection explicitly checks model, library and backend.

Unsloth documents system requirements and LoRA/QLoRA memory differences. For the environment covered by the reviewed documentation, use Python 3.11–3.13 with compatible dependencies. Install in a separate environment and retain versions. Read supplier minimum-memory examples with their conditions, then record actual peak memory in the pilot.

Build clean, representative examples

Every example includes input, required instruction and accepted output. Classification needs consistent labels. Structured writing needs consistent format and handling of missing fields. Correct contradictory examples before training and remove irrelevant data and secrets.

Separate training, validation and final testing using independent cases. Fragments of the same document or near-identical conversations must not create an indirect repetition of training examples in testing. Retain provenance, usage rights and version for each batch. A business-process owner confirms reference outputs.

Run a small, measurable experiment

Before an extended session, verify model loading and a short training batch. Initial experimental parameters can use a small batch and limited length, such as 1,024 tokens for examples fitting completely. These values define a starting trial; actual data lengths determine the necessary configuration.

Record adapter rank, target modules, learning rate, epochs and gradient accumulation. Change parameters in a way allowing meaningful comparisons. Monitor memory, training loss and validation performance. Lower loss on seen examples must be considered alongside behaviour on new cases.

Compare the adapter with the original baseline

Apply the same final test to the original and adapted models. For classification, inspect each category and business-critical errors. For formatting, check schema and field meaning. For writing, use a consistent review rubric. Retain cases where the adapted version performs worse as well.

Adoption considers quality, speed and administrative cost. The adapter becomes a system version rather than an untracked replacement file. If exported or merged with the base, repeat the trial in the inference engine. Conversion and quantisation can change the effective artefact, so verify the delivered form.

Handover must support administration

Final documentation retains the base model, licences, adapter, datasets, configuration and assessment. Define training-data access and retention. Keep the original version for rollback and treat base-model updates as changes requiring revalidation. An adapter does not automatically transfer to another model merely because its advertised size matches.

The experiment record and retained artefacts

For every experiment, retain the base identifier, adapter configuration and dataset version. Keep training, validation and test sets identifiable so data corrections remain traceable. Save preprocessing configuration and how conversations become examples, including separators and the template required by the model.

Evaluation also covers longer-than-usual examples, missing fields and similar terminology across categories. Identify truncated inputs before interpreting their results. Peak memory and completion time complement quality outcomes. Handover retains adapter files and the information needed to reconstruct the environment, under the access rules defined for project data. Keep these records with the exported version so the deployed configuration can be matched to the experiment that produced it and compared against its original acceptance criteria.

WebHat uses a customised local AI system and provides consultancy on selecting a customisation method and assessing its business value. A custom AI application can combine models, documents and working rules. Access criteria and data use are defined alongside application data protection.

Questions about model adaptation

Does this build a foundation model from scratch?

The project adapts an existing model and retains its base identity. LoRA changes adaptation parameters for a defined task.

Why does training need different memory from conversation?

Training uses activations, gradients and optimiser state alongside the model. Configuration includes example length, batch size and the selected method.

Can every training example also be used for evaluation?

The final test needs cases independent of training. It assesses behaviour on data not seen during the adaptation session.

What happens when the base model changes?

Check adapter compatibility and repeat assessment. Retain the previous configuration until the new version meets accepted criteria.