NER: Extracting Entities from Documents with AI

NER, short for named entity recognition, identifies text spans that represent entities and assigns types such as organisation, person, location or date. In a business, this component can support document organisation, preparation of records and finding information that needs checking. Its value comes from keeping every proposed value connected to the document that supports it.

The important distinction is between detecting a name and deciding what to do with it. A model may find a company name in an enquiry. The application must establish whether it is the correct partner, what role it plays in the document and whether the user may save that information to the CRM. These are separate checks.

This guide follows a proposed workflow from document to verified record. Examples and figures are invented for explanation, not customer results. Photographs show archiving, software development and infrastructure; they do not depict documents or internal systems belonging to a company where we implemented NER.

What does NER mean and what does it return?

An entity is a typed text span, not a complete commercial record

A NER result connects a portion of text to a defined category. For an organisation, it retains the name as written in the document and its position. Some implementations also return their own prediction scores. Interpret those scores according to the particular system; they neither verify an organisation's legal existence nor replace evidence in the document.

Entity recognition belongs to natural language processing for business. It differs from summarisation, translation, sentiment analysis or identifying a message's intent. One workflow can combine these functions, but their outputs need separate fields, definitions and checks adapted to each task. Otherwise, a convenient-looking record can conceal several different kinds of inference.

In our demonstrative passage, the model may mark the organisation, location and date. Associating the date with delivery involves interpreting a relationship between those pieces of information. Confirming the business partner involves identification in the company's records. A useful interface exposes these distinctions instead of displaying every field as an already approved fact.

NER, OCR and field extraction: where does each begin?

First, the application needs the document's text in a usable form

OCR recognises text within images, for example in a scanned document. NER analyses available text and labels entities. A digital file may allow direct text extraction without OCR. Choose the route according to the file's actual contents, not merely its extension or the way it appears in a browser. Both routes require checks of the extracted material.

A recognition error can alter a name or date before entity analysis starts. Our proposed application therefore retains a reference to the original page and supports visual verification. If reading order is incorrect in a document with columns, improving the NER model alone does not automatically repair its input text. The problem belongs to an earlier processing stage.

Extracting a commercial field is a broader requirement. A delivery-date field needs both detection of a temporal mention and interpretation of its role. A supplier field needs an organisation and its relationship with the document. Define the components separately so that errors can be located and corrected at their source, rather than hidden behind a single extraction score.

How do we choose entities relevant to the company's documents?

The schema follows the decision the application is intended to support

A general schema may include people, organisations and locations. A business project may need different concepts: equipment names, internal codes, order references or operational sites. Some values are better checked with rules and controlled lists than a model. Not every string needs to become a new AI category, particularly when its accepted format is already precisely defined.

For each type, establish its definition, span boundaries and exclusions. Does a company entity include its legal suffix, or just the trading name? How should abbreviations be treated? What should be labelled in a header repeated on every page? An annotation guide resolves these questions before training or evaluation, allowing reviewers to apply the same criteria consistently.

This choice relates to types of artificial intelligence through the actual task, not a marketing label. A workflow may combine rules for structured identifiers, a model for names in free text and application validation. Compare alternatives using relevant documents and error types, rather than the number of technologies included in the proposal.

A useful exercise is to have people familiar with the process annotate the same document. Discussing their differences exposes incomplete category definitions. If reviewers disagree about a span, clarify the rule and examples before asking a model to apply them consistently. Do not conceal disagreement by arbitrarily choosing a label and treating it as settled reference data.

How does a model recognise entities within a sentence?

Labelling spans requires coherent boundaries and appropriate context

Some models approach the problem as token labelling: each unit receives a role inside or outside an entity. Other approaches propose spans directly. In either case, output must be mapped back to the original text. Tokenisation and splitting words into subunits must not result in truncated names or incorrect positions when the application reconstructs the proposed entity.

The spaCy documentation describes non-overlapping entities for its EntityRecognizer component. That is a property of that particular component, not a universal restriction on every NER system. If a project's schema requires spans nested inside other spans, evaluate a representation and method compatible with that explicit requirement. A technical component's assumptions must match the intended annotation policy.

An LLM can propose structured information from a document, but correctly formatted output does not establish that its values are accurate. Choosing between RAG, fine-tuning and prompt engineering depends on the task, data and observed errors. Retrieving the relevant document is not a substitute for checking the fields subsequently extracted from it.

Why do context, position and document version matter?

A proposed span must remain traceable to the analysed source

The same name may occur in a header, signature and partner list. Its position and surrounding text help the reviewer understand the mention's role. Do not automatically select the first occurrence as the supplier, or assume every mention refers to the same organisation without an association rule. A readable evidence trail is more useful than an unsupported automatic match.

The W3C annotation model distinguishes selecting a text span by position from the representation of the document it refers to. If the text changes, those positions may identify different content. In our proposed design, the analysed version and supporting span are retained; a source revision triggers review of the affected fields rather than silently retaining stale evidence.

Long documents can be divided for processing while preserving their connection to the page and source. Context supplied by a heading or section must be considered before splitting. In particular, check whether a split separates a date from the phrase explaining its role, or an organisation from wording that establishes the relevant commercial relationship.

Normalisation and partner identification are separate operations

A similar name is not sufficient grounds for merging records

Normalisation converts a value to an agreed representation: a date into an internal format, duplicated spaces removed under defined rules, or an abbreviation resolved through a controlled list. Retain the original value for review. A change in representation does not establish company identity and does not justify rewriting a trading name found in the source document.

Entity linking associates a mention with an identity in a knowledge base or internal register. Within a CRM, a search can propose candidate records. Confirm the match using identifiers and contextual evidence authorised for the project. Where evidence is insufficient, keep the information as a pending proposal instead of changing a partner selected only because its name is similar.

OWASP separates validation of form from checking a value's meaning for the requested operation. Apply that distinction to project fields: date format, value type and permitted relationships are checked before saving. An identifier with a valid format does not grant the user access to its associated record. Access control remains a distinct application responsibility.

How do we integrate NER with a CRM while retaining control?

The interface proposes; the application validates and executes permitted actions

For a custom CRM for business, we propose a review area displaying the document, extracted fields and candidate partner. An operator can see which values are new and which would replace existing data. Important fields should not disappear inside a general model-generated summary, where the consequences of approval would be difficult to inspect.

On save, the server checks types, accepted fields and the user's permissions for that record. Approval applies to the values actually displayed. If either the document or proposal changes, the application reassesses the required control. An “approved” message within generated output is not technical authorisation and must not be treated as a substitute for it.

A repeated operation needs to be recognised by its identity so that it does not create duplicate partners or requests. When a response is delayed, check the effect before repeating the write. These rules belong to business automation with AI integration, alongside error handling and reconciliation with the source system.

How do we measure the correctness of entity extraction?

Check complete spans and types, not just easily classified tokens

Build a manually reviewed reference set using the definitions in the annotation guide. Keep the test set separate from examples used to adjust the system. In supervised learning, reference labels are an essential part of evaluation rather than a formality added after selecting a model. Their consistency affects what the resulting scores actually describe.

Our demonstrative scenario has 30 reference entities, 28 proposals and 24 exact matches. Precision is 24/28, approximately 85.7%, and recall is 24/30, or 80%. An incorrect boundary can create both an unmatched proposal and a missed reference entity. These categories should not be added together and described as separate documents, because they represent entity-level outcomes.

Report organisations, dates, locations and critical fields separately, then examine language, document type and scan quality. Token accuracy includes substantial text outside entities and is not equivalent to exact matching of complete entities. Measure operational outcomes as well: fully correct records, review effort and cases appropriately referred for clarification. A strong component score can still coexist with costly integration mistakes.

When analysing errors, distinguish an omitted name, an overlong span, an incorrect type and an incorrect association with the final record. The last problem can occur even when NER worked correctly. The evaluation record should identify the responsible stage and proposed correction so that improvements reach the appropriate component rather than changing unrelated parts of the workflow.

How do we protect documents and access to their extracted data?

Ordinary NER does not certify removal of personal information

A schema for organisations and dates does not automatically cover every item of personal information in a document. Incomplete detection and context require separate assessment. If data protection is the purpose, define the types sought, transformations, checks and access rules. Do not declare a document safe simply because a model returned no person names.

Plan data protection within an AI application around the actual workflow: which documents enter, who may read them, which components process them and which outputs are retained. Review records should contain what is needed for verification, rather than complete copies of every source document and processing request. Retention choices belong in the design.

Check access when displaying supporting spans, searching and exporting as well as when saving. Seeing a partner's name does not automatically entitle a user to read all its documents. An integration operation also respects the separation between companies or departments established in the application, rather than relying solely on the general permission to sign in.

What does a useful pilot look like for your business?

Start with a document family and a clear decision: preparing a request, filling specified fields or organising an archive. Inventory the formats and choose a representative sample, including incomplete documents and difficult scans. Exceptions belong in the requirements; they should not be removed merely to produce a more impressive demonstration or a conveniently clean test set.

The pilot compares a simple solution with a model-based option and includes human review. Establish what may be filled as a proposal and what requires confirmation. Select acceptance thresholds for the particular field and consequence using the relevant test, rather than copying a supplier's overall percentage. The effort needed to correct mistakes also belongs in the comparison.

A useful deliverable includes the schema, annotation guide, observed errors and review path. The business can then decide whether to extend document coverage, improve OCR or modify the integration. This sequence keeps the cause of an error visible and avoids replacing an entire system when the underlying issue concerns input quality or an incomplete controlled list.

When a document template changes, retain older examples and add cases from the new format to the relevant checks. The team can see what changed and which fields are affected. Apply the same control when changing the model, extraction rules or partner-identification list, before activating the new configuration in the operational workflow.

How does WebHat approach an entity-extraction application?

We begin with the company's process and the data authorised for analysis. Define the fields, the reviewer and the final operation, then select appropriate components. NER becomes part of a workflow that includes source text, evaluation, a control interface and monitoring of the saved result. The project is specified around usable information rather than a standalone model demonstration.

The principle behind AI alignment in business applications is applied concretely: process objectives, measured indicators and access boundaries are explicit. We do not optimise only the number of populated fields. We assess whether values are correct, traceable and usable for the operation that prompted their extraction, including the checks required before they affect an existing record.

For a meaningful estimate, prepare the document types, desired fields, volume and destination system. Examples can be prepared without unnecessary confidential information. That gives us a basis for defining the pilot and integration instead of offering a general promise about automatically reading every document. It also makes scope, reviewer responsibilities and acceptance criteria easier to discuss.

Frequently asked questions about NER for documents and CRM

Can NER read a scanned document?

NER analyses text. A scan requires a stage that recognises text within the image, usually OCR. Check the quality and order of that output before entity extraction. Assess the whole workflow, because a recognition error in the scan can change the final extracted value.

Does finding a company name automatically identify its CRM record?

No. A textual mention differs from a record's identity. The application proposes candidate partners and checks authorised identifiers and context. Without sufficient evidence, the proposal goes for clarification rather than triggering a name-based automatic merge or silently selecting a similarly named company.

Can we extract categories beyond people and organisations?

Yes, a project can define its own categories when suitable data, criteria and methods are available. Structured identifiers may be handled using rules. Test the choice on the company's documents using a consistent schema and manually reviewed examples, rather than assuming every custom category requires the same approach.

Does an LLM replace every validation stage?

An LLM can propose fields and relationships, but the application checks sources, types, business rules and permissions. Structurally valid output can still contain incorrect values. Saving is a separate operation with its own control and confirmation of the resulting effect on the destination system.

How do we compare NER solutions?

Use the same test set, definitions and processing conditions. Compare matching of spans and types, errors by class and review effort. Include difficult documents and complete workflow outcomes rather than relying solely on a percentage of correct tokens, which measures a different unit.

Do we need to replace the existing CRM?

Integration depends on the current system's interfaces and rules. A separate proposal and review area may precede transfer. Access control, duplicate prevention and reconciliation of saved values are designed around the operations actually permitted by the destination system, not an assumed ability to overwrite every field.