Meta outlined its consultation plan for proposed changes to its commercial terms on 18 November 2026. The company says the proposals had already been put on hold following concerns from news publishers and other stakeholders about web crawlers. The new announcement describes consultation and revision stages, with further discussions expected in early 2027 before revised terms are rolled out. For businesses publishing on their own websites, both the status of the proposals and the different purposes of Meta's crawlers are relevant.

The proposals remain on hold through a three-stage consultation

The first announced stage is listening to publishers and other stakeholders. Meta says briefings and discussions have begun with news organisations across markets and industry groups. The stated objective is to identify specific problems raised by the proposed wording. The company expects these conversations to continue over the following months. The announcement does not establish a new, fixed effective date for the revised commercial terms.

The second stage involves reviewing feedback and revising the proposed language. The third is further engagement with publishers, other stakeholders and regulators, expected in early 2027. Meta also plans an explanatory document to accompany the revised terms. This is a timetable for consultation, rather than confirmation that a final text has already been approved and is applicable to every account or business using Meta products.

Meta says account holders will be notified before new terms take effect, with a period for review. Accordingly, 18 November is the date of the public explanation of the process, not the date on which the paused proposals took effect. For digital administration, distinguishing a proposal, a notification and implementation matters: each stage corresponds to a different document and a different point in time.

A consultation timetable, not an effective-date schedule

  1. 18 November 2026

    Meta explains the process; proposals remain on hold

  2. Following months

    Discussions and identification of concerns

  3. Revision

    Feedback review and wording changes

  4. Early 2027

    Further engagement before rollout

Close-up of a smartphone resting on a laptop
Ahmed Lishane

Link-preview crawlers and AI crawlers serve different purposes

Meta's technical documentation identifies FacebookExternalHit as a crawler used primarily when a link is shared in applications such as Facebook, Instagram or Messenger. It collects preview information, including the title, description and thumbnail. For a commercial website, these elements determine how a shared page is described. Their quality depends on the information supplied by the page and on the crawler being able to retrieve it.

Meta-WebIndexer has a different purpose: improving Meta AI search results. Meta states that allowing this crawler helps it cite and link to content in its responses. This describes access and the platform's stated mechanism, rather than confirming that any particular page has been selected. In SEO and search visibility optimisation, technical accessibility and an actual appearance in an answer therefore need to be assessed separately.

Meta-ExternalAds is described for advertising-related activities and business products. Meta-ExternalAgent can collect content for training foundation AI models or improving products through indexing. Meta-ExternalFetcher retrieves individual links at a user's request and supports agent-related functions. These names are not interchangeable. A decision about one crawler should follow its documented purpose, rather than being automatically extended to every request associated with Meta's platforms.

For previews, the documentation requires Open Graph properties to appear before the first megabyte of the page and the content to be retrievable within a few seconds. This concerns access to metadata, rather than an article's editorial length. Separating metadata from the body helps identify the cause of an incomplete preview without assuming the news item or product must be removed from the website.

robots.txt expresses access preferences, with documented exceptions

In its consultation announcement, Meta says it uses the standard robots.txt mechanism. Its webmaster documentation also lists exceptions: Meta-ExternalFetcher may bypass the rules when retrieving a user-requested link, and FacebookExternalHit may do so for security or integrity checks. Stating that every Meta request follows every robots.txt rule in exactly the same way would omit those explicit conditions from the documentation.

The documentation also says robots.txt changes can take up to 24 hours to take effect because crawlers may cache the file. That period concerns distribution of technical instructions to crawlers. It is not the effective-date timetable for commercial terms, nor a deadline by which a page will appear, be cited or be removed from a particular platform product or response.

A robots.txt file does not replace authentication and access controls protecting confidential information. It supplies instructions for crawlers applying those rules; private content is protected through application and server mechanisms. A business's inventory of public pages and protected areas is therefore a separate matter from its preference about collecting publicly accessible content for a particular purpose, such as previews, indexing or model training.

Meta crawlers have different documented purposes

CrawlerDescribed main purposeRelevant condition
FacebookExternalHitPreviews of shared linksExceptions for security and integrity checks
Meta-WebIndexerMeta AI search-result qualityAccess supports content citation and linking
Meta-ExternalAgentTraining and product improvementCrawler-specific robots.txt access rules
Meta-ExternalFetcherUser-requested link retrievalMay bypass robots.txt in this context

What matters to businesses publishing on the web

For a business sharing articles, services or products on social networks, two decisions should be handled separately: how links are presented and how specialised crawlers access the content. A preview's title, description and image have a concrete role in sharing. Access rules concern retrieval of the page. Changing one of these components does not automatically establish the condition or effect of the other.

Within social media management, a record can bring together the shared public page, its preview and any observed access errors. Crawler analysis relies on server-recorded requests and their documented identifiers. This information supports discussion of a concrete issue, such as an outdated thumbnail or an access rule, without confusing that issue with the separate consultation on the proposed commercial terms and their future wording.

The November announcement keeps the proposals on hold and sets out a phased revision. For business websites, following official notices and understanding crawler roles supplies verifiable reference points. Technical configuration follows the published content, sharing objectives and the owner's access policy. The announcement does not, by itself, require blocking every Meta crawler or making a general change across the entire website and its existing pages.