ByTheWeb GEO can automatically provide two Markdown resources at the root of your WordPress website:
https://example.com/llms.txthttps://example.com/llms-full.txt
These files provide a structured, text-focused representation of selected public and indexable website content.
llms.txt is designed as the more concise file, while llms-full.txt can provide a fuller version of the selected content.
For a product overview, see:
llms.txt & llms-full.txt Feature Overview
Both files are generated locally by ByTheWeb GEO and do not require a ByTheWeb AI API Key or AI credits.
What Is the Difference Between llms.txt and llms-full.txt?
| llms.txt | llms-full.txt | |
|---|---|---|
| Purpose | Concise site summary and content index | Fuller representation of selected content |
| URL | /llms.txt | /llms-full.txt |
| Default status | Enabled | Disabled |
| Default maximum size | 50,000 characters | 200,000 characters |
| Short Answer | Supported | Supported |
| AI Summary | Supported | Supported |
| Content links | Supported | Supported |
| Taxonomy descriptions | Supported | Supported |
| Fallback excerpts | Supported | – |
| Cleaned full page/post content | – | Supported |
| Content type and last-modified information | – | Supported |
The same Post Type and Taxonomy selections are used to determine which content is eligible for both files.
Where Do I Configure the LLM Files?
In your WordPress Dashboard, go to:
ByTheWeb > GEO Settings > llms.txt Settings
This screen controls both llms.txt and llms-full.txt.
Enable llms.txt
The Enable llms.txt Generation option controls the main llms.txt resource.
It is enabled by default.
When enabled, ByTheWeb GEO serves the file at:
After saving the settings, the LLM settings screen displays the active URL together with Copy URL and Open options.
Enable llms-full.txt
The fuller file is optional and is disabled by default.
To use it:
- Go to ByTheWeb > GEO Settings > llms.txt Settings.
- Enable Enable llms-full.txt Generation.
- Save the settings.
The file will then be available at:
The settings screen also provides Copy URL and Open options for the active file.
Choose Which Content Is Eligible
Under the LLM settings, you can select the public Post Types and Taxonomies that ByTheWeb GEO should consider for the files.
The default selections are:
Post Types
- Posts
- Pages
Taxonomies
- Categories
- Tags
Additional public Custom Post Types and public Taxonomies registered on the website can also be selected.
Selecting a Post Type or Taxonomy makes that content eligible for the LLM files.
It does not mean that every item belonging to it will automatically appear.
ByTheWeb GEO also applies its indexability and content eligibility checks before including individual content.
Which Content Is Excluded?
ByTheWeb GEO is designed to build the LLM files from public, indexable content.
For posts and other supported content, the plugin excludes content that does not meet its eligibility rules, including:
- Content that is not published
- Password-protected content
- Content that is considered
noindexby the active SEO provider - Content affected by the global WordPress Search Engine Visibility setting
This prevents simply selecting a Post Type from automatically exposing every item stored under that Post Type.
Exclude Individual Content from LLM Files
ByTheWeb GEO also lets you exclude individual content from the LLM files without changing its normal search-engine indexing settings.
On supported Posts, Pages, and other enabled content, you can use:
Exclude from LLM files
When enabled, that individual item is excluded from both:
llms.txtllms-full.txt
The content can remain published and indexable, and it can remain in your XML Sitemap.
Exclude an Individual Category, Tag, or Taxonomy Term
The same LLM exclusion is available for supported Taxonomy terms.
When editing a Category, Tag, or other supported Taxonomy term, you can enable:
Exclude from LLM files
That term will then be excluded from both LLM files without changing its normal Index/Noindex setting or Sitemap eligibility.
This is useful when a page or taxonomy should remain available to search engines but you do not want ByTheWeb GEO to include it in the generated LLM files.
LLM Exclusion vs. Noindex
These settings serve different purposes.
Exclude from LLM files
Removes the individual item from llms.txt and llms-full.txt only.
Noindex
Controls the item’s search-engine indexing directive and also affects its eligibility for other ByTheWeb outputs that require indexable content.
Use Exclude from LLM files when you want the content to remain normally indexable but do not want it represented in the ByTheWeb LLM files.
How Indexability Works with Yoast SEO and Rank Math
The LLM feature remains available when Yoast SEO or Rank Math SEO is active.
However, ByTheWeb GEO respects the active SEO provider when deciding whether individual content should be treated as indexable.
If Yoast SEO is the active SEO provider, its supported indexability settings are respected.
If Rank Math SEO is the active SEO provider, its supported indexability settings are respected.
If neither supported external SEO provider is active, ByTheWeb GEO uses its own supported indexing settings.
This allows the LLM files to follow the same provider-aware approach used elsewhere in ByTheWeb GEO.
For the complete provider behavior, see:
WordPress Search Engine Visibility
WordPress includes a global setting under:
Settings > Reading > Search Engine Visibility
If Discourage search engines from indexing this site is enabled, ByTheWeb GEO respects that global setting when determining indexability for the LLM files.
The global WordPress setting overrides individual page or taxonomy indexing decisions for this purpose.
Add a Custom AI Introduction
The Custom AI Introduction field lets you add your own text near the beginning of both llms.txt and llms-full.txt.
For example, you may use it to provide a short factual description of the website, organization, or purpose of the content.
This field is optional.
The Custom AI Introduction does not replace your WordPress Site Description.
It adds separate context to the LLM files.
The same introduction is used in both files.
Set the Maximum File Size
ByTheWeb GEO provides a separate character limit for each file.
The default and recommended values are:
- llms.txt: 50,000 characters
- llms-full.txt: 200,000 characters
The settings interface allows values from 10,000 to 500,000 characters for each file.
These limits help prevent very large websites from producing oversized LLM responses.
When the configured maximum is reached, the generated output is limited rather than continuing indefinitely through all eligible content.
For most websites, the default values are a good starting point.
What Does llms.txt Include?
The concise llms.txt file can compile selected public and indexable content into Markdown using information such as:
- Content titles and links
- Short Answers
- AI Summaries
- Fallback excerpts where applicable
- Taxonomy descriptions
- Website and content structure information used by the generator
The exact content available for an individual entry depends on the data stored for that post, page, or taxonomy.
Adding strong AI Summary and Short Answer content therefore gives ByTheWeb GEO additional structured information it can reuse in the LLM files.
What Does llms-full.txt Include?
llms-full.txt can provide a more complete representation of eligible website content.
For supported posts and pages it can include:
- Title
- URL
- Content type
- Last-modified information
- Short Answer
- AI Summary
- Cleaned page or post content
ByTheWeb GEO cleans stored content before including it in the full file.
The cleaning process removes markup and elements that are not useful as text output.
For content created with Elementor, ByTheWeb GEO can also extract supported textual content from stored Elementor data and combine it with the usable WordPress content.
This Elementor extraction is specific to supported textual data and should not be interpreted as universal support for every field or every third-party page builder.
Do I Need to Create Physical Files?
No.
ByTheWeb GEO serves /llms.txt and /llms-full.txt dynamically through WordPress when the corresponding feature is enabled.
You do not need to manually create, upload, or maintain physical .txt files in the root directory of the website.
Changes to eligible content or LLM settings are therefore reflected through the content generated by ByTheWeb GEO.
How Do I Check the Files?
After saving your LLM settings, use the Open button shown for each active file.
You can also open the URLs directly in a browser:
and, if enabled:
Review the output to confirm that the expected public content, links, summaries, and descriptions are represented.
If content you expected to see is missing, first confirm that:
- Its Post Type or Taxonomy is selected.
- The individual content is public and indexable.
- The content is published where applicable.
- It is not password protected.
- The global WordPress Search Engine Visibility setting is not preventing indexing.
What If Another Plugin Also Generates llms.txt?
ByTheWeb GEO can detect supported situations where another SEO plugin also has an llms.txt generator active.
Because both generators may attempt to use the same /llms.txt URL, the ByTheWeb GEO settings can display an:
Another llms.txt generator is active
warning.
If you want to use the full ByTheWeb LLM file functionality, including ByTheWeb content selection, Short Answers, AI Summaries, and llms-full.txt, disable the overlapping llms.txt generation feature in the other plugin.
You do not need to disable Yoast SEO or Rank Math SEO itself.
Their supported SEO and indexability settings can remain active while ByTheWeb GEO provides the LLM files.
Do llms.txt and llms-full.txt Use AI Credits?
No.
Generating and serving the LLM files is a local ByTheWeb GEO feature and does not consume ByTheWeb AI credits.
You also do not need to connect a ByTheWeb AI API Key just to use these files.
Credits are only relevant if you choose to use connected ByTheWeb AI features to generate or improve content such as:
- AI Summary
- Short Answer
- FAQ
- Other supported AI-generated fields
For current credit rules, see:
Are Short Answer and AI Summary Used in the LLM Files?
Yes.
When eligible content contains a saved Short Answer or AI Summary, ByTheWeb GEO can include those values in its LLM output.
In llms.txt, these fields help provide a concise representation of the content.
In llms-full.txt, they can appear as dedicated information alongside the fuller cleaned page content.
For more information about these fields, see:
Are Taxonomies Included in the LLM Files?
They can be.
ByTheWeb GEO supports selected public Taxonomies such as:
- Categories
- Tags
- Supported Custom Taxonomies
A Taxonomy must be selected in the LLM settings before its eligible terms can be considered for inclusion.
The individual term must also meet the applicable indexability requirements.
When available, ByTheWeb GEO can use information such as:
- Term name
- Term URL
- Taxonomy description
- Short Answer
- AI Summary
according to the relevant file output.
Does Selecting a Taxonomy Include Every Term?
No.
Selecting the Taxonomy makes its terms eligible for consideration.
Individual terms can still be excluded when they do not satisfy the applicable indexability and content rules.
This is similar to selecting a Post Type: the selection enables the content type, while the individual content still has to pass the applicable checks.
Can Custom Post Types Be Included?
Yes, supported public Custom Post Types registered by WordPress can be selected in the LLM settings.
They are not automatically enabled simply because they exist.
Select the Custom Post Types you want ByTheWeb GEO to consider.
Their individual published content must still meet the applicable indexability requirements.
Can Custom Taxonomies Be Included?
Yes.
Supported public Custom Taxonomies can be selected through the same LLM settings.
As with Categories and Tags, their individual terms are considered only when the Taxonomy is enabled and the term satisfies the applicable eligibility rules.
Does llms.txt Guarantee That AI Engines Will Use My Content?
No.
ByTheWeb GEO makes the files available on your website.
It does not control whether a third-party AI engine, crawler, agent, search engine, or other service:
- Requests the file
- Reads the file
- Uses the content
- Indexes the content
- Cites the website
- Includes the website in an AI-generated answer
The files provide a structured resource that external systems may choose to use.
They do not guarantee AI visibility, citations, rankings, or traffic.
Is llms.txt a Replacement for robots.txt?
No.
The files serve different purposes.
llms.txt and llms-full.txt provide structured information about selected website content.
robots.txt provides crawler directives and policies.
ByTheWeb GEO provides separate controls for applicable AI crawler directives when it is responsible for the website’s SEO layer.
For more information, see:
Is llms.txt a Replacement for an XML Sitemap?
No.
An XML Sitemap and an LLM file are separate resources.
The XML Sitemap provides structured URL discovery information for supported search-engine workflows.
The LLM files provide Markdown-oriented content information for selected website content.
A website can use both resources at the same time.
For ByTheWeb GEO Sitemap configuration, see:
What Happens If I Disable llms-full.txt?
If you disable:
Enable llms-full.txt Generation
ByTheWeb GEO stops providing its dynamic llms-full.txt resource.
This does not disable the main llms.txt file if that file remains enabled.
It also does not delete your WordPress content, AI Summaries, Short Answers, or other stored ByTheWeb GEO data.
What Happens If I Disable llms.txt?
If you disable:
Enable llms.txt Generation
ByTheWeb GEO stops providing its dynamic llms.txt resource.
The setting is separate from the llms-full.txt option.
Disabling the LLM resource does not delete the underlying WordPress content or ByTheWeb GEO fields used to build it.
What Happens If ByTheWeb GEO Is Deactivated?
The LLM URLs are dynamically provided by ByTheWeb GEO.
If the plugin is deactivated, ByTheWeb GEO no longer provides its dynamic llms.txt and llms-full.txt output.
The WordPress content and ByTheWeb data already stored in your website database are not automatically deleted simply because the plugin is deactivated.
If ByTheWeb GEO is activated again, its saved configuration and content can be used again as long as that data is still present in WordPress.
Why Is Content Missing from llms.txt or llms-full.txt?
Check the following:
- Confirm that the relevant LLM file is enabled.
- Confirm that the content’s Post Type or Taxonomy is selected.
- Confirm that the content is published where applicable.
- Confirm that the content is public.
- Confirm that it is not password protected.
- Confirm that it is not
noindexaccording to the active SEO provider. - Check Settings > Reading > Search Engine Visibility.
- Confirm that the configured maximum file size has not limited the generated output.
If another plugin also generates llms.txt, review the conflict notice under the ByTheWeb GEO LLM settings.
For additional checks, see:
Recommended Setup
A practical first configuration is:
- Open ByTheWeb > GEO Settings > llms.txt Settings.
- Keep llms.txt enabled.
- Decide whether you also want the larger llms-full.txt file.
- Select the public Post Types you want included.
- Select the public Taxonomies you want included.
- Add a short factual Custom AI Introduction if useful.
- Keep the default character limits unless your website requires different limits.
- Save the settings.
- Open
llms.txtand review the generated output. - If enabled, open
llms-full.txtand review the fuller output. - Check that important content is published and indexable.
- Review the files again when you significantly change your website structure or LLM settings.
Summary
ByTheWeb GEO provides two optional LLM-oriented website resources.
llms.txt
- Enabled by default
- Available at
/llms.txt - Designed as the more concise resource
- Default maximum size: 50,000 characters
- Can include selected public and indexable Posts, Pages, Custom Post Types, and Taxonomies
- Can use Short Answer, AI Summary, descriptions, excerpts, URLs, and related content information
- Does not use AI credits
llms-full.txt
- Disabled by default
- Available at
/llms-full.txt - Provides a fuller representation of eligible content
- Default maximum size: 200,000 characters
- Can include cleaned page or post content
- Can include Short Answer and AI Summary
- Can include content type and last-modified information
- Does not use AI credits
Both files respect the applicable content selections and provider-aware indexability rules.
Where Should I Go Next?
For the product overview:
llms.txt & llms-full.txt Feature Overview
For Short Answer and AI Summary:
For crawler directives:
For XML Sitemap configuration:
For SEO provider compatibility:
For LLM-file problems: