⚑
LiteSpeed

Setting up the LiteSpeed crawler for better caching

πŸ“… 10 October 2026 ✏️ 10 October 2026 ⏱ 5 min leestijd

Learn how to set up the LiteSpeed crawler to automatically keep your LSCache warm for faster page loads for every visitor.

The LiteSpeed crawler is a built-in tool that automatically visits your pages and builds up the cache in advance, before a real visitor arrives. This way you never load an empty cache right when a cached page has just been refreshed. In this article you’ll learn what the crawler does, how to set it up in LiteSpeed Cache for WordPress or in Plesk, and which settings really make a difference.

What the LiteSpeed crawler does

If a page is not in the cache, the server has to build it completely on the first visit. That takes time and puts a load on your server, especially if this happens exactly when a cache expires. The crawler solves this by walking through all pages in your sitemap in the background, according to a schedule you set, and pre-filling the cache. As a result, visitors always see a ready-made, fast page.

The crawler works together with the LSCache module on the server and the LiteSpeed Cache plugin in WordPress. Without this combination, no crawler is available, so first check whether your hosting supports LiteSpeed with LSCache.

SitemapLiteSpeed CrawlerLSCache filled

Setting up the crawler in the LiteSpeed Cache plugin

In WordPress, go to LiteSpeed Cache > Crawler. Here you enable the function by setting the Crawler option to On. Next, you can set the number of concurrent threads; keep this low (for example 1 to 3) to avoid overloading the server during crawling. Also set an interval between crawl runs, for example every few hours, so the cache is refreshed regularly without running continuously.

At server level, the crawler cronjob must be configured. This usually happens automatically via the plugin, but if in doubt, check whether the cronjob is actually running. You can find cronjobs and server settings in your control panel; at Tandata you manage this easily via Plesk.

πŸ’‘ Tip: Let the crawler run outside busy hours, for example at night, so that full server capacity remains available for real visitors during the day.

Configuring sitemap and exclusions

By default, the crawler uses your XML sitemap to know which pages should be visited. Check that the correct sitemap address is entered in the crawler settings. If you have multiple sitemaps, for example per language or content type, you can add them all.

Not every page needs to be crawled. Exclusion lists are useful for page types that aren’t cached anyway, such as the shopping cart, the checkout page, or a login page. By excluding these, you save crawl capacity for the pages that really matter.

1

Check your sitemap

Make sure the sitemap is up to date and includes all important pages.

2

Activate the crawler

Turn on the crawler in the LiteSpeed Cache plugin and add the sitemap.

3

Set up exclusions

Exclude dynamic pages such as cart and account from crawling.

4

Choose a calm interval

Schedule the crawl runs outside peak hours and with limited threads.

5

Check the result

Review the cache status and loading times after a few crawl cycles.

Tuning crawler speed and server timing

The crawler can put significant strain on the server if you set too many threads or too short an interval, especially on a site with thousands of pages. Start conservatively: one thread and a generous interval. Then monitor server load and loading times. If everything runs stable, you can carefully increase the number of threads.

Also pay attention to the order in which pages are crawled. Important pages, such as the homepage and frequently visited product pages, should ideally be refreshed first. Some configurations let you set priorities, so these pages are always handled first.

If you don’t have LiteSpeed hosting yet and want to use this functionality, take a look at our web hosting options. If you’re unsure about switching from your current provider, read more about switching to Tandata.

Frequently asked questions

Does the crawler work without the LiteSpeed Cache plugin?

No, for WordPress you need the plugin to configure and activate the crawler. Without the plugin, you cannot control the LSCache functionality.

Can the crawler slow down my server?

This can happen with too many threads or too short an interval. Therefore, start with low settings and build this up gradually while monitoring server load.

Does the crawler also work for non-WordPress sites?

The LiteSpeed crawler function is specifically tied to the LiteSpeed Cache plugin for WordPress. Other platforms use different caching mechanisms.

Do I need to update the sitemap manually?

Usually the sitemap is automatically generated and updated by your SEO plugin, so the crawler always uses the current list of pages.

Where can I find more help with cache settings?

Our knowledge base contains more articles about caching and server settings.

Conclusion

The LiteSpeed crawler ensures your cache is always filled before a visitor arrives, resulting in consistently fast loading times. Set up the crawler carefully with a low thread setting, a clear sitemap, and the right exclusions, and build further from there.

View web hosting β†’

Was dit artikel nuttig?