WP Rocket Not Caching Your Paginated Pages? Let's Fix That

WP Rocket doesn't cache your paginated pages. Example, these won't be warmed up in your cache after preloading:

• /author/your-name/page/2/
• /author/your-name/page/3/
• /category/news/page/2/
• /category/news/page/3/
• etc

Is it a big deal? Probably not. But, is it a speed factor? Does Google care? Probably. Ideally, you want all of your pages to be cached and to load quickly. Plus, SEO audit tools like Ahrefs will flag these pages as "Slow" since they are the ones initially warming up the page cache for you when they come by and visit them for the first time, which is why they're so slow: they were never preloaded.

This script will fix this for you and enable you to preload all of your pages reliably.

First, create a folder in your WordPress root called /preloader/. Inside it, you will be creating 6 new files:

• /preloader/index.php
  (This will just be a blank file, which hides other files in this directory. Not needed if you have directory indexes turned off in Apache/NGINX)
• /preloader/cache-monitor.php
  (This checks your WP Rocket cache to see if it needs to be primed)
• /preloader/sitemap-warmer.php
  (This runs the actual preload based on your sitemap.xml file)
• /preloader/sitemap.xml
  (The sitemap that contains your pages)
• /preloader/timestamp.txt
  (This is when the preloader was last ran, automatically generated)
• /preloader/wprocket-preload.php
  (This clears the WP Rocket cache and preloads it again, adding your new custom preloaded pages to this process)

Let's start with the first file, index.php. Just open your text or code editor and save a blank file called index.php. Easy!

Before we create the other five, you should know your complete working directory. If you know this by heart already, skip this step. If not, upload this temporary script to your WordPress site, which will tell you your full working directory. You can just save it as test.php and delete it after:

<?php
echo getcwd();
?>

Then, visit yourdomain.com/test.php. Take note of this directory. It will probably look something like this:

• /home/servername/public_html/domain.com/

Save this directory for later. You can delete test.php now.

Next, we're going to create /preloader/cache-monitor.php. Remember to replace the working directory /home/servername/public_html/yoursitename/ with your real working directory that you found earlier.

Also, you should choose a stable pagination page that is normally not pre-loaded by WP Rocket. In my example below, I chose an author page. So for my site, it might look like: /wp-content/cache/wp-rocket/www.contentpowered.com/author/james-parsons/page/*. This way, if these files don't exist when the script runs, it will trigger the script to generate all missing cache files automatically.

Modify your working directory accordingly and save this as /preloader/cache-monitor.php:

<?php

$total_items  = count( glob("/home/servername/public_html/yoursitename/wp-content/cache/wp-rocket/www.yourdomain.com/author/your-name/page/*", GLOB_ONLYDIR) );

if ($total_items < "1") {
	require( 'wprocket-preload.php' );
}
else {
	echo "WP Rocket Cache is already preloaded";
	echo "<br><br>";
}

?>

If a file exists in this directory, it means that your script worked and your cache is already primed, so it won't run this whole process again. If the pagination cache is empty here, it means the others are empty too, and the script will know to prime the cache again automatically.

Next is wprocket-preload.php. Same as before, remember to replace the two working directories with the one you found earlier:

<?php

// If preloader has already ran less than 10 minutes ago,
$now = time();
if ($now - filemtime("/home/servername/public_html/yourdomain/preloader/timestamp.txt") > 60*10) {

  // Reset timestamp
  $input = date('M-d-Y h:i:s');
  $fp = fopen("timestamp.txt", "r+");
  fwrite($fp, $input).' ';
  fclose($fp);

  // Load WordPress.
	require( '../wp-load.php' );

	// Clear cache.
	if ( function_exists( 'rocket_clean_domain' ) ) {
		rocket_clean_domain();
	}

	// Preload cache with WP Rocket preloader
	if ( function_exists( 'run_rocket_sitemap_preload' ) ) {
		run_rocket_sitemap_preload();
	}

	// Preload pagination with Sitemap Warmer script
	$bgproc = popen('php "/home/servername/public_html/yourdomain/preloader/sitemap-warmer.php"', 'w');
	if($bgproc===false){
	  die('Could not open bgrnd process');
	}else{
	  pclose($bgproc);
	}

}
else {  // script hasn't ran yet in the past hour
	echo "Already preloaded";
}

Next is sitemap-warmer.php. This file is based on the work by Pixel Envision (shoutout to E. Gonenc!). Same as before, remember to replace the working directory. You should also name the user agent down below. If you use Cloudflare or any sort of firewall, this is useful as you can whitelist your custom user agent so your firewall doesn't block your preloader:

<?php
// CACHE WARMER (PRELOADER) by Pixel Envision (E.Gonenc)
// Configuration options

$priority = false; //Use priorities defined in sitemap.xml (true/false)
$ppi = 1; //Pages to be cached per interval
$delay = 2; // Delay in seconds between page checks, default is half a second
$quiet = true; // Do not output process log (true/false)
$trailing_slash = false; // Add trailing slash to URL's, that might fix cache creation problems (true/false)
$sitemap = "sitemap.xml"; //Path to sitemap file relative to the warm.php
$index = "index-https.html"; //Cache file to check
$rootp = "/home/servername/public_html/domain.com/wp-content/cache/wp-rocket/www.domain.com"; //Root of cache

//Do not change anything below this line unless you know what you are doing

ignore_user_abort();
set_time_limit(600);
$xml = simplexml_load_file($sitemap);
$UL = $UP = array();
foreach ($xml->url as $url_list)
{
    $UL[] = $url_list->loc;
    $UP[] = $url_list->priority;
}
unset($xml);
if ($priority == true)
{
    arsort($UP, $sort_flags = SORT_NUMERIC);
}
$i = 0;
foreach ($UP as $key => $val)
{

    $path = $rootp;
    $url = $UL[$key];
    $sub = explode("/", $url);
    if ($sub[3])
    {
        $path .= "/" . urldecode($sub[3]);
    }
    if ($sub[4])
    {
        $path .= "/" . urldecode($sub[4]);
    }
    if ($sub[5])
    {
        $path .= "/" . urldecode($sub[5]);
    }
    if ($sub[6])
    {
        $path .= "/" . urldecode($sub[6]);
    }
    if ($sub[7])
    {
        $path .= "/" . urldecode($sub[7]);
    }
    if ($sub[8])
    {
        $path .= "/" . urldecode($sub[8]);
    }
    $path .= "/" . $index;

    if (file_exists($path))
    {
        if ($quiet != true)
        {
            echo "Priority: " . $val . " => Skipped: " . $path . "\n";
        }
    }
    else
    {
        if ($trailing_slash == true)
        {
            $url = rtrim($url, "/") . "/";
        }
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
        curl_setopt($ch, CURLOPT_HEADER, true);
        curl_setopt($ch, CURLOPT_USERAGENT, 'Domain.com Sitemap Warmer');
		curl_setopt($ch, CURLOPT_NOBODY, false);
		curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $ret = curl_exec($ch);
        curl_close($ch);
        if ($ret)
        {
            $i++;
        }
        else
        {
            echo "Unable to connect $url, exiting...";
            break;
        }
        usleep($delay * 1000000);
        if ($quiet != true)
        {
            echo "Priority: " . $val . " => Warmed: " . $path . " by visiting " . $url . "\n";
        }
    }
    if ($i < $ppi)
    {
        flush();
    }
    else
    {
        flush();
    }

}
exit;
?>

Next is a sitemap (sitemap.xml). Here, you'll want to add all of your pages that you want to preload.

Note: You can add more pages than you actually have, in anticipation of those pages eventually existing. For example, if you only have 10 pages right now on your category or author pagination, you can add 15 or 20 to future-proof this a bit. Those pages might not exist yet, but they will eventually, and then you won't have to update this file as much. Just copy the URL lines and add your new URLs inside of these tags.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd http://www.google.com/schemas/sitemap-image/1.1 http://www.google.com/schemas/sitemap-image/1.1/sitemap-image.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>http://www.domain.com/</loc></url>
<url><loc>https://www.domain.com/blog/page/2/</loc></url>
<url><loc>https://www.domain.com/blog/page/3/</loc></url>
<url><loc>https://www.domain.com/blog/category/blogging/</loc></url>
<url><loc>https://www.domain.com/blog/category/blogging/page/2/</loc></url>
<url><loc>https://www.domain.com/blog/category/blogging/page/3/</loc></url>
<url><loc>https://www.domain.com/blog/author/your-name/</loc></url>
<url><loc>https://www.domain.com/blog/author/your-name/page/2/</loc></url>
</urlset>

Last is timestamp.txt. Just create the file and make sure it's writeable. You can add this in there if you'd like as a placeholder, to make sure it's overwriting the date properly when it checks your sitemap:

Feb-21-2024 06:26:02

To recap, you should now have a /preloader/ folder in your WordPress with six files inside of it.

Next, let's test to make sure this works. You'll need FTP or a file manager to do this. Here's how:

• Delete all files and folders in your WP Rocket cache:
  /home/yourserver/public_html/yoursite.com/wp-content/cache/wp-rocket/www.yoursite.com/
• Manually run your preloading script at:
  https://yoursite.com/preloader/cache-monitor.php
• As this is running, monitor your cache directory, refreshing it occasionally: /home/yourserver/public_html/yoursite.com/wp-content/cache/wp-rocket/www.yoursite.com/ - You should see folders populating as the preload process runs. Most importantly, though, you want to look for the URLs that you manually added to sitemap.xml. You may have to drill down into the folders. For example, if I want to make sure my /blog/page/2/ page is being preloaded, I have to look inside of this directory: /contentpowered/wp-content/cache/wp-rocket/www.contentpowered.com/blog/page/2/. If you see files in there, it worked - your pages are being cached!

Now that you've confirmed that all of your pages are being preloaded successfully, you're ready to set this up so it loads automatically without you having to think about it. How do you do that?

Cron jobs. You want to create a cron job to load this file every hour (replace yourdomain.com with your website URL):

• https://www.yourdomain.com/preloader/cache-monitor.php

There are two ways to do this:

• Use the built-in cron on your hosting, if it has one. All cPanel hosts have cron built in, so if you have cPanel, just log into cPanel and visit the cron job section. You want to load the full URL. You can run this as often as you'd like. I run mine every hour.
• Use Easycron.com. Easycron is easy to set up and works even if you don't have an accessible cron system with your web host. It just triggers a visit to your script within your specified time. The only downside here is that very large sitemaps might take a long time to load, and Easycron.com won't stick around to wait for it to finish unless you have a paid plan.

Once your cron is set up and you've verified it's working, you're done! Now 100% of the pages on your site are being pre-loaded, not just the pages in your sitemap. Your site is a little bit faster. Congrats!

Did this help you? Any questions for me? Please leave me a comment below!

Related Code Entries

• Automatically Write Post Summaries in WordPress with ChatGPT
• WP Rocket Not Caching Your Paginated Pages? Let's Fix That
• Replace Smart/Curly Quotes in WordPress With Regular Ones
• Fix Multicollab Bug Where You Can't Click and Add a Comment
• How to Prevent a WP Website Lockout With "Critical Errors"