Peakhour.IO - Caching

Origin shield

2022-06-10T13:00:00+10:00

CDN providers often promote the size of their network, and how many Points of Presence (POPs) they have. Higher capacity, more resilient networks are useful from a security point of view (think DDoS attacks), but more POPs can also work against what the CDN was designed to do: take load off an origin and improve performance for end users.

The POP Problem

Modern CDNs are what's called 'Pull' CDNs. That means the CDN won't store content/resources until a user requests it. The first time a user requests a resource, it goes to the CDN POP, checks its local cache, gets a miss, and then passes the request through to origin. As the resource is returned, the CDN stores a copy for the next time someone wants it. If your CDN has 100 POPs, then this process has to be repeated 100 times to fully 'warm' the CDN for that specific resource. That's 100 requests to origin. The more POPs your CDN has, the more likely you are to get a miss and hit the origin.

When the caches at POPs are fully populated, the effect on your application can be minimal. During a cache MISS event, typically either due to resource expiration or a manual purge, many requests can be sent to the origin server concurrently while the individual POPs rebuild their caches. The more POPs, the longer the process takes.

This can be a problem, especially when caching dynamic pages that need to be server side rendered, large resources, or transformed resources. For example, take a busy ecommerce store running Magento during a sale, Magento will purge content when sales are made, forcing the cache to rebuild each time. During a busy period it can reduce your cache hit rate and degrade site performance.

Enter Origin Shield

CDN Origin Shield is a feature that lets you nominate the CDN Point of Presence closest to your server as a shield. All requests that hit other POPs and receive a cache miss will then go to the nominated shield before hitting the origin. The shield becomes a 'super cache' and can reduce the amount of requests to your origin in a cache miss.

Peakhour.IO implements origin shield as a simple dropdown on an origin pool where you can select the geographic location that should be used as a shield. Requests to your origin are now routed through this geographic location before reaching your origin in a cache miss scenario.

Clients who use multiple geographic origins can also benefit from Origin shield. Peakhour.IO allows the specification of an origin shield per origin. For geographic load balancing, you will need to contact support for setup.

Seeing is believing

The Peakhour.IO summary now includes your edge CHR, your shield CHR and your overall CHR so that you can see the effect in action.

Some of our clients have seen typical increases of 10-20% of their overall Cache Hit Rate, and greater than 40% when specifically looking at often flushed dynamic content.
Quicker cache convergence
Fewer hits to origin
Better end-user experience
Higher conversions

Final Thoughts

Origin Shield is an important feature for certain types of site, or when you're looking to maximise your cache hit rate. CMSs that offer built in full page caching, like Magento and Drupal, flush content often, and are susceptible to performance degradation as load increases. Minimising hits to the origin in these cases is vital.

If you are interested in getting more out of your CDN, need a bespoke CDN solution, or need a provider that offers performance, optimisation and security services, reach out to discuss the right setup.

Origin shield with request collapsing helps minimise origin hits, improve CHR and maintain user experience for your web application.

CDN Cache Keys

2022-05-16T13:00:00+10:00

Content Delivery Networks (CDNs) are common on business websites. CDNs speed up websites by storing website content and assets in caches on servers spread around the world. These caches are typically key/value stores. For example, let's say we are requesting an image found at:

https://www.example.com/someimage.jpg

Depending on how the CDN works internally, it may use /someimage.jpg (the key) to retrieve the image (the value) from its cache. Most CDNs are fairly rigid in what they use as keys, complicated to configure (eg requiring programming skills), or charge more for flexibility.

The Peakhour.io CDN cache keeps configuration flexible while still supporting fine-grained control of caching behaviour for your application.

The Peakhour.IO cache key consists of a primary key, sub key, and secondary key. Both the primary and secondary key must match to constitute a cache hit.

Primary key

The primary key is the key from the client request that uniquely identifies a resource. It consists of the scheme, host, path and query string from the request. Within our cache, the primary key may be augmented from other elements of the request, such as the presence of a particular header, or a header value such as a cookie value. At Peakhour.IO, we call these augmented keys cache subkey vars.

The following options can further manipulate the primary key by specifying how the query string is handled. These options include ignore query string (useful for defeating cache-busting techniques) or stripping certain tags (commonly UTM tags).

Secondary key

The secondary key describes parts of the request message that influenced the content of the response from the origin. This information is stored in the secondary key and the following headers are used:

Vary
Content-Encoding
User-Agent: (browser/mobile)
Accept-Language
Variant-06

Servers are commonly misconfigured and can send back headers that don't match the behaviour of the content. For example, it is very common for a server to send back a Vary header with Vary: user-agent. When you check the behaviour by sending different user agents, though, the content doesn't actually change. This can cause unnecessary cache fragmentation and lower hit rates. Peakhour allows you to override/ignore origin behaviour to correct this.

If you prefer a visual view, the diagram below shows all the elements of a request/response that could make up a cache key.

For more flexibility, Peakhour.io can use specific Cookie values in the secondary key. This is useful for Content Management Systems like Magento, which set a special cookie, X-Magento-Vary, to enable special caching behaviour that wouldn't normally be achievable; for example, whether the user is logged in, is part of a special group, or has chosen a different currency.

How can I see my keys?

Cache keys are generated on the fly based on the client's request and the origin's response. To see a cache key, enable Debug headers in the Peakhour.IO dashboard. As an example, for the given request/response pair:

GET / HTTP/1.1
Accept-Encoding: gzip, deflate, br

HTTP/1.1 200
Vary: accept-encoding
content-encoding: br

Peakhour.IO would generate the following cache-status header:

cache-status: peakhour.io; fwd=uri-miss; key="https://example.com/what-is-new.html"; secondary-key="encoding::br"; stored; ttl=31536000

Here, the key is the scheme://host/path and the secondary-key is the encoding served by the origin.

What's in it for me?

By understanding cache keys and how they are constructed, a web application can tailor its responses to better utilise a cache. The configuration options give you direct control for fine-tuning cache-key handling, improving user experience and cache hit rates.

Request collapsing

2022-05-16T11:00:00+10:00

Request collapsing protects busy origin servers that serve changing content. On a high traffic site, a cache miss after content expires can amplify one expired resource into many simultaneous origin requests. This behaviour is commonly called a cache stampede or dog-piling.

When request collapsing is enabled, only a single request is sent to an origin server for a given resource, then the resulting body is used to satisfy pending requests.

This can stop a popular cached resource from causing a flood of requests to an origin server when it expires. On high traffic sites, enabling the feature for the right resources can smooth request volume and traffic to origin servers.

How it works

Request collapsing is implemented internally using cache keys and queues. Cache keys are used as keys to a map, with client requests tracked using a queue. Secondary response keys are then used to match waiting client requests to origin responses. Matching requests can then be fulfilled using the same response.

Failed requests are retried in the same manner.

If a response to an active request is marked as private with Cache-Control: private or Set-Cookie, then all queued and future matching requests will go directly to the origin without trying to match them.

This implementation allows Peakhour to serialise requests to an origin, so it's important to enable the feature only on resources that you know will be cacheable, either through cache-control headers or Peakhour configuration.

How do you know its working?

Request collapsing can be verified with Debug enabled and the cache-status header in the response. A collapsed request looks like the example below, with 'collapsed' at the end.

cache-status: peakhour.io; fwd=uri-miss; key="https://website.com/home-eco.jpg"; ttl=86400; collapsed

How its used in practice?

Image optimisation

Peakhour.IO uses request collapsing internally for image transforms. The feature ensures we only transform a single image when a resource expires on the edge, reducing latency sensitive image transformation work, client transform costs and avoidable delays for end users.

Dynamic page caching

Request collapsing is a good fit when enabling caching of expensive dynamically written pages. Peakhour.IO can cache server heavy WordPress, Magento, PrestaShop, Drupal and other platforms via our caching plugin. The plugin keeps the CDN in sync by notifying us when content changes.

Content still needs to change during busy periods such as sales or item purchases. During these periods servers can be overwhelmed when pages incur a cache miss. Request collapsing can help smooth out this traffic and maintain response times for users.