Caching and purging
Content Delivery NetworksLink copied!
In order to provide customers with a fast and reliable delivery system, Amplience makes use of Content Delivery Networks (CDNs) to cache content so that it can be delivered to the users faster.
CDNs work by caching identical copies of data throughout a network of servers, known as "edge" servers, situated around the world at geographically significant locations. When a user requests a file, the request first goes to the CDN which will then route the request to the server geographically closest to them. For example, a request from a user in Germany might be routed to an edge server in Frankfurt.
This approach improves response times when delivering content, reduces the likelihood of errors occurring during transfer and helps to ensure that servers do not become overloaded during times of greatest demand. To provide another layer of resilience, the Amplience architecture also includes a mid-tier CDN to cache data and ensure that the web servers containing the original assets, the "origin servers", do not get sent unnecessary requests.
CachingLink copied!
To understand the way that caching works, let's examine the typical lifecycle of an asset:
The asset is uploaded to Content Hub and published. The asset is now available from the Dynamic Media web servers (referred to as the "origin server").
The asset is requested by the end user, by loading a webpage, for example.
The asset is requested from the CDN
The asset is requested from the mid-tier cache
The asset is requested from the origin server.
The asset is cached on the mid-tier cache
The asset is cached on the CDN
The asset is served to the user
The next time the asset is requested, then the asset it will already be in the CDN cache and will be served from the CDN cache to the user.
Time to Live (TTL)Link copied!
An asset will automatically be cleared from the CDN cache after a specified period known as the TTL (Time to Live). After the TTL has expired, when a request is made for the asset, it will no longer be in the cache and an updated version will be requested from the Amplience servers.
Dynamic assets TTLLink copied!
For assets consumed with a dynamic URL, the default TTL is set to 24 hours for the CDN servers and 30 minutes for a browser. So, if you update an asset consumed using a dynamic URL (that is, most images), it will take up to 24 hours for this change to become visible when you visit that URL. You can also purge a dynamic asset using the Dynamic Media app. This will clear it from the CDN cache and cause it to show up sooner than 24 hours (it's not possible to predict exactly how much sooner because CDNs may be all over the world, so the new version may show up in some places before others).
Static assets TTLLink copied!
For assets consumed with a static URL the TTL is one year (365 days). Therefore, if you update a static asset (for example: SVG, GIF or PDF), the change won't show up for 1 year. Static assets cannot be purged using Dynamic Media App, you will not be able use a purge credit to make changes show up sooner, as you can for dynamic assets.
If a Static Asset requires purging please reach out to Amplience Support who will be able to purge this asset for you.
PurgingLink copied!
In some circumstances you will want certain assets cached on the Content Delivery Network (CDN) edge servers to be refreshed quicker than the default Time to Live (TTL). For example, there may be:
- An error in an image
- An incorrect version of an asset posted
- Image quality issues
In these cases you can "purge" the assets from the CDN cache so that the next time the asset is requested an updated version will be retrieved. Purging will increase the response time the next time the asset is requested, since it will not be cached on the CDN and will instead be retrieved from the origin servers and then cached.
Assets consumed with a static URL cannot be purged.
The purge feature is available from the Dynamic Media App. If the app is not available on your account, please contact your Customer Success Manager.
You can purge selected assets one at a time, purge asset variants using the purge path function or purge all assets for an account.
CDNs place limits on the number of purge requests to be made and as such each account has a monthly allocation of purge credits. These limits are as follows:
1 account purge
120 asset purges
10 path purges
Each type of purge will use up one purge credit. You will be warned before starting a purge that it will reduce your credits.
Purging an assetLink copied!
Purging will remove the asset from the cache of all the CDN edge servers. The time required to purge an asset can vary according to a number of factors, including:
Assets may be cached across multiple CDNs
Assets may still be cached in a user's browser cache and the browser cache TTL can vary between users and browsers
The amount of network traffic across each CDN
When the asset has been purged, the next request for that asset will take longer, since it will first request the latest version from the mid-tier CDN. If it's not found in the mid-tier CDN cache, then the asset will be requested from the origin servers.The asset is then cached on the CDN edge servers so that subsequent requests are handled faster.
To purge an asset, select the asset from the list in the Dynamic Media app and choose "Purge" from the menu on the right. Assets are purged one at a time and each request will use one of your purge credits. The use cases for using purge include replacing an asset with a newer version and completely deleting an asset from the CDN.
Replacing an asset with a newer versionLink copied!
In this case, a new version of an asset is available (perhaps the wrong version has been published) and you want to ensure that this is served to the user as soon as possible. The newer version of the asset is uploaded to Content Hub, replacing the existing version. It is then published. You then select the asset from the list in the Dynamic Media app and choose the "Purge" item from the menu to the right of the asset. The asset will then be removed from the CDN cache and when the purge is complete the newer version of the asset will be served to the user.
This use case is shown in more detail below.
We upload an asset named "armchairpurgetest", an image of a red armchair, to Content Hub. We can either publish this asset from Content Hub or from within the Dynamic Media App. When an asset is published, it is made available to serve from the Dynamic Media server (the origin server). When you choose "Toggle Preview" the version uploaded to Content Hub and the version on the origin server will be the same, in this the preview of the asset is the red armchair image.
Let's say that instead of using an image of a red armchair, we wanted to use a green armchair instead. Rather than wait up for 24 hours for the updated image to be available we want to use the purge function to make it available faster. We upload the green armchair image to Content Hub and publish it. Then we launch the Dynamic Media App, select the "armchairpurgetest" asset and choose "Toggle Preview" and can see that the asset image is now the green armchair.
However, the red armchair image will be cached on the CDN edge servers until the TTL expires, which could be up to 24 hours. So if we request the asset in a browser, the red armchair image will be retrieved from the CDN.
To purge the asset from the CDN, select it in the list and choose "Purge".
Purging an asset uses one of your purge credits and a warning dialog will be displayed asking if you want to continue.
Purging will remove the asset from the cache of all the CDN edge servers, although the time that this takes to complete will depend on a number of factors, as explained above.
When the asset has been purged, the next request for that asset will request the updated version from the origin servers. So in this case, when the asset purge is complete, the green armchair will be displayed. You may still need to clear the browser cache in order for the new image to be loaded.
Removing an assetLink copied!
This refers to the case where you want to completely remove an asset from the CDN cache without waiting for the TTL to expire. Removing an asset is a two step process. The asset will still be available in Content Hub, it just won't be available to serve to the user.
Unpublish the asset, by selecting it in the list in the Dynamic Media app and choosing "unpublish". Unpublishing an item will remove it from the mid-tier CDN cache as well as the origin servers. The asset will not be automatically purged from CDN edge servers.
Purge the asset from the CDN edge cache, by selecting the asset in the list and choosing "Purge". When the purge is complete, any request for the asset will fail since unpublishing the asset has removed it from the mid-tier cache and the origin.
When an asset is purged, all the derivatives of that asset will be purged as well. This is explained in more detail below.
Simply deleting an asset from Content Hub will not unpublish the asset or remove it from the CDN.
SetsLink copied!
Adding ".jpg", ".png" or other image format to the URL of a set will return a preview image created from the first image or video that the set contains. This makes it easier to ensure that an image is always available to represent a product. For example:
will return a JPEG created from the first element of the set.
Because this preview image is created using the URL of the set, and not the URL of an individual asset within the set, the image will only be purged when the set itself is purged. So, if the individual elements of the set have been updated, you will need to purge both the updated assets and the set itself.
Variants of an AssetLink copied!
The Dynamic Media service allows you to manipulate images "on the fly", transforming an image with parameters including size and resolution, to create a version of an image appropriate for a particular device or to use as a thumbnail, for example. These derivatives of an image are created by adding parameters to the image URL. In the case of the armchair asset, we could create a version with a width of 300 pixels using the following URL:
The first time the armchair asset with these parameters is requested, a new variant of the asset will be cached on the CDN. The asset may be used in several places with different parameters and many variants of any given asset may be cached on the CDN.
A group of parameters can also be included in a transformation template.
Rather than adding the parameters directly to the URL:
These parameters qlt=15&w=600 might be included in a transformation template named "exampletemplate" and referred to as follows:
All of the variants of an asset, whether generated directly by adding parameters to the URL or by using a transformation template, will be purged from the CDN cache when the asset itself is purged.
Purging a PathLink copied!
The Dynamic Media app allows you to purge one or more individual URLs rather than purging all derivatives of an asset. For example, you may have made a change to a transformation template and don't want to purge all variants of an asset created using that template. You might also be using multiple transformation templates and only want to purge the derivatives of a particular asset created from one template.
To purge a path, choose the "Purge Path" item from the menu on the right hand side of the app window.
Enter the path to purge. The path must be specified using a partial rather than a full path. For example, to purge specified derivatives of the armchairpurgetest asset you would enter the following:
Purging a path will use one of your purge path credits. Paths can be added line by line, separated with a return character, with a maximum of 20 lines included in each request.
A detailed example of a path purge is shown below.
If we wanted to purge only the derivatives of the armchairpurgetest with the widths of 600 and 900 then we could enter
Alternatively we can choose the "Apply Suffixes" button and enter the parameters on separate lines in the "Add Suffixes" window.
Clicking the "Apply" button will append the parameters to the path we entered previously.
Click "Purge Path". The specified paths will now be purged.
Account purgingLink copied!
An account purge will remove from the CDN cache all the cached assets for this account, including images, sets and video. The cache will be completely cleared of all assets, so the next asset request will be slower, since the CDN will fetch each asset from the origin servers and then cache the assets on the mid tier CDN and CDN edge servers. An account purge should only be required in exceptional circumstances. However, it does guaranteed that the cache is completely cleared and the latest version of all assets will be retrieved when the user requests them.
To perform an account purge, launch the Dynamic Media app and choose "Purge Account" from the menu on the right hand side of the window.
By default you have one account purge credit per month and a whole account purge will use up this credit.
To continue and purge all the assets from the CDN cache, click the "Purge Whole Account" button.
Publish and unpublishLink copied!
The Dynamic Media app includes the functionality to publish and unpublish assets. Publish works in the same as it does in Content Hub, making an asset available to serve from the Dynamic Media servers and creating the necessary files to support the Dynamic Imaging functionality.
To publish an asset, select it from the list and choose "Publish" from the menu on the right hand side.
The asset list in the Dynamic Media app shows the publish status for each asset, together with the last published date.
Purge functionality is only available for assets that have been at some point been published, even if the asset has since been unpublished. So, if an asset only exists in Content Hub and has never been published, that asset cannot be purged because it has never been stored in the CDN cache.
UnpublishLink copied!
Unpublishing an asset will remove it from the origin servers and purge it from the mid-tier CDN cache. The asset will remain in Content Hub and is available to be published again. The asset will not be removed from the CDN edge servers until the TTL for that asset expires. To completely remove an asset you will need to unpublish it first and then purge the asset. Once the asset has been purged from the CDN edge servers, referencing the asset via its URL will fail.
To unpublish an asset, select it in the list and choose "unpublish".
It usually takes a few seconds for the unpublish operation to complete, at which point the status icon (1) is updated.
Unpublishing a set does not unpublish all the assets within it. Each asset must be unpublished individually.
Dynamic Media AppLink copied!
The Dynamic Media App provides a number of administration functions that allow you to publish, unpublish and purge assets, as well as access a log of asset related actions.
The Dynamic Media app is separate from the Dynamic Media API. The API allows you to manipulate images on the fly simply by changing its URL. For more information see the Media delivery API.
The app is launched from the apps pane in Content Hub, as highlighted in the image below. If the app is not available on your account, please contact your Customer Success Manager.
When you launch the app the Service Management screen is displayed. This lists all the assets in your account, shows information for each asset, including its publish status and allows you to access the service management functions. You can use the search box to find the assets you want.
To perform an action on the asset, select the option you want from the menu on the right. The account purge and path purge options can be accessed from the menu on the top right of the screen (1).
The service management functions are explained in the pages in this section, together with CDNs, caching and purging.
LoggingLink copied!
The log window displays a list of asset related actions, including the date each request was made and when it completed. For purge requests this represents the time the purge was successfully sent to the CDN, rather than the time is was purged from the CDN cache. The menu on the right hand side of the window is used to filter the log by the request type. An example log of publish asset requests is shown below.
Media cache settingsLink copied!
There are 3 types of cache in Amplience:
- Browser Cache
- Mid-tier CDN Cache
- Edge CDN Cache
Browser cacheLink copied!
Tells the browser how long to cache loaded files for. This is best for performance as the browser will not attempt to request the file from Amplience, but you do not have any business control over this cache.
Available settings levels:
- Account level
- Template level ( only available when removed from account level )
Minimum browser cache in seconds: 30
| Feature | Parameter | Description |
|---|---|---|
| Browser cache | h-ttl | The time a successful media request is cached in your browser for in seconds. |
| Browser error cache | h-e-ttl | The time a failed media request is cached in your browser for in seconds. |
While extended browser caching can have an increase in repeat view performance, it does also lose business control of the media. Amplience can clear the cache on our CDN architecture, but we have no access to the browser cache.
Template level browser cache settingsLink copied!
If you choose to remove your account level settings for browser cache, there are a few things to note:
- If no settings are provided in TT's, the system will default to 30 minutes.
- There is no maximum browser cache setting. However, please note the warning above.
- This abides by the normal Dynamic Media rules (the last instance of the same parameter in a URL request wins). For example, if you had 2 templates. $tt1$ and $tt2$, where $tt1$ had a browser cache of 1 hour and $tt2$ had a browser cache of 2 hours, the actual browser cache would depend on the order in the URL:
https://cdn.media.amplience.net/i/myaccount/myasset.jpg?$tt1$&$tt2$ - would give you 2 hours
https://cdn.media.amplience.net/i/myaccount/myasset.jpg?$tt2$&$tt1$ - would give you 1 hour.
Mid Tier CDN cacheLink copied!
This is not configurable. This tier is automatically purged when new content has been published.
Edge CDN CacheLink copied!
This setting tells our Edge CDN networks how long to cache files for before requesting the latest version on the Mid Tier CDN:
Available settings levels:
- Account level
Minimum Edge CDN cache in seconds: 86400 (24 hours).
| Feature | Parameter | Description |
|---|---|---|
| Edge CDN Cache | h-s-ttl | The time a successful media request is cached in on the Amplience Edge CDN. |
| Edge CDN error cache | h-e-s-ttl | The time a failed media request is cached in on the Amplience Edge CDN. |
For any requests for settings outside of these bounds, please contact your Customer Success Manager to discuss it.
Advanced Media Cache SettingsLink copied!
You can set browser and edge CDN TTLs for specific media types from Amplience. These will override any settings mentioned above.
Available settings levels:
- Account level
| Feature | Parameter | Description |
|---|---|---|
| Image Meta Data Edge CDN Cache | h-s-ttl-im | The meta data response when requesting images from the Amplience platform |
| Image Meta Data Edge CDN Error Cache | h-e-s-ttl-im | |
| Image Meta Data Edge Browser Cache | h-ttl-im | |
| Image Meta Data Edge Browser Error Cache | h-e-ttl-im | |
| Set Meta Data Edge CDN Cache | h-s-ttl-set | The meta data response when requesting sets from the Amplience platform |
| Set Meta Data Edge CDN Error Cache | h-e-s-ttl-set | |
| Set Meta Data Edge Browser Cache | h-ttl-set | |
| Set Meta Data Edge Browser Error Cache | h-e-ttl-set | |
| Template Meta Data Edge CDN cache | h-s-ttl-tpl | The meta data response when requesting template information from the Amplience platform |
| Template Meta Data Edge CDN Error Cache | h-e-s-ttl-tpl | |
| Template Meta Data Edge Browser Cache | h-ttl-tpl | |
| Template Meta Data Edge Browser Error Cache | h-e-ttl-tpl | |
| Video Meta Data Edge CDN Cache | h-s-ttl-vm | The meta data response when requesting videos from the Amplience platform |
| Video Meta Data Edge CDN Error Cache | h-e-s-ttl-vm | |
| Video Meta Data Edge Browser Cache | h-ttl-vm | |
| Video Meta Data Edge Browser Error Cache | h-e-ttl-vm |