If your startup is relatively new and doesn’t take much network traffic, there’s probably no reason to be interested in caching. The problem arises when users begin to complain about the time to load several pages. Well, users hate waiting. If you care about their convenience or you are curious about caching this article is especially for you.

General information

Caching is the way to reduce latency and network traffic. Web servers and browsers are saving copies of received (or generated) files. These copies are sended as client requests them once more. There are several kinds of web caches:

1. Browser cache - is controlled by your client’s browser. It checks if representations (e.g. images, scripts, HTML files) are fresh, once a session. Thanks to browser cache when we click back button, page loads instantly. As a webmaster you may control this cache by sending different headers with content. I’ll list and explain them later.
2. Proxy caches - work same way as browser cache, but in larger scale. There could be large number of users using one proxy cache.  They are usually maintained by network administrators to save bandwidth.
3. Gateway caches - also known as “reverse proxy caches” or “surrogate caches”. They are used by large sites and maintained by your administrators or Content Delivery Networks (CDN) which are distributing gateway caches all over the Internet (content is served from the nearest gateway cache).
4. Application cache - runs on origin server and prevents regenerating of content. You may choose to cache your database queries, rendered pages, statistics and so on. They may be used even if part of requested page is changed. Their importance can’t be overestimated.

As long as their behavior is considered. Browser, proxy and gateway caches are much the same. They are controlled by file headers and store entire files. On the other hand application cache is controlled directly from application and may store single queries or parts of document.

How web caches works

Let me quote part of mnot tutorial.  Note that pages could be fresh, stale or uncacheable:

1. If the response’s headers tell the cache not to keep it, it won’t.
2. If the request is authenticated or secure (i.e., HTTPS), it won’t be cached.
3. A cached representation is considered fresh (that is, able to be sent to a client without checking with the origin server) if:
   • It has an expiry time or other age-controlling header set, and is still within the fresh period, or
   • If the cache has seen the representation recently, and it was modified relatively long ago.

   Fresh representations are served directly from the cache, without checking with the origin server.

4. If an representation is stale, the origin server will be asked to validate it, or tell the cache whether the copy that it has is still good.
5. Under certain circumstances — for example, when it’s disconnected from a network — a cache can serve stale responses without checking with the origin server.

Browser cache as standalone solution

It is pretty easy to implement, but should we rely only on them? Browsers have limited space to store cached files (usually around 50MB). Bjӧrn Kasier have noted on his blog some important things:

1. Most users never clear their cache.
2. A rising number of (2.0) websites are constantly increasing in size coupled with well-documented instructions availabe for everyone in the web (e.g. Steve Souders) mean that operators are constantly having to work on squeezing as much efficiency out of user caches as possible.
3. Youtube, last.fm and other streaming stations are blowing up the cache with sometimes just one video/stream.
4. When looked at simplistically, the cache is a kind of “least recently used” (LRU) list, i.e. the files that come in and are not used for week may be dropped out, if meanwhile other files come in.
5. With some browsers, only the newer versions cache SSL content like that of xing.com (Firefox version 3.0 and above), and only if the cache header is ‘public’.

An increasing amount of content and optimized caching headers (see 2. + 3.) will lead to user disk caches filling up faster than ever. When a user’s cache is full (see 1.), data from the browser are deleted that actually need to be retained.

Around half of users would have empty cache when load pages from our site. Because of that browser caching has positive effect on our users experience but we cannot rely on it. Care about application cache, minimal requests, lazy loading, gzip compressing and others. Treat browser caching as additional feature.

How to make my site cacheable?

I have listed 10 steps to accomplish that:

1. Use the lastest version of web server

It will care about caching of static files itself. Web server checks if file have been modified and send Last-modified headers that force caches to reload. You must remember to update only new files. For instance, when updating your site, don’t copy over the entire site; just move the files that you’ve changed.

2. Use common library of static content

Static content are images, JS, CSS and other downloadable files. Once downloaded they will be cached even if used on different sites. Good idea is to host static files using light http servers like Nginx. If you are using jQuery or MooTools, use Google AJAX Libraries API. There is high probability that user have already downloaded this framework into his browser cache.

3. Use Expires and Cache-control and E-tag headers

They control browser and proxy caches. We must consider that caches aren’t required to keep a representation and reuse it; they’re only required to not keep or use them under some conditions. We may tell cache to keep file for 1 hour, but it’s cache decision if to follow that request. If we tall cache to not keep that file, it have no choice.

Expires header makes cache keep file until some moment. Specified cache will return cached resource instantly, without verifying each time if it’s fresh.

Second type of cache-controlling header is Cache-control. It may include several key-value pairs and flags, described below:

• max-age=[seconds] — specifies the maximum amount of time that an representation will be considered fresh. Similar to Expires, this directive is relative to the time of the request, rather than absolute. [seconds] is the number of seconds from the time of the request you wish the representation to be fresh for.
• s-maxage=[seconds] — similar to max-age, except that it only applies to shared (e.g., proxy) caches.
• public — marks authenticated responses as cacheable; normally, if HTTP authentication is required, responses are automatically uncacheable (for shared caches).
• no-cache — forces caches to submit the request to the origin server for validation before releasing a cached copy, every time. This is useful to assure that authentication is respected (in combination with public), or to maintain rigid freshness, without sacrificing all of the benefits of caching.
• no-store — instructs caches not to keep a copy of the representation under any conditions.
• must-revalidate — tells caches that they must obey any freshness information you give them about a representation. HTTP allows caches to serve stale representations under special conditions; by specifying this header, you’re telling the cache that you want it to strictly follow your rules.
• proxy-revalidate — similar to must-revalidate, except that it only applies to proxy caches.

E-tag is kind of hash of cached page. When browser wants to download logo.png, it asks cache server: “Can I get logo.png, if nothing matches tag ead145?”. Ead145 is e-tag generated by cache server before.

To enable above headers, follow this tutorial or use functions specific to your web-development language (perhaps PHP or Ruby).

4. Don’t use Cookies, POST and SSL

Encrypted pages are not stored by shared caches, use them only when you have to. Use images on SSL pages sparingly. Responses to the POST method aren’t kept by most caches; if you send information in the path or query (via GET), caches can store that information for the future. Cookies are difficult to cache, and aren’t needed in most situations. If you must use a cookie, limit its use to dynamic pages.

If you want to SSL representations be cached, use described above Cache-control: public header.

5. GZip resources

Compressing doesn’t take much extra load of server. Compressed files are easy to cache and there’s less network traffic and time to send them. But beware. If you use manually generated E-tag header. GZipped content contains different timestamp each time it is compressed. Without attention you would generate different E-tag for each requested fileFor tutorial go here.

6. Test your page with RED bot.

RED bot means Resource Expert Droid. It examines HTTP resources to determine how they will interact with Web caches, and generally how well they use the protocol. You may also check all resources on page by clicking “check assets” hyperlink. RED bot homepage.

7. Use Memcached!

Memcached is “Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load”.

Danga Interactive developed memcached to enhance the speed of LiveJournal.com, a site which was already doing 20 million+ dynamic page views per day for 1 million users with a bunch of webservers and a bunch of database servers. memcached dropped the database load to almost nothing, yielding faster page load times for users, better resource utilization, and faster access to the databases on a memcache miss.

It is the fastest solution for application cache and may speed-up your application even several hundred percent. For tutorial go to http://blog.ajohnstone.com/archives/installing-memcached Also consider checking your distribution’s package management system (apt, yum, etc).

8. Familiarize with your framework’s caching system

Every framework has tools that make caching easier. I listed below some of them:

• KohanaPHP
• CakePHP
• Symfony
• CodeIgniter
• Django
• Rails
• Pear

9. Use CGI_Buffer

GGI_buffer is a group of libraries (for Perl, Python and PHP) that automatically handle performance-improving HTTP features that most Web servers don’t implement (at least for scripts).

Currently, these include:

• HTTP/1.0-style persistent connections, using Content-Length. Persistent connections allow reuse of the TCP connection, making Web browsing much faster and more efficient. Web servers usually handle this for static content (i.e., files), but not for dynamic content.
• ETag generation and validation. ETags allow Web caches (in browsers as well as proxies) to store and validate responses, so that they can be reused; this avoids downloading the entire body when it hasn’t changed.
• Gzip Content Compression using Content-Encoding. By compressing the response, it is possible to make browsing perceptibly faster while saving bandwidth at the same time. Almost every modern browser supports this automatically, but Web servers are only beginning to do it (usually with a special module).

10. Don’t worry, be happy

If your site needs more than above steps, be proud that people are using your service, hire some professionals and enjoy your coffee

Useful links:

• How to optimize your site with HTTP caching
• MNot Caching Tutorial