EC-Cache Engine

(patent-pending at USPTO, s/n: 09/805,340)

One of the innovative features of the WhizzBee product family is the Extensible Cooperative Cache Engine (EC-Cache Engine). Each web server node in a WhizzBee cluster contributes a portion of its physical memory as cache to the cluster. By integrating these disparate memory pieces, a large virtual file cache is formed. A file fetched from the backend I/O store to a server node will be cached in the physical memory of the server node. When another server node needs the same file, it asks that other node to perform a cache-forwarding operation, instead of initiating an I/O access itself. The diagram below illustrates the software configuration of the EC-Cache Engine.

As shown in the figure, each server node runs a special "M-Agent" and a modified Apache web server. The M-Agents running at the server nodes periodically send dynamic load information to the central CLSERV (Cache-load server). The CLSERV running at the dispatcher node considers the real-time load information of the web server nodes and coordinates the access of the cooperative caches.

In order to better utilize the limited physical memory space, WhizzBee periodically removes outdated or less-frequently accessed files from the cache by using an efficient garbage-collection algorithm. The goal is to maximize the cooperative-cache-hit rate, while maintaining the highest data locality at each of the server nodes.

WhizzBee's innovative EC-Cache Engine considerably reduces backend I/O traffic. This removes the I/O bottleneck that is present in most server clusters. Furthermore, the cache-forwarding algorithm ensures better utilization of physical memory as the collective memory could hold a larger portion of the whole web site than traditional buffer caches. Most importantly, the consideration of dynamic load information prevents the web server nodes from being over-loaded by cache-forwarding requests. The benefit of using EC-Cache Engine is further illustrated in our performance study.