Throttling

The Content Engine has a a number of throttle services that you can use to limit the number of concurrent requests that various parts of the system will attempt to handle. Once the specified threshold is reached, requests to the overloaded part of the system will be refused.

The following throttle services are available:

WebServiceThrottle

Limits access to the Content Engine web service used by Content Studio.

DatabaseUpdateThrottleService

Limits the number of concurrent database updates.

DatabaseReadThrottleService

Limits the number of concurrent database reads.

JspThrottleService

Limits the number of concurrent page requests.

The throttle services are all enabled by default and set up with default configurations. You should not switch the throttle services off in a production environment, as overload situations are then likely to be handled in an unpredictable manner. You can, however, configure the throttle services by editing the appropriate files in one of your configuration layers (see Configuring The Content Engine).

All the throttle services are instances of the ResourceThrottle class, and are configured by setting ResourceThrottle properties. The most important property you can set is maximumConcurrent, which determines the maximum number of concurrent requests that will be handled.

For WebServiceThrottle, DatabaseUpdateThrottleService and DatabaseReadThrottleService, maximumConcurrent is set by default to 100, which is a relatively high value that can most likely be left unmodified. Database accesses should normally be controlled by the database system itself, so DatabaseUpdateThrottleService and DatabaseReadThrottleService can be seen as "failsafe" devices that will only ever be needed if something is badly configured elsewhere. Similarly, usage of the Content Engine's web service is unlikely under normal operation ever to reach a level of 100 concurrent accesses, even in large installations, so if this limit is ever reached, it is probably a sign that something is wrong.

JspThrottleService, on the other hand, is not just a failsafe device, it is vital to ensuring that the Content Engine handles periods of high activity in a controlled manner. Moreover, the optimum setting for maximumConcurrent is entirely installation-dependent, and must be based on experience and testing. For this reason, the default value is deliberately set set to a low value of 10. There is no sensible default: you must observe the Content Engine's performance and arrive at the optimum setting by trial and error.

In order to find out the optimum settings in a production environment, you need to examine performance numbers, and the number of HTTP 503 messages returned. The escenic-admin application's Performance summary option displays a page of performance data including an Activity Monitors section containing throttle activity data (see Activity Monitors).

The Current Usage column in the Activity Monitors section shows the current number of concurrent accesses. Above the Current Usage section, the /neo/io/reports/HitCollector entry in the Load Averages section shows the request load reaching the Content Engine. The Failures field shows how many requests have failed or been rejected. If failures are being recorded by the /neo/io/reports/HitCollector, and you see that incrementations of this value coincide with high Current Usage values for the JspThrottleService, then maximumConcurrent is probably set too low.

All the throttles are implemented using the ResourceThrottle class, and therefore have the same set of configuration properties, described in the following section.