Tips on controlling and increasing the capacity of your system.
Before attempting to tune your system, be sure that you have read and
fully understand the Cyclone Configuration
Guide and all of the options to cycloned. (Type
"bin/cycloned -help" to display all of the options).
Be sure to examine the overall performance of your system. Use
"netstat -i", "vmstat", "iostat -x", "mpstat", "top", and "sar
-A" to monitor your network utilization, network collisions,
CPU utilization, paging, scan rate, swapping, disk I/O rates, RAM,
virtual memory consumption, and overall system load. Understand these
tools and their output before making radical adjustments to your
system.
Use "tools/nntpTime" to measure the NNTP performance
of your system from both local and remote locations. (Type
"tools/nntpTime -help" to display all of the
options). "nntpTime" is very good at measuring
NNTP performance. Periodically running "nntpTime -config
../cyclone.conf -feeds ../feeds.conf" can be useful for
monitoring the performance of all of your peers.
When you are making performance adjustments, it is usually a good
idea to lower the interval between statistics reports. For example,
you can use "-update 300" to make the system output
statistics data every 5 minutes. In addition, you can run the
"statsnow" command script to output statistics at any
time.
The key to good performance is spreading system load across multiple disk drives. To do this, use the cyclone.conf file to create a Spool object on every disk drive in your system. If you have access to RAID technology, create a number of spools on the RAID drive. For maximum performance, use multiple disk controllers to increase bandwidth and parallel access to your disk drives.
In addition, you may wish to place the history database and the
"spool" directory on separate drives. You can use the
"-history" option to change the location of the history
database.
If you have lots of RAM and swap space, you may wish to place the
history database in "/tmp" and set up an hourly cron(2)
job to copy the database back to disk.
Another key to good performance is parallel feeding. Using the NumberOfStreams directive, you can use multiple connections to feed your peers. This can greatly increase the throughput to both slow and fast peers. However, parallel feeding should be used cautiously since not every site has the capacity to handle multiple connections.
If you have the capacity, invite your peers to use multiple
connections to feed your sites. Be sure to use nntpTime
to keep an eye on your own performance and use the MaxIncomingNumberOfStreams
directive to ensure that peers do not unfairly consume your
resources.
In most situations, you should NOT override the default values for AllowStreaming (True) or MaxChecks (35). Cyclone's NNTP streaming mode is highly adaptive and will make the most of the available network bandwidth. In addition, you may want to turn OFF the AllowTryAgain directive to your peers that are very well connected.
Unlike most News systems, Cyclone doesn't really "expire" articles. The system considers an article "expired" when it is overwritten in the spool area before it has been sent to all the interested peers.
To maximize performance, you should try to avoid expiring articles. Although expiring articles is perfectly fine, it is more efficient to "drop" articles. The results are the same, so, if you can avoid article expiration it will help performance.
If you want to avoid expiring articles, you can increase the size of your spool by creating more or larger spool objects. Alternatively, you can use the MaxDepth directive to decrease the backlog for the peer in question.
The MaxDepth directive gives you control of the outgoing article queue depth (or "backlog") for each feed. To maximize the capacity and efficiency of your system, it is important to understand the following observations about article queues:
Larger queues provide buffering in the event of network congestion, peer throttling, or high-speed incoming bursts of articles.
When full, smaller queues ensure that the articles at the front of the queue are more recent. Recent articles are more likely to be accepted by peers that are fed by more than one site.
When a queue is full, it will overflow at the same rate regardless of the size of the queue. A full 1,000,000 article queue and a full 1,000 article queue will BOTH overflow at the difference between the filling and emptying rates.
From these observations, HighWind recommends the following "rules of thumb":
Start with a MaxDepth between 2000 and 10,000 for all peers. Do NOT adjust MaxDepth until you are sure it will make a difference.
For peers that depend on you as their only or primary feed, provide a larger MaxDepth.
If the outgoing statistics report shows "Average Backlog" pegged at the MaxDepth and a large percentage of dropped articles, the peer in question can NOT keep up. You will ALWAYS drop articles to this peer. Reduce the MaxDepth. The REAL solution is for the peer to upgrade their server or accept a smaller feed.
For peers that are dropping articles but do not have statistics reports showing "Average Backlog" pegged at the MaxDepth, increase the MaxDepth. This will prevent articles from being dropped due to temporary incoming article bursts that are beyond your peer's incoming capacity.
For "fast" peers, increase MaxDepth to provide buffering in the event of minor outages. Fast peers will "catch up" on any backlog, so, giving them a huge MaxDepth will avoid dropping articles in the event of temporary outages. Rely on the statistics reports and nntpTime rather than your own intuition in identifying "fast" peers.
The more feeds and connections you have the more memory you will consume. If you are short on RAM, make every effort to minimize the number of feed objects.
For example, use the NumberOfStreams directive rather than distinct feed objects to enable parallel feeding. Also, you can use a single feed object with an IncomingHostname and an OutgoingHostname directive rather than two separate feed objects.
Since Cyclone's RAM usage varies depending on incoming and outgoing article rates, disk throughput, simultaneous incoming and outgoing streams, and article access patterns, it is difficult to gauge exactly how much you will need for high-performance. As always, more RAM is better.
If we were pressed, we'd say that having ~1MB of RAM per connection would be more than adequate for most installations.
We recommend having PLENTY of swap space. Since Cyclone is highly parallel, it can be accessing hundreds of articles simultaneously. If all these articles are large, it can require huge amounts of virtual memory. Since running without enough virtual memory is difficult, be very generous with swap space.
|
Back to Cyclone Documentation |