Disabling certain forum features on high loads

[Phil Harvey]

I have just enabled the forum "Load Balancing" feature, which disables certain forum features if the server load gets high.  Please let me know if you are experiencing any problems due to this.

This is the order in which the features get disabled:

• Search is already disabled (I may be able to re-enable it if this feature works, but it will be one of the first things to get disabled on high load)
• Disable unread topics
• Disable showing user posts
• Disable unread replies
• Disable the forum completely on very high loads

- Phil

[Phil Harvey]

Well, the load just exceeded the threshold to disable the forum, which was set to 40.  The actual load hit a peak of about 69.  This was nothing to do with the exiftool.org site, and was due to other processes on the shared server.  I'm raising this threshold to 96, and bumping the other thresholds as well.

Currently the load average is 21.89, but when I was testing initially it was typically running from 8 to 15 or so.  (For my reference, this is obtained at the command line with "cat /proc/loadavg".)

It looks like the server has 32 cores, so we shouldn't get too concerned unless the load average gets close to 32 or higher.

- Phil

[Phil Harvey]

The load average went above 96 a couple of times this morning which disabled the forum briefly.  I saw a high of 101.  The high load isn't due to our forum, but I will l keep the load balancing feature enabled so the dreamhost administrators won't get the urge to dump our forum when trying to solve high load problems.

- Phil

[Phil Harvey]

I've re-enabled forum searches at load levels of 32 or lower.  Read here for details.

- Phil

Edit:  I've just changed the forum search to use a Custom search index which should reduce the server load at the expense of using more disk space.

Edit2: I wasn't happy with the behaviour of the Custom search index, so I have changed this to use a Fulltext index, which actually uses less disk space but is supposed to be a bit slower.  However, the behaviour is much better and the searches actually return sensible results for me now.

[Phil Harvey]

I've been seeing more periods recently when the load gets above 96, which temporarily disables the forum.  This is inconvenient for sure, but by temporarily disabling the forum during these times I am hoping to prevent DreamHost from permanently disabling the forum (because they seem to start disabling things willy-nilly when the server load gets too high).  So far, so good, and DreamHost hasn't disabled the forum since December last year.

I've also upped the search threshold to 40.

- Phil

FYI: The DreamHost server has 32 CPU's, so the threshold of 96 is 3 running processes per CPU, which is quite high for a server.