I've got a hypothesis on what's happening.
The software that TFL runs on is a resource hog. It's particularly sucky in the Search Engine department.
Instead of having a true Search engine, the UBB creates a giant "index file" filled with a mash of every word typed on the UBB (minus words like the, a, and, etc). Every time you make a post, the contents get added to the search index file.
Second, there's an integral file that maintains a list of all threads made in the forum. Naturally, that file can get pretty hefty as well.
Now, when you've got a giant forum the size of Handguns, getting a ton of traffic, you get a significant drain on the server. Rich has a pretty powerful box running this site, a lot newer than ours, which is why they've been able to keep Handguns and the like open for so long without problems.
But, I'm going to make a Scientific Wild Ass Guess and say that somehow, something happened when someone made a post or two at the same time, that bogged down the search index file, whether with simultaneous access, or file locking, or something - causing the threads index file to become corrupted. When the file size gets too large, it takes longer for the processor to complete it's task, and if it gets interrupted for whatever reason - kB!
Once it became corrupted, it's screwed, because the UBB doesn't automatically rebuild the threads file from scratch, as that would be a huge waste of resources.
My suggestions?
Suggestion 1 - Shut down the forums for this weekend. Mass move most of the threads out of Handguns, Legal, and General to new, alternate forums and make them read only archives. Reindex and rebuild thread counts. Turn the forums back on.
The site may be down for about 12 hours, but everything should be fixed. The big forums will be less likely to suffer this problem in the near future.
Suggestion 2 - Turn off the search feature, and rebuild the thread files. Temporary fix - but it should work.
I'm still investigating UBB alternatives to get away from the drains this software puts out.
Thus far, the alternatives I've seen aren't going to cut it. They are
vBulletin -
http://www.vbulletin.com
wwwthreads -
http://www.wwwthreads.com
Both run on SQL / mySQL backends, but neither has a solid following or immensely great track record IMHO.
Spark
------------------
Kevin Jon Schlossberg
SysOp and Administrator for BladeForums.com
www.bladeforums.com