What happed to TFL?

ahenry · May 7, 2002, 01:43 PM

Does anybody know what caused the down time?

fix · May 7, 2002, 01:46 PM

Judging from the dates on some posts, it wasn't down the whole time that it appeared to be. I couldn't get on for 3 days, but others apparently did.

**Rich Lucibella** · May 7, 2002, 01:50 PM

Unknown.
At 4 AM Monday, I was doing some file maintenance. Noted that the database backup directory was at 97% occupancy due to one large backup file...the most recent.....so I blew it away.

As I hit the Return button, I did a "DOH!". Then I checked the automated tasks...at 4 AM, the system is building the backup...that's why it was temporarily so large.

CRASH!

(Actually, the system should not have crashed...the data files were still in tact. The problem was server load due to the server trying to read/write to non-existent files....but it hung anyway.)

Tried a remote reboot, which would have resolved it and that's when problem 2 reared it's ugly head:

The system does not want to boot up reliably. It continues to hang up at different points in the boot sequence and requires that we have someone physically standing by at the location. Roger went out there at 5:30PM EST yestereday. Between us, we got the system up and I did a repair of the database (nothing lost).

I then ran out and came back at 10:30PM to find the server load maxed out. Tried some ginger manipulations, but ginger wasn't co-operating. Result...I could FTP in, but not get on thru the Web. SWAT was working fine, though...on the same box! As I'd been out to dinner and had a couple of drinks, I left it as I found it....I learned a long time ago that alcohol doesn't mix with server maintenance.

This AM, GlockTalk's Eric called to ask if I needed help. I got him into the server and we started looking around. Finally restarted the Apache server and all looks in order.

Now Known:
We need more memory. 1.25GB ain't making it. I'll start looking to up it to 2GB.

Now Suspected:
There does appear to be a network conflict, probably outside our box at the Web Host center. Eric's suspicion: they never changed the pointers from the temp page that was up during the crash and the network is trippin' and skippin'. Will be working on this also.
Rich

ahenry · May 7, 2002, 04:18 PM

Quote:

Tried some ginger manipulations, but ginger wasn't co-operating.

Damn that ginger!

Thanks for the hard work in keeping TFL up and running. I appreciate it.

coati · May 8, 2002, 08:46 AM

I thought I'd been blocked at work. Monday, no joy at work. Went home, everything's fine. Back to work Tuesday, no luck. Checked on GlockTalk (usually a note "where's TFL???"), no note.

I thought I was going to have to sweettalk a sysop. Ewww.

**Mike Irwin** · May 8, 2002, 02:27 PM

I thought I had gotten work blocked, too. Luckily not.

May 7, 2002, 01:43 PM	#1
ahenry Senior Member Join Date: January 25, 2001 Location: Alabama Posts: 1,764	What happed to TFL? Does anybody know what caused the down time? __________________ Doing what you've done, gets you what you've got.

May 7, 2002, 01:50 PM	#3
Rich Lucibella Staff Join Date: October 6, 1998 Location: South Florida Posts: 10,229	Unknown. At 4 AM Monday, I was doing some file maintenance. Noted that the database backup directory was at 97% occupancy due to one large backup file...the most recent.....so I blew it away. As I hit the Return button, I did a "DOH!". Then I checked the automated tasks...at 4 AM, the system is building the backup...that's why it was temporarily so large. CRASH! (Actually, the system should not have crashed...the data files were still in tact. The problem was server load due to the server trying to read/write to non-existent files....but it hung anyway.) Tried a remote reboot, which would have resolved it and that's when problem 2 reared it's ugly head: The system does not want to boot up reliably. It continues to hang up at different points in the boot sequence and requires that we have someone physically standing by at the location. Roger went out there at 5:30PM EST yestereday. Between us, we got the system up and I did a repair of the database (nothing lost). I then ran out and came back at 10:30PM to find the server load maxed out. Tried some ginger manipulations, but ginger wasn't co-operating. Result...I could FTP in, but not get on thru the Web. SWAT was working fine, though...on the same box! As I'd been out to dinner and had a couple of drinks, I left it as I found it....I learned a long time ago that alcohol doesn't mix with server maintenance. This AM, GlockTalk's Eric called to ask if I needed help. I got him into the server and we started looking around. Finally restarted the Apache server and all looks in order. Now Known: We need more memory. 1.25GB ain't making it. I'll start looking to up it to 2GB. Now Suspected: There does appear to be a network conflict, probably outside our box at the Web Host center. Eric's suspicion: they never changed the pointers from the temp page that was up during the crash and the network is trippin' and skippin'. Will be working on this also. Rich __________________ S.W.A.T. Magazine Weapons, Training and Tactics for the Real World Join us at TFL or at AR15.com or on Facebook

May 8, 2002, 08:46 AM	#5
coati Senior Member Join Date: November 2, 2001 Location: Just outside the beltway, VA Posts: 288	I thought I'd been blocked at work. Monday, no joy at work. Went home, everything's fine. Back to work Tuesday, no luck. Checked on GlockTalk (usually a note "where's TFL???"), no note. I thought I was going to have to sweettalk a sysop. Ewww. __________________ coati

May 8, 2002, 02:27 PM	#6
Mike Irwin Staff Join Date: April 13, 2000 Location: Northern Virginia Posts: 41,388	I thought I had gotten work blocked, too. Luckily not. __________________ "The gift which I am sending you is called a dog, and is in fact the most precious and valuable possession of mankind" -Theodorus Gaza Baby Jesus cries when the fat redneck doesn't have military-grade firepower.

May 7, 2002, 01:46 PM	#2
fix Senior Member Join Date: December 28, 2001 Posts: 919	Judging from the dates on some posts, it wasn't down the whole time that it appeared to be. I couldn't get on for 3 days, but others apparently did.