Connection reset errors

[Ben Swenson]

I've been getting connection reset errors from TFL on a fairly regular basis (maybe 5% of page loads).

Anyone else getting these?

FWIW, refreshing usually brings up the page.

[tyme]

After a significant delay or within 20 seconds or so?

I haven't noticed anything, but it would explain some other things...

[Ben Swenson]

After a significant delay or within 20 seconds or so?
Oddly enough, no.

On Firefox, if I'm paying attention I can see the page start to load and then immediately flash to the Connection Reset error page.

Total time from click to error is less than five seconds.

I am not getting this error with other sites. It may well just be my machine or connection, but I thought I'd see if others were noticing a problem. I've seen a bunch of multiple posts that might be caused by a similar error (post is submitted, connection is reset, poster doesn't think post was submitted, page refreshed, post is resubmitted).

[tyme]

I have a sneaking suspicion the firewall isn't natting connections reliably. Will look into it.

[Ben Swenson]

Thanks Tyme.

And thanks for all the work you've put into geting TFL back online.

[tyme]

The new colo has some agressive network filters. It's possible you're inadvertently tripping one. When it happens again, get an accurate timestamp (plus timezone), and pm me that and your ip (if different from the one you used to post these last few messages) and I'll ask the colo people to check.

[Ben Swenson]

Will do.

[Ben Swenson]

Just had a handful of these errors. Sent you a PM or two. Hopefully two.

Got another one when trying to load the Reply screen here.

[tyme]

Has anyone else noticed occasional connection resets?


I sent the info to the colo for them to check. There's nothing relevant in the apache error log.

Assuming they claim it's not a firewall issue, can you run a packet sniffer and log traffic to/from tfl until it happens again?

I'm running tcpdump on the server for your ip. If you can get a packet log from your side and a rough timestamp (just note the url and rough time and I can get an accurate timestamp), comparing the two logs should conclusively resolve whether the firewall's at fault.

Even without the log, given another timestamp we can at least find out if the tfl server is the culprit. There just won't be any evidence that the colo's firewall is sabotaging the connection.

There are other strange network lags/disconnects with the server, but I haven't noticed anything of that sort on the website.

[Ben Swenson]

Assuming they claim it's not a firewall issue, can you run a packet sniffer and log traffic to/from tfl until it happens again?
Sure thing. Running EtherDetect filtered for TFL's IP right now.
I'm running tcpdump on the server for your ip.
Argh! Big brother! Big brother! ;)
If you can get a packet log from your side and a rough timestamp (just note the url and rough time and I can get an accurate timestamp), comparing the two logs should conclusively resolve whether the firewall's at fault.
You got it. I'll PM you if I get another one of those disconnects.

[Ben Swenson]

Got one when going to http://www.thefiringline.com/forums/showthread.php?t=211533 at about 0951 eastern.

Packet log forthcoming.

[tyme]

Your firewall seems to be broken.

You got two TCP resets 6 seconds after the http request, and they weren't sent by the TFL server. 15 seconds after the http request, the TFL server got a RST out of the blue, with reset cause given as "ehnc", documented here:
http://72.14.209.104/search?q=cache:8P1WR-r5pbUJ:www.sonicwall.com/support/pdfs/technotes/SonicOS_TCP_RST.pdf+sonicwall+ehnc&hl=en&gl=us&ct=clnk&cd=2

The response to any given page request has a half dozen or more tcp fragments. In this case, one of them got lost (seq num 5841 is missing, the next one you got was seq 7301 in packet 1968 in your log).

SonicWall must have decided, inappropriately, that a missing tcp fragment constituted a Breach of the Peace, and proceeded to wreck the connection by sending a RST to your machine. After it closed its connection, the firewall dropped the connection from its connection table, and when TFL retransmitted the lost fragments, it got a RST with the "ehnc" message.

If there's a newer sonicwall firmware than you're using, try upgrading.

[Rich Lucibella]

I coulda figured that out....
.
.
-
.
.
.
.
.
.
.
.
.
.
.
.
.
NOT!

[Capt Charlie]

Heck Rich, at least you understood what he said! I'm still trying to figure out what language he's speaking! :D

[Rich Lucibella]

He's speaking a decipherable language?
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Yeah, I knew that.
Rich

[Al Norris]

LOL!!

Ben, ya just gotta thank your lucky stars that tyme is a good guy!

Charlie? Ya really don't want to know!

[Capt Charlie]

Charlie? Ya really don't want to know!
Know what? That's he's got transistors instead of neurons and thinks in binary? :D

Just kidding Tyme! You're one sharp cookie and I'm envious :) .

Back when DOS was Boss, and Veronica and Archie were more than comic book characters, I was halfways sharp at this stuff (I still use DOS for a lot of things), but I just couldn't keep up with it. Today, I'm so far behind I could never catch up. Besides, the mind just ain't as sharp when you're pushing 60 :( .

[Al Norris]

That's he's got transistors instead of neurons and thinks in binary?
Reminds me of the old sigline: There's 10 kinds of people. Those who know binary and those who don't.

[Ben Swenson]

Thanks, Tyme! Sorry to burn so much of your ... er ... time.

We're moving offices in a few weeks and after we get settled in I'll rebuild the firmware on our firewall.

[Edward429451]

It cleared up for a few days but I'm getting garbled E-Mail notifications again. Not all of them but maybe 1/2 or so.

[tyme]

Are you sure they're not old notifications? I got a few notifications yesterday from posts on May 23rd-ish. They may or may not have been "garbled" (the garbling only happens on old/utf8-incompatible email clients, and I didn't check for that in the email before deleting it)

If you're getting notifications dated this week that are garbled, please repost.

[Edward429451]

I got 4 garbled ones total. The last good one was at 139 pm on 05-30 followed by three garbled ones at 2:09 and one more garbled one at 2:10 pm and havent got any more garbled ones since.

[tyme]

I think the colo people didn't notice that I ditched qmail for postfix and were messing around with stuff they didn't understand.