Track this topic | Email this topic | Print this topic

A lot of people have noticed that the site has been having serious issues as of late. These outages will appear slightly different depending on web browsers used or other slight differences from user to user, but generally take the form of:

• A "connection refused" error;
• A blank white screen when loading the site;
• Clicking a link on the page will act as if the link is loading, but will eventually stop without loading anything new;
• A "timeout exceeded" error;
• And, most often, a SQL error that results in data not showing up on a page (on the main site) or a white screen with lots of text you probably can't decipher (when on the forums).

I want you to know now that the reason for these problems is simple. I'm jerking all of you around for my own personal amusement.

No, not really. The real reason that the site is going down at random times is that for some reason there is a fluctuating server load on our the computer that holds all of the CoN data (as well as the data for my personal site and other projects I have going, so know from that I take this issue quite seriously). As you may know, CoN hosts on a shared server that, by nature, other sites "share." See where they get the name? As best as I can discern, one of the sites on the server with us is doing something that is causing the high load from time to time. It could be a matter of some web developer who simply doesn't know what he or she is doing, and has coded a script that overloads the CPU. It could be something more nefarious, like someone using an account fraudulently to bounce spam. It could be someone using the control panel to back up their site much more often than is necessary. It could be a lot of things, and unfortunately I am not in a position to research and resolve the problem myself - or else it would be done already.

It seems that the host has some idea about the source of the problem and the staff are actively working to resolve it. I don't have a better timeframe than that at this time. At this time, I have two options to proceed and am weighing them to see which will be most beneficial to us.

And that's about it. I just wanted to let you guys know that we're not leaving you out in the cold here. But there might be questions still, and I welcome those. I'll start.

Q: What are the two options you have?
A: Simply put, stay on this server or ask to be moved to another server on the farm. Each has pros and cons.

Q: Why don't you buy a better server?
A: Again, to put it concisely, money. I'd love to have a virtual private or total-private server, but those are quite expensive; generally they're two to three times the cost per month as what I pay now. Maybe someday fibreoptic will be nice and affordable and I can set up an *nix box in my home office, eh?

The floor is yours. Discuss.

--------------------

"To create something great, you need the means to make a lot of really bad crap." - Kevin Kelly

Why aren't you shopping AmaCoN?

Out of curiousity, what are the pros and cons of asking to be moved? The downtime incurred by a move is an obvious con, and if the host is working to resolve it there's always the possibility that it'd be resolved soon and that a move would be wasted effort (at least, that's how I see it..)

If moving's costly (either financially or in terms of time/effort), and if the issue is being worked on by the host, it doesn't seem like a problem we couldn't live with (unless people have been filling up your inbox with complaints of downtime...) If CoN can live with the occasional ten-fifteen minutes of downtime for a (hopefully) short amount of time, then I say go for it. But if the issue persists and/or worsens, and your host doesn't seem to be able to resolve it, how much effort would moving the site to another server involve?

I'll be keeping my fingers crossed for things to work out in the end. It'd be quite ironic if CoN itself had a runaway script occasionally latching onto the CPU and driving the server into the ground for a few minutes. (Doubt it. The problem's fairly recent, AFAIK; I doubt a bug this serious would've laid dormant until just recently. It would be ironic though. )

--------------------

"Judge not a man by his thoughts and words, but by
the quality and quantity of liquor in his possession
and the likelyhood of him sharing."

Quote

I want you to know now that the reason for these problems is simple. I'm jerking all of you around for my own personal amusement.

We still love ya' big guy.

Quote (Silverlance @ 28th July 2005 16:11)

Out of curiousity, what are the pros and cons of asking to be moved? The downtime incurred by a move is an obvious con, and if the host is working to resolve it there's always the possibility that it'd be resolved soon and that a move would be wasted effort (at least, that's how I see it..)

If moving's costly (either financially or in terms of time/effort), and if the issue is being worked on by the host, it doesn't seem like a problem we couldn't live with (unless people have been filling up your inbox with complaints of downtime...) If CoN can live with the occasional ten-fifteen minutes of downtime for a (hopefully) short amount of time, then I say go for it. But if the issue persists and/or worsens, and your host doesn't seem to be able to resolve it, how much effort would moving the site to another server involve?

I'll be keeping my fingers crossed for things to work out in the end. It'd be quite ironic if CoN itself had a runaway script occasionally latching onto the CPU and driving the server into the ground for a few minutes. (Doubt it. The problem's fairly recent, AFAIK; I doubt a bug this serious would've laid dormant until just recently. It would be ironic though. )

You're spot on with those cons. In addition, CoN has a few custom configuration settings that may or may not be automatically propagted with a move. The downtime would be influenced by any reconfiguration necessary, as well as downtime associated with DNS changes that would need to be made.

Moving is free. Upgrading to a new server would be more costly. I'm not certain how often you come to the site, but it's not an "occasional" downtime. For the last couple days, it has been more along the lines of four or five times an hour for between five and 20 minutes. At least when I'm sitting here I can watch the server monitor and give chatters about a minute of advance warning when it's going to happen.

The loads seem to be better since last evening, it seems to me. In the afternoon yesterday, we had a server load peak of roughly 37, with repeated lower (but still extreme) values in the 9 to 15 range. As of this writing, though, the one-minute average load is sitting at a little under three, with the five-minute average under two and the fifteen-minute average at around 1.5. I'd be happier were it under one, but I think it's looking better so far.

And I can virtually guarantee that CoN is not the source of the problem. The scripts that the site uses were not only written by professionals in the field, we haven't added any major new functionality since February, and those scripts have been run on two different server farms since then too.

IP:

--------------------

"To create something great, you need the means to make a lot of really bad crap." - Kevin Kelly

Why aren't you shopping AmaCoN?

So, how's the situation? It seems things have gotten considerably better lately (here's to hoping I'm not jinxing the situation... ^^; ) but maybe it's just me.

Any news from your hosting? Maybe the guy pummelling the server with bad PHP scripts finally smartened up..?

--------------------

"Judge not a man by his thoughts and words, but by
the quality and quantity of liquor in his possession
and the likelyhood of him sharing."

Quote (Silverlance @ 5th August 2005 20:49)

So, how's the situation? It seems things have gotten considerably better lately (here's to hoping I'm not jinxing the situation... ^^; ) but maybe it's just me.

Any news from your hosting? Maybe the guy pummelling the server with bad PHP scripts finally smartened up..?

Heh, well, as I write this I watched the load spike to 13. I'm thinking it's because Sunday is when the maintenance scripts on the server are run, but they should still try to take steps to minimize that impact.

Beyond that, it does seem to have improved, doesn't it? I haven't been around much this week, as it has been a long work week and a long weekend, but I've heard fewer complaints and seen fewer issues firsthand.

There's no official word from the host as yet, and I don't really expect one unless they're 120% sure that the issue has been resolved, since we've all hung them up on crosses over this issue for a couple weeks now. That said, I see no way that the issue will ever be finally resolved, because it's simply the nature of the beast on a shared server that holds this many clients.

In the long run, to ensure optimum performance I will need to move the site to a VPS solution, I think. But that won't come until I have more clients to offset the cost, and it won't come until I have the time to oversee the setup of the server, as VPS tend to be a more DIY operation.

--------------------

"To create something great, you need the means to make a lot of really bad crap." - Kevin Kelly

Why aren't you shopping AmaCoN?

OK, here's the newest update.

We've purchased a VPS for the server. Things are looking good for me to be adding possibly two clients to my hosting service in the next month or two, and as such we're going to take the chance. I have been working on setting up the server for about three hours now, and I believe that Tiddles and I will be checking through things and porting over CoN files in the next twenty-four hours.

More information can probably be found here; I'm just giving you a bit of warning, so if you see the forums closed or the traffic dropping considerably, this is probably why. Let me know if there are more questions.

--------------------

"To create something great, you need the means to make a lot of really bad crap." - Kevin Kelly

Why aren't you shopping AmaCoN?

Moderator Edit

Golly, Outlook, it's nice of you to advertise yourself on our site. Unfortunately, we do not allow such things around here, even when they come from big time company stuff like you... - Mr Thou

This post has been edited by Mr Thou on 26th September 2005 09:22