Website Rollback - Latest Info

[Mike Furness]

Just had to log in again from the last time ... usually don't have to do that. But after the quick login the site seems to be working very well. No worries here Steve ... I know you are doing the best for the forum ... No complaints from me! 😀

[Steve Herschbach]

We have still made progress, only rolled back 24 hours this time. So the gains made as far as fixes and performance were not lost. It appears we may be so lucky as to have multiple issues, both on the software end, and on the hosted database end. Corrupt indexes point to database issue. I have the hosting people looking into that, also checking on a dedicated sever option, and also looking into a move to an entire new service location hosted directly by Invision as options. Doing that eliminates the finger pointing issue at least.

Can't swear there will not be another crash and 24 hour rollback in the meantime. I'm limited to one backup a day, made the last one at 2am last night. So next one will be tomorrow morning assuming everything goes well until then. But until then, be aware posts made, well, no promises. I'm tempted to shut off new member options for now as it's a bit of a bait and jerk having people join only to get erased when I roll back.

[Rattlehead]

Not sure what kind of traffic you get here or what type of hosting you have, but I've had good luck using a VPS through a company called Knownhost. My site generally has 500-800 users online in peak hours and still loads lightning fast. Been using them for several years now with no complaints. If the problem turns out to be the host having you on an overly crowded sever, VPS might be a cheaper option rather than paying for a dedicated box.

[Steve Herschbach]

The forum is not a trivial load but not impossibly large either. 640MB database, 40GB of server space used (mostly images and pdf plus forum files), 1200GB of monthly bandwidth generated by 9000 page views a day.

I'm on a shared virtual private server, and have had it moved in the past due to other users causing me problems. A dedicated option would cost 50% more but eliminate that problem. Not impossible but I need to know it would really be worth the additional expense.

Due to the forum size and bandwidth requirements having Invision host may be very expensive, but I am getting a quote as I really do not know. I'd do that rather than try another host as that option still leaves me with two vendors both tending to disclaim issues instead of knowing one is firmly in the crosshairs. Invision rightfully tends to tell people self hosting that it also comes with responsibilities beyond their control. Pair of course tends to just say server is running or not, but they are one of the oldest and most reliable in the business, not a low cost option. I have been with them for 30 years on several websites and forums. I have no reason to distrust them - they have always taken care of me. So we will see.

In summary, getting a quote from Invision on a move, have Pair backtracing database problems, also getting quote on dedicated server option, will report back.

There is a small chance even that depending on what I find out, we can go back to the last backup made before the two month rollback, in effect bringing all the lost stuff back and forum back up to date. 

[Clay Diggins]

1 hour ago, Steve Herschbach said:

The forum is not a trivial load but not impossibly large either. 640MB database, 40GB of server space used (mostly images and pdf plus forum files), 1200GB of monthly bandwidth generated by 9000 page views a day.

That's a pretty light load Steve. 640 Mb database is puny (Mine just hit 63 Gb!). Land Matters runs about 120,000 pages per day and I just restarted the server a week ago for routine maintenance. It had been running 7 years straight without a glitch. I don't think your problems are related to the load.

You might try a vacuum and analyze on your database. That should fix any cross table blocks. My only other suggestion would be to try splitting your really weak database into two instances to reduce the workload? That's most likely where the problem is coming from. I understand you don't want to try an actual working database like Oracle or Postgres - they are easy to set up but would be a lot of work getting the db to interface with the forum software. I doubt Invision will ever update their database choice so splitting the load might be your best bet.

I wish you luck. I feel for you. Dealing with two companies when you have a software glitch often leads to finger pointing with no results. If there is anything I can do to help just PM me.

Thanks for all you do Steve.

[Steve Herschbach]

32 minutes ago, Clay Diggins said:

That's a pretty light load Steve. 640 Mb database is puny (Mine just hit 63 Gb!). Land Matters runs about 120,000 pages per day and I just restarted the server a week ago for routine maintenance. It had been running 7 years straight without a glitch. I don't think your problems are related to the load.

You might try a vacuum and analyze on your database. That should fix any cross table blocks. My only other suggestion would be to try splitting your really weak database into two instances to reduce the workload? That's most likely where the problem is coming from. I understand you don't want to try an actual working database like Oracle or Postgres - they are easy to set up but would be a lot of work getting the db to interface with the forum software. I doubt Invision will ever update their database choice so splitting the load might be your best bet.

I wish you luck. I feel for you. Dealing with two companies when you have a software glitch often leads to finger pointing with no results. If there is anything I can do to help just PM me.

Thanks for all you do Steve.

Thanks Clay. I ran a database alanlyze this morning after loading the backup from yesterday and all reports happy days. Before that it was telling me the reputations index was corrupt and unrepairable, and apparently that makes posts unfindable. It might be a configuration error tied to a nightly routine Pair runs on their servers. Something is causing a couple indexes to go corrupt and be unrepairable, at least with phpmyadmin.

This forum is 13 years old. In that time I have only had to repair the database a handful of times, and never once needed to roll to a backup. For over a decade I had to do nothing - nothing. It just worked. It has not gotten larger, been running about the same for several years. And like you say, nothing super large.

So something happened. Either software end or hardware end or, it seems, both. For me and where I am in life I have exactly two options. Run Invision on a hosted mysql server, or run Invision on their servers. This is something that is worth doing for me only as long as I really don't have to do anything, which is the way it is supposed to be and has been. Now it's like split the database? Use a new backend and create an interface? Migrate to a new forum software? The question from my perspective is "why?" Why bother? If the ninnies that charge me monthly for the software and the ninnies that charge me for the hardware can't tell me what is going on and fix it, then it's up to this little ninniest of ninnies to somehow make it work? It worked before and they are the pros, so fix it and make it run again. Is that too much to ask?

There was a time I'd love the computer challenge, dig in, rise to the occasion. Now? Sorry, but I'd rather just go walk my dogs. Maybe go play Fallout some more. I'm sure some Super Mutants need killing. 

So for people that want this forum to stay all I can say is Pair and Invision better come up with something that is palatable to me so I can get back to my real life.

Thanks again Clay. If it was you it would be dialed but it's not, so hopefully it sorts out with minimal effort on my part, because I'm not up for more than that.

[King-Of-Bling]

8 minutes ago, Steve Herschbach said:

Thanks Clay. I ran a database alanlyze this morning after loading the backup from yesterday and all reports happy days. Before that it was telling me the reputations index was corrupt and unrepairable, and apparently that makes posts unfindable. It might be a configuration error tied to a nightly routine Pair runs on their servers. Something is causing a couple indexes to go corrupt and be unrepairable, at least with phpmyadmin.

This forum is 13 years old. In that time I have only had to repair the database a handful of times, and never once needed to roll to a backup. For over a decade I had to do nothing - nothing. It just worked. It has not gotten larger, been running about the same for several years. And like you say, nothing super large.

So something happened. Either software end or hardware end or, it seems, both. For me and where I am in life I have exactly two options. Run Invision on a hosted mysql server, or run Invision on their servers. This is something that is worth doing for me only as long as I really don't have to do anything, which is the way it is supposed to be and has been. Now it's like split the database? Use a new backend and create an interface? Migrate to a new forum software? The question from my persepctive is "why?" Why bother? If the ninnies that charge me monthly for the software and the ninnies that charge me for the hardware can't tell me what is going on and fix it, then it's up to this little ninniest of ninnies to somehow make it work?

There was a time I'd love the computer challenge, dig in, rise to the occasion. Now? Sorry, but I'd rather just go walk my dogs. Maybe go play Fallout some more. I'm sure some Super Mutants need killing. 

So for people that want this forum to stay all I can say is Pair and Invision better come up with something that is palatable to me so I can get back to my real life.

Thanks again Clay. If it was you it would be dialed but it's not, so hopefully it sorts out with minimal effort on my part, because I'm not up for more than that.

I think this is a great forum. With that said , the way I look at it is I was fine before I came on here and I'll be fine without it. But I'm not like you or everybody else. But I totally agree that sometimes things aren't worth the stress , let alone the cost. Whatever you do Steve , life is short. Enjoy it while you can.

[Rattlehead]

32 minutes ago, Steve Herschbach said:

 

There was a time I'd love the computer challenge, dig in, rise to the occasion. Now? Sorry, but I'd rather just go walk my dogs. Maybe go play Fallout some more. I'm sure some Super Mutants need killing. 

 

I can relate to that. Back in the early 2000s I spent countless hours working on websites and coding my own custom plugins for vBulletin forum software. I actually enjoyed doing it back then because it was all brand new to me and I liked the challenge of learning new things. Nowadays, I just don't have the drive or desire. Heck, I barely even read my own forum anymore since I no longer own the type of motorcycle it was designed for. As long as the site runs hassle free, I'm okay with keeping it going. It does make me a little extra $$$ and has 15 years worth of useful info stored there. If it ever gets to the point that it takes too much of my time, it would have to go.

[Steve Herschbach]

7 minutes ago, Rattlehead said:

I can relate to that. Back in the early 2000s I spent countless hours working on websites and coding my own custom plugins for vBulletin forum software. I actually enjoyed doing it back then because it was all brand new to me and I liked the challenge of learning new things. Nowadays, I just don't have the drive or desire. Heck, I barely even read my own forum anymore since I no longer own the type of motorcycle it was designed for. As long as the site runs hassle free, I'm okay with keeping it going. It does make me a little extra $$$ and has 15 years worth of useful info stored there. If it ever gets to the point that it takes too much of my time, it would have to go.

That's exactly where I am.