 |
|  |
 |
|
Technical Assistant
Of the Peanuts Gallery
|
Apr 2004 time: 09:41
| |
|
|
The purpose of this locked thread will be to notify users of any planned maintenance, including planned outages/downtime, in a proactive manner, as well as communicate information about unplanned outages after the fact once our technical staff are aware of the nature of any outage.
We will attempt to give as much notice in advance as possible of planned outages, but as with any circumstance this may prove to not always be possible.
Please 'subscribe' to it if you want to be notified of any planned outage via email in advance (if you have set your thread notification settings to receive emails of subscribed threads).
This thread will be topped for a period in order to allow everyone to see it and subscribe to it, and then likely will be allowed to drop until it is needed for the next (hopefully distant) outage.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
Earlier today a subsystem failure caused a lockup which required a reboot and some additional recovery work to bring the database back online. All should be working now.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
We are currently experiencing some trouble related to the work done yesterday, and are preparing to revert parts of the work for now. We will schedule additional downtime to redo parts of the changes once we have isolated the error.
|
|
|  |
 |
|
Technical Assistant
Of the Peanuts Gallery
|
Apr 2004 time: 09:41
| |
|
|
The site seems to be currently stable, and no maintenance is currently in progress. Some maintenance will be required, including at least one reboot, tomorrow morning swedish time. Assuming the reboot ends up as a planned reboot (as opposed to a server crash) we will make a notification here ahead of time.
However, the backup solution still must be placed as planned, so there will need to be a period of sporaic downtime later on (in a few weeks in all likelihood).
|
|
|  |
 |
|
Technical Assistant
Of the Peanuts Gallery
|
Apr 2004 time: 09:41
| |
|
|
The site decided it preferred the unscheduled reboot, fortunately gramphos was able to bring it back up relatively quickly. The software that was causing the lock (we believe) was unloaded, so there should hopefully not be any more issues with reboots.
We will be undergoing some backup-related disk/cpu activity, however, that should last a few hours from now. Gramphos is adjusting the cpu priorities to give mySQL priority so this should have a small impact on the forums, but not the significant impact of yesterday. This should continue into the early morning US time.
We apologize for the delays and thank you for your patience!
|
|
|  |
 |
|
Co-Administrator
Latvia, Riga
|
Sep 2000 time: 17:41
| |
|
|
About an hour and a half ago, the site's server went down completely. It was rebooted and Apache was restored quickly enough, however, problems persisted with getting the MySQL server up. Operation has currently been restored and the site is currently stable. Thanks for your continued patience!
|
|
|  |
 |
|
Co-Owner/ Administrator
De Hel van Enschede
|
Nov 1999 time: 16:41
| |
|
|
Update: We've now opened an off-site service log as well in the form of a Wordpress blog: http://apolyton.wordpress.com.
The purpose of this blog is to keep you guys informed of what's going on during (planned or unplanned) server downtime. By maintaining this blog off-site we can keep the lines of communication open when everything else fails. It does not replace this on-site log but rather supplements it when the forums are unreachable.
Whenever the site is down we will let you know what's happening at the blog as soon as we find out ourselves (which will be in advance of the outage if it's planned, as soon as possible afterwards if not) and keep you up-to-date as we work to bring the site back online.
If you cannot get to the site but see no update on the blog, feel free post a comment in the first post to ask what's going on. If it's an issue on our end we will post a new blog entry to provide details and allow for further discussion as soon as possible, if not we will try to help you figure out the problem if we can.
You can subscribe to the blog's RSS feeds to be automatically notified of updates through your news reader, or use it in combination with a service like RSS FWD to receive email notifications.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
Earlier today the forum went unresponsive as I were doing some work with the database, underestimating the time inserting 7302 posts from backup would take.
On a related note the C4-SPDG now have 7302 posts restored, and post counts accounted for.
|
|
|  |
 |
|
Co-Owner/ Administrator
De Hel van Enschede
|
Nov 1999 time: 16:41
| |
|
|
As of about an hour ago, Apolyton’s hosting provider started having network connectivity issues. The server itself is up, but there are extended periods where no traffic can get to it. This has been an on-and-off affair (though mostly off) that’s outside of our direct control. Our hosting provider is handling it so we can only hope the issue will be resolved quickly.
Also note that in initially diagnosing the problem, I rebooted the server. Because of this, once connectivity to the site is fully restored you may initially experience some issues (like IRC chat not working). These will be resolved as soon as possible, but while we can’t get to the server we obviously can’t do anything about them.
Our apologies for any inconvenience, we hope it won’t last long.
|
|
|  |
 |
|
Co-Owner/ Administrator
De Hel van Enschede
|
Nov 1999 time: 16:41
| |
|
|
Just to follow-up on what you probably already knew: the problem seems to have gone away, the site has been continuously reachable pretty much ever since my previous post. All services have been fully restored. If you're still having problems, please let us know.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
I'm about to make some debugging of the Apache server running Apolyton. This will cause some minor interruption as I restart the server at the start and end of the debug session at a minimum.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
All done.
|
|
|  |
 |
|
Technical Assistant
Of the Peanuts Gallery
|
Apr 2004 time: 09:41
| |
|
|
Locutus' note from another thread on yesterday's interruption:
quote: originally posted by Locutus
Site is (obviously) accessible now, after a reboot this morning, but the problem isn't really solved yet, so it may come back to bite us again. I'm completely occupied with RL stuff now, because of that it may take me a while to track down the cause (and I don't think Gramphos is around ATM). |
|
|
|  |
 |
|
Technical Assistant
Of the Peanuts Gallery
|
Apr 2004 time: 09:41
| |
|
|
The server was down for a few hours around 2PM-5PM EDT.
quote: originally posted by DanQ SettlerIV stopped responding to HTTP requests, so I requested ThePlanet reboot
it manually. |
MYSQL did not come up smoothly after that reboot, and also was required to be rebooted, hence the further delay.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
I'm going to do a few tests to see if I can figure out what happened with the server the other day. This will lead to connectivity and/or database issues for the next 10 minutes.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
The tests are now completed.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
We are currently having issues with parts of our httpd setup. This mainly affects a subset of the apolyton subdomains, but might cause issues with the main domain or other subdomains as we are working on a fix.
|
|
|  |
 |
|
Technical Director
Chalmers, Sweden
|
Apr 2001 time: 16:41
| |
|
|
All systems should now be operational again.
|
|
|  |
All times are GMT. The time now is 15:41. Apolyton Time is 10:41. |
top of page
|
|
|
|
|
|