RPTools.net

Discussion and Support

Skip to content

It is currently Fri Apr 28, 2017 9:11 am 






Reply to topic  [ 32 posts ]  Go to page 1, 2, 3  Next

Previous topic | Next topic 

  Print view

Author Message
User avatar  Offline
Site Admin
 
Joined: Mon Jun 12, 2006 12:20 pm
Posts: 12064
Location: Tampa, FL
 Post subject: Site Outage from 8/23-8/24
PostPosted: Sat Aug 24, 2013 4:03 pm 
We had an outage of the site for about 24 hours this weekend. The cpu and memory use of the site spiked and stayed pegged for almost an hour and HostGator had to disable the site. Then it took me a little time to work with them regarding what the issues were and the best steps to correct them.

The primary culprit seems to be the gallery software. I have renamed the directories that contain the software and the data to avoid them from being used. During the week I'll take a look and see if: (1) the software is upgradeable, (2) whether the upgrades provide a performance improvement, (3) whether the work involved is justified to keep the gallery, and (4) whether there are forum users who link directly to the gallery for their avatar image or whether there are posts that link directly to the gallery.

Trevor has indicated in the past that he wanted to wipe it out -- there are plenty of file/image sharing sites on the web so does RPTools really need to provide one?

That will be part of the evaluation process this week. I may be able to bring the gallery back up for a short amount of time, but don't expect it. I might simply create a read-only directory of images (if I can) or simply leave it turned off (if I have to).

The good thing about all this is that the inode count will drop below the threshold where the site receives automatic backups from HostGator. The site hasn't been below that threshold for a few years now (!) which means no backups of the data other than the ones I make when I'm about to do a forum upgrade or perform similar maintenance.

I'll post here again when I have more information, but it won't be until 8/26 because my weekend is pretty full.


Top
 Profile  
 
User avatar  Offline
TheBard
 
Joined: Tue Mar 21, 2006 7:26 pm
Posts: 3467
Location: Austin, Tx
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 24, 2013 4:56 pm 
Thanks for the update, Az, and thanks for working the problem. I'll be sad to lose the gallery but it's far preferable to losing the site!

_________________
ImageImage ImageImageImageImage
MapTool/Savage Worlds Blog
Free SW/MT Modules
SW Framework for MT


Top
 Profile  
 
 Offline
Cave Troll
 
Joined: Fri Jul 02, 2010 3:43 am
Posts: 67
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 24, 2013 5:32 pm 
Ah crap I knew it was bad idea to have images linking back to this site, no offense. It just seems like a better idea to host them on actual image hosts. Ya literally when I first started using the fancy image buttons for macros I was like....hmmm what happens if the server goes down or becomes unreachable.


Top
 Profile  
 
User avatar  Offline
Giant
 
Joined: Thu Sep 15, 2011 6:52 pm
Posts: 120
Location: Grand forks ND
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 24, 2013 6:22 pm 
How big is the gallery ?


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Fri Mar 20, 2009 4:40 am
Posts: 9191
Location: Netherlands
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 24, 2013 7:01 pm 
there are some pretty nice things on the gallery it would be a waste to can them. Maybe theres a possibility to zip them up and put them on some torrent, download site or such?

_________________
GETTING STARTED WITH MAPTOOLS - TUTORIALS, DOCS, VIDEOS, TOOLS, ETC

My stuff
Excel Tools: Table and Light editors
MT Tools: Bag of Tricks: Tools for Maptool, Dungeon Builder I, Dungeon Builder II and onMouseOverEvent
Frameworks: Dark Heresy, Rogue Trader, Deathwatch, Black Crusade, Only War, SET Card Game, RoboRally
Wiki: Debugging Tutorial, Speed Up Your Macros, Working With Two CODE Levels, Shortcut Keys, Avoiding Stack Overflow, READ THIS


Top
 Profile  
 
User avatar  Offline
Demigod
 
Joined: Mon Jun 29, 2009 9:37 am
Posts: 3108
Location: Germany
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sun Aug 25, 2013 3:18 am 
Yeah, it would be a shame to throw it all a way.

An archive should be possible to host somewhere.

_________________
HELP: GETTING STARTED WITH MAPTOOLS - TUTORIALS, DOCS, VIDEOS, TOOLS, ETC.

most complete list of my maptool stuff


Top
 Profile  
 
 Offline
Cave Troll
 
Joined: Fri Jul 02, 2010 3:43 am
Posts: 67
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sun Aug 25, 2013 6:16 pm 
I vote for a torrent that would be nice, don't have to front the cost.


Top
 Profile  
 
User avatar  Offline
Cave Troll
 
Joined: Fri Jun 01, 2007 12:54 pm
Posts: 85
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Mon Aug 26, 2013 10:25 am 
wolph42 wrote:
there are some pretty nice things on the gallery it would be a waste to can them. Maybe theres a possibility to zip them up and put them on some torrent, download site or such?


CoveredInFish wrote:
Yeah, it would be a shame to throw it all a way.

An archive should be possible to host somewhere.


Volomon wrote:
I vote for a torrent that would be nice, don't have to front the cost.



Yes! I support this idea too. Please. I like the torrent distribution.

PS: If it is possible i prefer a compression in multiple volumnes (1 GB or so).

DJuego


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Sun Jan 13, 2008 5:52 pm
Posts: 871
Location: Lakewood, CO North America
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Mon Aug 26, 2013 9:42 pm 
Please just give us enough time to get our stuff off there. I don't have back ups of much of my stuff and put tons of hours into that.

..not quite sure how the access works though as I'm not tech saavy.. .php what?

:)

jh

_________________
Yes, I'm a chiropractor. Gamer fitness at Hafner Chiropractic in Lakewood, CO: http://www.HafnerChiropractic.com


Top
 Profile  
 
User avatar  Offline
Site Admin
 
Joined: Mon Jun 12, 2006 12:20 pm
Posts: 12064
Location: Tampa, FL
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Tue Aug 27, 2013 10:25 pm 
I still haven't looked at this, life being busy and all. But yes, I understand it would be a bummer to lose it.

It looks like the next couple of days are going to be full of "non-billable-but-have-to-do" work, so Friday at the earliest before I can look into this.

Some initial investigations indicate there was some kind of injection attack (no research yet) which might explain the cpu spike. Maybe a gallery update/upgrade would be enough. We'll see.

I'll post again when I have something to report.


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Jul 16, 2010 3:22 pm
Posts: 367
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Wed Aug 28, 2013 12:16 am 
Thanks. I look forward to finding out what the verdict is. Moving all my sub-forum stuff is going to suck, but if it keeps the site going, it is worth it.

_________________
Image


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Sun Jan 13, 2008 5:52 pm
Posts: 871
Location: Lakewood, CO North America
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Wed Aug 28, 2013 9:39 am 
Thanks for following up!

jh

_________________
Yes, I'm a chiropractor. Gamer fitness at Hafner Chiropractic in Lakewood, CO: http://www.HafnerChiropractic.com


Top
 Profile  
 
User avatar  Offline
Site Admin
 
Joined: Mon Jun 12, 2006 12:20 pm
Posts: 12064
Location: Tampa, FL
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 31, 2013 9:38 am 
I just got another trouble ticket from HostGator that the cpu usage spiked up to 3380 seconds during the "past hour" (ie. 94% cpu).

I'm not sure I believe them though, because the load average was 0.10, 0.02, and 0.01. If you're a Unix person you know those are 1 minute, 5 minute, and 15 minute load averages. So how could there be 3380 seconds of cpu in an hour that has 3600 seconds, and yet the last 900 seconds has a load average of 0.01??

Unfortunately, my email addy is not the billing addy for the account so they send the ticket to Trevor who then has to forward it to me. We're going to change that. Today. :)


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Nov 10, 2009 6:11 pm
Posts: 7680
Location: Bay Area
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sat Aug 31, 2013 9:58 am 
I forget what time it was, but there was some wonkiness going on this morning for a couple of minute. Pages were loading without the templates, but were fixed the next minute. Looking at my history, I think it happened just before 6:40 PST.

_________________
Downloads:


Top
 Profile  
 
User avatar  Offline
Site Admin
 
Joined: Mon Jun 12, 2006 12:20 pm
Posts: 12064
Location: Tampa, FL
 Post subject: Re: Site Outage from 8/23-8/24
PostPosted: Sun Sep 01, 2013 10:04 pm 
Another big outage. Same problem.

It seems that the gallery.rptools.net/main.php page was able to be accessed even though I put the rest of the gallery into maintenance mode. Apparently "maintenance mode" is so people can't add new stuff, but existing stuff can still be accessed.

I saw a ps(1) listing with about a half-dozen accesses to the gallery at once, each with 20-35% of the cpu (yes, I know that's more than 100%). I'm going to try upgrading the gallery to 3.0.9 (v3 is considered "post-beta" and we have v2.3 which is supposed to be the latest "stable" release). The current package was installed manually and I'm going to use the cPanel interface so it can keep it up to date for me. (I should do the same for the phpBB stuff, when I get a chance.)

I've got some work deadlines for Wednesday, so it'll likely be the end of the week before I can work on this unless I find myself with a few minutes. I did have an hour or so to tinker with MTL (the new MapToolLauncher) so who knows, maybe I'll get some more time...


Top
 Profile  
 
Display posts from previous:  Sort by  
Reply to topic  [ 32 posts ]  Go to page 1, 2, 3  Next

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:

Who is online

In total there are 2 users online :: 0 registered, 0 hidden and 2 guests (based on users active over the past 5 minutes)
Most users ever online was 243 on Sun Nov 04, 2012 6:14 am

Users browsing this forum: No registered users and 2 guests





Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

Style based on Andreas08 by Andreas Viklund

Style by Elizabeth Shulman