Welcome to MilkyWay@home

Database troubles

Message boards : Number crunching : Database troubles
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Dave Przybylo
Avatar

Send message
Joined: 5 Feb 08
Posts: 236
Credit: 49,648
RAC: 0
Message 10220 - Posted: 9 Feb 2009, 20:35:02 UTC
Last modified: 9 Feb 2009, 20:35:15 UTC

We've had numerous database problems within the past few days. The one we just fixed minutes ago was innodb log files getting too large due to debugging of the original problems. Problems on top of problems. Hopefully we're ok now.
Dave Przybylo
MilkyWay@home Developer
Department of Computer Science
Rensselaer Polytechnic Institute
ID: 10220 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Phil
Avatar

Send message
Joined: 13 Feb 08
Posts: 1124
Credit: 46,740
RAC: 0
Message 10222 - Posted: 9 Feb 2009, 20:37:11 UTC - in response to Message 10220.  

We've had numerous database problems within the past few days. The one we just fixed minutes ago was innodb log files getting too large due to debugging of the original problems. Problems on top of problems. Hopefully we're ok now.


Sorry you guys are having such troubles.
ID: 10222 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 10223 - Posted: 9 Feb 2009, 20:38:49 UTC

Thanks for fixing it that quickly.... phew! ;-)

Lovely greetings, Cori
ID: 10223 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevint
Avatar

Send message
Joined: 22 Nov 07
Posts: 285
Credit: 1,076,786,368
RAC: 0
Message 10224 - Posted: 9 Feb 2009, 20:39:38 UTC
Last modified: 9 Feb 2009, 20:40:33 UTC

Good news.

And think of these as great learning experiences, at least next time you will know how to fix these issues quicker.

I knew there was a reason that I did not start up a BOINC project -

Thanks for working so hard on keeping this up, and communicating with us on the problems.


Still getting connect errors :)

2/9/2009 13:39:28|Milkyway@home|Message from server: Server error: feeder not running
2/9/2009 13:39:28|Milkyway@home|Deferring communication for 1 hr 0 min 0 sec
2/9/2009 13:39:28|Milkyway@home|Reason: project is down
.
ID: 10224 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caspr
Avatar

Send message
Joined: 22 Mar 08
Posts: 90
Credit: 501,728
RAC: 0
Message 10225 - Posted: 9 Feb 2009, 20:40:40 UTC

That word "hopefully" might have saved you this time! Otherwise it seems like you're jinxing yourself!! ;o)
A clear conscience is usually the sign of a bad memory



ID: 10225 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Purple Rabbit
Avatar

Send message
Joined: 9 Nov 08
Posts: 44
Credit: 128,043,914
RAC: 0
Message 10226 - Posted: 9 Feb 2009, 20:49:08 UTC

These things are like peeling an Onion. There's always more...sigh

I'm sure that you'll finally get to the end tho.
ID: 10226 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 2
Message 10227 - Posted: 9 Feb 2009, 20:49:13 UTC - in response to Message 10220.  

Hmm:


2/9/2009 1:46:21 PM|Milkyway@home|Sending scheduler request: Requested by user. Requesting 0 seconds of work, reporting 2 completed tasks
2/9/2009 1:46:26 PM|Milkyway@home|Scheduler request completed: got 0 new tasks
2/9/2009 1:46:26 PM|Milkyway@home|Message from server: Project is temporarily shut down for maintenance


We've had numerous database problems within the past few days. The one we just fixed minutes ago was innodb log files getting too large due to debugging of the original problems. Problems on top of problems. Hopefully we're ok now.


ID: 10227 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Phil
Avatar

Send message
Joined: 13 Feb 08
Posts: 1124
Credit: 46,740
RAC: 0
Message 10229 - Posted: 9 Feb 2009, 20:53:00 UTC - in response to Message 10227.  

Hmm:


2/9/2009 1:46:21 PM|Milkyway@home|Sending scheduler request: Requested by user. Requesting 0 seconds of work, reporting 2 completed tasks
2/9/2009 1:46:26 PM|Milkyway@home|Scheduler request completed: got 0 new tasks
2/9/2009 1:46:26 PM|Milkyway@home|Message from server: Project is temporarily shut down for maintenance




Yes well, check the status page.
ID: 10229 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 10230 - Posted: 9 Feb 2009, 20:53:48 UTC - in response to Message 10227.  

Dave didn't start up the services :P It should be running now.
ID: 10230 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 2
Message 10233 - Posted: 9 Feb 2009, 21:21:05 UTC - in response to Message 10229.  

Yes I did -- it's running now.



Yes well, check the status page.


ID: 10233 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 2
Message 10234 - Posted: 9 Feb 2009, 21:21:51 UTC - in response to Message 10230.  

It is -- so did you ask him it he plugged it in? <g>


Dave didn't start up the services :P It should be running now.


ID: 10234 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 10236 - Posted: 9 Feb 2009, 21:29:40 UTC - in response to Message 10234.  

He did a good job fixing the database while i was asleep though :D

I come back, see some messages "oh @(#*&$ it broke again" then check my stuff on milkyway and see it's back working again :) that was nice.
ID: 10236 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Phil
Avatar

Send message
Joined: 13 Feb 08
Posts: 1124
Credit: 46,740
RAC: 0
Message 10237 - Posted: 9 Feb 2009, 21:33:29 UTC - in response to Message 10236.  
Last modified: 9 Feb 2009, 21:34:06 UTC

He did a good job fixing the database while i was asleep though :D

I come back, see some messages "oh @(#*&$ it broke again" then check my stuff on milkyway and see it's back working again :) that was nice.


Makes up for working all Saturday, that does!
ID: 10237 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Debs

Send message
Joined: 15 Jan 09
Posts: 169
Credit: 6,734,481
RAC: 0
Message 10244 - Posted: 9 Feb 2009, 23:27:49 UTC - in response to Message 10236.  
Last modified: 9 Feb 2009, 23:29:01 UTC

He did a good job fixing the database while i was asleep though :D


Sleep? You got time to SLEEP!?! That's just not good enough! <j/k>

Seriously though, you're both doing a great job keeping on top of this, and I hope you get a trouble-free week this week :)
ID: 10244 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Odd-Rod

Send message
Joined: 7 Sep 07
Posts: 444
Credit: 5,712,523
RAC: 3
Message 10255 - Posted: 10 Feb 2009, 3:33:26 UTC - in response to Message 10220.  

We've had numerous database problems within the past few days.


Really? ;)

But, seriously, well done and thanks for all the hard work sorting it out!

ID: 10255 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 10256 - Posted: 10 Feb 2009, 3:50:13 UTC

It would seem there is still problems as the site won't load at times & these:

2/9/2009 9:15:36 PM|Milkyway@home|Message from server: No work sent
2/9/2009 9:15:36 PM|Milkyway@home|Message from server: (reached per-CPU limit of 12 tasks)
2/9/2009 10:45:27 PM||Project communication failed: attempting access to reference site
2/9/2009 10:45:28 PM||Access to reference site succeeded - project servers may be temporarily down.
2/9/2009 10:45:31 PM|Milkyway@home|Scheduler request failed: Couldn't connect to server

It comes and goes, 1 min nothing, the next it works.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 10256 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jedirock
Avatar

Send message
Joined: 8 Nov 08
Posts: 178
Credit: 6,140,854
RAC: 0
Message 10257 - Posted: 10 Feb 2009, 3:53:58 UTC - in response to Message 10256.  

It would seem there is still problems as the site won't load at times & these:

2/9/2009 9:15:36 PM|Milkyway@home|Message from server: No work sent
2/9/2009 9:15:36 PM|Milkyway@home|Message from server: (reached per-CPU limit of 12 tasks)
2/9/2009 10:45:27 PM||Project communication failed: attempting access to reference site
2/9/2009 10:45:28 PM||Access to reference site succeeded - project servers may be temporarily down.
2/9/2009 10:45:31 PM|Milkyway@home|Scheduler request failed: Couldn't connect to server

It comes and goes, 1 min nothing, the next it works.

Yeah, I've noticed it too. That, and Firefox sometimes times out on a refresh request. I'm guessing the server's still under some heavy load.
ID: 10257 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John Clark

Send message
Joined: 4 Oct 08
Posts: 1734
Credit: 64,228,409
RAC: 0
Message 10262 - Posted: 10 Feb 2009, 8:52:14 UTC

Very true
The site can be loaded quickly some times, and others it times out completely. It seems slow ATM.

I guess jedirock is correct, from comments Travis posted when the server/DB crash was in progress but before the servers were shutdown.
ID: 10262 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Database troubles

©2024 Astroinformatics Group