Welcome to MilkyWay@home

Server Trouble

Message boards : News : Server Trouble
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · Next

AuthorMessage
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73343 - Posted: 6 May 2022, 10:14:51 UTC - in response to Message 73332.  

There's no 'old stuff' as we brought our tv's with us when we sold our old home to move into this new one, the stuff was in storage for a year in between but it seems ll is working at least so far. Yes I have 4 tv's, actually I have a 5th upstairs that will be just for streaming at least right now. I have a great room, master bedroom and 2 guest rooms on the main level and then a room the size of my 2 car garage upstairs that is an overflow bedroom and will be for the grandkids as the grow up, they are 3 and 6 right now. My whole house is about 1800 square feet and mostly fills up the lot I'm on so there's not alot of grass etc. The key here is location, location, location.


Not enormous then, I was expecting bigger. Mine is 500 sq ft plus 134 sq ft for the extension. I only paid $61K for it in 2000. It's worth three times that now.


Nope not enormous at all, in fact some things are downsized from my last home and I now have lots of pc stuff that has no place for it right now, but I will figure out as time goes by. I put the contract down in December 2020 but then they figured out the house wouldn't fit on the lot so I had to do a new contract in March 2021 but the same home I am in right now is selling for $150K more than I paid for it, and I have some upgrades that new homeowners won't get. The Builder has you sooo locked into buying a home once you sign the contract that they also put in there that they absorb any price increases or decreases. So when the price of EVERYTHING soared I was still locked in at the pre-pandemic pricing structure. ie the roof plywood has insulation built in at the factory, pre-covid they were $12 a sheet post-covid they are now over $40 a sheet!! Someone stole a whole pallet load from another job site and the builder lost $35K for a single pallet of the stuff!!
ID: 73343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73347 - Posted: 6 May 2022, 10:23:09 UTC - in response to Message 73343.  

Nope not enormous at all, in fact some things are downsized from my last home and I now have lots of pc stuff that has no place for it right now, but I will figure out as time goes by. I put the contract down in December 2020 but then they figured out the house wouldn't fit on the lot so I had to do a new contract in March 2021 but the same home I am in right now is selling for $150K more than I paid for it, and I have some upgrades that new homeowners won't get. The Builder has you sooo locked into buying a home once you sign the contract that they also put in there that they absorb any price increases or decreases. So when the price of EVERYTHING soared I was still locked in at the pre-pandemic pricing structure. ie the roof plywood has insulation built in at the factory, pre-covid they were $12 a sheet post-covid they are now over $40 a sheet!! Someone stole a whole pallet load from another job site and the builder lost $35K for a single pallet of the stuff!!
ROFL! Sounds like you did well out of that. I'm still on pre-Covid electricity prices.
ID: 73347 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73358 - Posted: 7 May 2022, 2:03:50 UTC - in response to Message 73347.  

Nope not enormous at all, in fact some things are downsized from my last home and I now have lots of pc stuff that has no place for it right now, but I will figure out as time goes by. I put the contract down in December 2020 but then they figured out the house wouldn't fit on the lot so I had to do a new contract in March 2021 but the same home I am in right now is selling for $150K more than I paid for it, and I have some upgrades that new homeowners won't get. The Builder has you sooo locked into buying a home once you sign the contract that they also put in there that they absorb any price increases or decreases. So when the price of EVERYTHING soared I was still locked in at the pre-pandemic pricing structure. ie the roof plywood has insulation built in at the factory, pre-covid they were $12 a sheet post-covid they are now over $40 a sheet!! Someone stole a whole pallet load from another job site and the builder lost $35K for a single pallet of the stuff!!


ROFL! Sounds like you did well out of that. I'm still on pre-Covid electricity prices.


We did VERY well!!! We sold our 4 year old home, literally around the corner from our new home, bought a brand new home that has a LARGE room over the garage with a bathroom and NO pull down steps to get to it and STILL kept money in our pocket!! The people that bought our old house are very happy that it has also increased in value and they are loving it.
ID: 73358 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 73646 - Posted: 22 May 2022, 19:28:22 UTC

I took the WU generators offline after the recent explosion in tasks ready to send. No clue what happened there, but I'm monitoring it. Server appears to be processing things fine, it's just that there's a lot there to work through.
ID: 73646 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73650 - Posted: 22 May 2022, 20:05:37 UTC - in response to Message 73646.  
Last modified: 22 May 2022, 20:06:50 UTC

I took the WU generators offline after the recent explosion in tasks ready to send. No clue what happened there, but I'm monitoring it. Server appears to be processing things fine, it's just that there's a lot there to work through.
Having trouble getting GPU units, either getting none or a tenth of normal.

For examples:
Scheduler request completed: got 0 new tasks
Scheduler request completed: got 67 new tasks
Project is temporarily shut down for maintenance
Scheduler request failed: Timeout was reached

If the server would wake up we could get through the buffer fairly quickly.
ID: 73650 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alanb1951

Send message
Joined: 16 Mar 10
Posts: 213
Credit: 108,848,024
RAC: 27,895
Message 73652 - Posted: 22 May 2022, 23:34:24 UTC - in response to Message 73646.  
Last modified: 22 May 2022, 23:49:56 UTC

I took the WU generators offline after the recent explosion in tasks ready to send. No clue what happened there, but I'm monitoring it. Server appears to be processing things fine, it's just that there's a lot there to work through.
Tom,

I think will happen when the supply of tasks waiting to be sent runs out at the same time as there is a transitioner backlog.

If I'm looking at the right work unit generator code, there's a comment at the end of the main loop which I'll quote below...
	// Now sleep for a few seconds to let the transitioner
	// create instances for the jobs we just created.
	// Otherwise we could end up creating an excess of jobs.
	sleep(SLEEP_TIME);

Given that the test for unsent results can only see results that have passed through the transitioner, it's fairly easy to see how that can run wild if there's a substantial transitioner backlog!

Now, it did run out of Separation work units to send round about the time the run-away started, and there has been a substantial transitioner backlog for a couple of days now :-( -- a perfect storm, unfortunately...

And if you're using the TAO generator is it possible that a big enough transitioner backlog may negatively impact assimilation and result in the generator being unable to produce new work for a while?

With the constant appeals for more GPU work, I fear that there could be repeats of this sort of incident in future -- the best avoidance is probably to make sure the transitioner never gets too far behind; if transitioner backlogs become common it may be worth considering running a pair of transitioners, one for odd-numbered work units and one for even-numbered work units, as per the BOINC documentation...

Cheers - Al.

P.S. I suspect issues like this are why the BOINC site seems to think that a transitioner backlog of several minutes is too much :-)

[Edited to add the remark about the TAO generator.]
ID: 73652 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73664 - Posted: 23 May 2022, 14:05:37 UTC - in response to Message 73652.  

There must either be a way around this, or this is a severe bug in Boinc, and you need to speak to them in Github. When something is backlogged, it should not create another backlog!
ID: 73664 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73665 - Posted: 23 May 2022, 14:13:38 UTC

Whatever is causing the validation queue (CPU speed?) needs upgraded. As I've said before, many of us are willing to throw money at the project. Just ask.
ID: 73665 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 73667 - Posted: 23 May 2022, 15:41:07 UTC - in response to Message 73652.  

Thanks Alan, I'll look into that.

I think (hope) we are transitioning to the MW@h administration being done by a couple of new graduate students (who work on MW@h full time) rather than me. This process should start over the summer. So I think we will be able to spend some time digging into these problems rather than me pushing them off because I have a pile of other work that needs to get done.
ID: 73667 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73668 - Posted: 23 May 2022, 16:15:15 UTC - in response to Message 73667.  

What happened to Dylan? I thought he was going to have a play with the server?
ID: 73668 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Spatzthecat

Send message
Joined: 1 Dec 10
Posts: 82
Credit: 15,452,009,012
RAC: 0
Message 73695 - Posted: 25 May 2022, 19:11:26 UTC - in response to Message 73667.  

Hi Tom,
That is good news, as you have been the main stay for some time, much appreciated.

The project status suggests only 150K Separation Units left, will they automatically replenish or will there be a need for intervention?
ID: 73695 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73696 - Posted: 25 May 2022, 19:33:22 UTC - in response to Message 73695.  

Hi Tom,
That is good news, as you have been the main stay for some time, much appreciated.

The project status suggests only 150K Separation Units left, will they automatically replenish or will there be a need for intervention?
They had better keep going, I just resurrected a 7990! Got it cheap on Ebay because "customer return, fans don't spin, makes a loud noise". Except the fans did spin and it didn't make a loud noise. It did run at 102C on one end though (dual chip), which I fixed with some heatsink gunk. That's me doing a task every 4 seconds now, and it's getting very warm in here.
ID: 73696 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Spatzthecat

Send message
Joined: 1 Dec 10
Posts: 82
Credit: 15,452,009,012
RAC: 0
Message 73697 - Posted: 25 May 2022, 19:54:26 UTC - in response to Message 73665.  

Whatever is causing the validation queue (CPU speed?) needs upgraded. As I've said before, many of us are willing to throw money at the project. Just ask.


Ditto.
ID: 73697 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73698 - Posted: 25 May 2022, 20:04:39 UTC
Last modified: 25 May 2022, 20:05:20 UTC

Strangely, the validation queue is now not increasing or decreasing.

I'd love a set of real time graphs of server disk/cpu/network usage.
ID: 73698 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Spatzthecat

Send message
Joined: 1 Dec 10
Posts: 82
Credit: 15,452,009,012
RAC: 0
Message 73699 - Posted: 26 May 2022, 0:37:44 UTC

No Separation Units, Intervention is the answer to my earlier question.
ID: 73699 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73700 - Posted: 26 May 2022, 3:02:49 UTC - in response to Message 73698.  

Strangely, the validation queue is now not increasing or decreasing.

I'd love a set of real time graphs of server disk/cpu/network usage.


I think BoincStats has some close to that but I don't know how to find them
ID: 73700 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bill F
Avatar

Send message
Joined: 4 Jul 09
Posts: 99
Credit: 17,423,831
RAC: 2,069
Message 73704 - Posted: 26 May 2022, 11:06:28 UTC - in response to Message 73700.  

Strangely, the validation queue is now not increasing or decreasing.

I'd love a set of real time graphs of server disk/cpu/network usage.


I think BoincStats has some close to that but I don't know how to find them


Try this .... https://grafana.kiska.pw/d/boinc/boinc?orgId=1&var-project=milkyway@home&chunkNotFound=&refresh=1m&theme=dark
In October of 1969 I took an oath to support and defend the Constitution of the United States against all enemies, foreign and domestic;
There was no expiration date.


ID: 73704 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73705 - Posted: 26 May 2022, 12:42:30 UTC - in response to Message 73704.  

Strangely, the validation queue is now not increasing or decreasing.

I'd love a set of real time graphs of server disk/cpu/network usage.


I think BoincStats has some close to that but I don't know how to find them


Try this .... https://grafana.kiska.pw/d/boinc/boinc?orgId=1&var-project=milkyway@home&chunkNotFound=&refresh=1m&theme=dark


Now that's cool, Thanks!!
ID: 73705 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 73707 - Posted: 26 May 2022, 14:53:34 UTC

I've brought the Separation WU generator back online now that we've cleared through the huge excess of WUs. Still keeping the Nbody generator offline until we work through that excess.
ID: 73707 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 73708 - Posted: 26 May 2022, 16:22:24 UTC

I just got drowned by hundreds of tasks coming through. Working brilliantly here. Seemed to be a slight slowdown while the server coped with everyone demanding tasks at once, but it's caught up again.
ID: 73708 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · Next

Message boards : News : Server Trouble

©2024 Astroinformatics Group