Welcome to MilkyWay@home

new workunit queue size (6)

Message boards : Number crunching : new workunit queue size (6)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

AuthorMessage
Profile Debs

Send message
Joined: 15 Jan 09
Posts: 169
Credit: 6,734,481
RAC: 0
Message 13315 - Posted: 28 Feb 2009, 14:09:18 UTC - in response to Message 13215.  

I don't know how many GPUs are already in use on this project, but with more likely to be used soon I would think it a good idea to increase the size of each wu, so there will not need to be so many requests.


I'd rather have a real fix for the problem, because if we do this the same problem will just show up again when we get more users and starting seeing the same amount of workunit requests...


Larger work units would be a real fix, and would give you more time to sort out the problem with how slowly work is being released at times.
ID: 13315 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 13317 - Posted: 28 Feb 2009, 14:11:32 UTC - in response to Message 13315.  

I don't know how many GPUs are already in use on this project, but with more likely to be used soon I would think it a good idea to increase the size of each wu, so there will not need to be so many requests.


I'd rather have a real fix for the problem, because if we do this the same problem will just show up again when we get more users and starting seeing the same amount of workunit requests...


Larger work units would be a real fix, and would give you more time to sort out the problem with how slowly work is being released at times.


I brought this up a couple weeks ago and Travis said they contain enough info and the 'group' doesn't want to increase the size any more.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 13317 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Debs

Send message
Joined: 15 Jan 09
Posts: 169
Credit: 6,734,481
RAC: 0
Message 13319 - Posted: 28 Feb 2009, 14:15:41 UTC - in response to Message 13317.  

I don't know how many GPUs are already in use on this project, but with more likely to be used soon I would think it a good idea to increase the size of each wu, so there will not need to be so many requests.


I'd rather have a real fix for the problem, because if we do this the same problem will just show up again when we get more users and starting seeing the same amount of workunit requests...


Larger work units would be a real fix, and would give you more time to sort out the problem with how slowly work is being released at times.


I brought this up a couple weeks ago and Travis said they contain enough info and the 'group' doesn't want to increase the size any more.


LOL, I'd love to know what majority group thinks everything is fine as it is :)
ID: 13319 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 13320 - Posted: 28 Feb 2009, 14:22:22 UTC - in response to Message 13319.  



I brought this up a couple weeks ago and Travis said they contain enough info and the 'group' doesn't want to increase the size any more.


LOL, I'd love to know what majority group thinks everything is fine as it is :)


Whoever he is supposed to see about the project. Probably the same people who don't think a second server is needed. :P
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 13320 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 13321 - Posted: 28 Feb 2009, 14:26:18 UTC - in response to Message 13301.  


I looked randomly at 10 of the top 100 computer’s RAC – they are all ATI equipped.

I expect the top crunchers amassed most of their credits before the ATIs were available for optimized crunching. ATI GPU crunching is relatively quite recent and I expect people want a stable period of crunching with the ATIs before saying how they are doing with them.


ID: 13321 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 13328 - Posted: 28 Feb 2009, 15:31:39 UTC - in response to Message 13310.  
Last modified: 28 Feb 2009, 15:32:42 UTC



Wait a second...

Resetting the project as a workaround to get more work might be counter productive. The reason is that even if you already have the third party apps to install afterwards, your host is going to have to re-download all the input files it needs that got dumped from the reset. If running stock, it will have to download the app again too.

This is probably not going to do much to help the overall situation. ;-)

Alinator


No, Resetting the Project won't get rid of any Optimized App's, Third Party App's I don't know about but the Optimized ones don't get taken out when Resetting. Yes you'll have to re-download the input files & if you have Dial-Up it could take some time or pose a problem.

I do it all the time but I have Cable so it's no big deal for me to re-download the input files, it only takes a few seconds for me to do that. It works for me but if somebody has a problem with it then don't do it, more work for me then ... :)


Agreed, not all CC's will dump the whole project directory on a reset. However, all will dump the stock app on one.

However, that wasn't the real point. The new GPU apps are bandwidth and work sponges from the Projects POV.

Therefore, 'casual' resetting forces those hosts to pull all the input files and a whole new set of work from empty, with the attendant DB and and bandwidth load that goes along with that. Given that the project is having trouble keeping up as it is, anything that increase that even more only makes things worse for the big picture. Also, there is no guarantee that the scheduler queue hasn't been sucked dry just as the host gets reset, so the whole process would have been done for no net gain in that case.

Alinator
ID: 13328 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Neal Chantrill
Avatar

Send message
Joined: 17 Jan 09
Posts: 98
Credit: 72,182,367
RAC: 0
Message 13331 - Posted: 28 Feb 2009, 15:40:50 UTC

I know have work on all computers and including the GPU where I was getting practically nothing.
ID: 13331 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 13334 - Posted: 28 Feb 2009, 15:47:00 UTC - in response to Message 13331.  

I know have work on all computers and including the GPU where I was getting practically nothing.


Server-side, it looks like the change helped work availability quite a bit.
ID: 13334 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 13336 - Posted: 28 Feb 2009, 15:53:48 UTC - in response to Message 13334.  

I know have work on all computers and including the GPU where I was getting practically nothing.


Server-side, it looks like the change helped work availability quite a bit.


LOL...

Well, we had our three steps backwards over the last 36 hours or so.

So I guess that would be about 2 1/2 forward now? ;-)

Alinator
ID: 13336 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JAMC

Send message
Joined: 9 Sep 08
Posts: 96
Credit: 336,443,946
RAC: 0
Message 13341 - Posted: 28 Feb 2009, 16:28:40 UTC - in response to Message 13334.  

I know have work on all computers and including the GPU where I was getting practically nothing.


Server-side, it looks like the change helped work availability quite a bit.



The only problem- and it's a big one- is when you have 'got 0 new tasks' multiple times in a row, which does still happen, and run dry and the reconnect time goes to 2 or 3 hours and you have nothing to crunch until then... any way to change/fix(?) that?
ID: 13341 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 13343 - Posted: 28 Feb 2009, 16:36:01 UTC - in response to Message 13341.  

I know have work on all computers and including the GPU where I was getting practically nothing.


Server-side, it looks like the change helped work availability quite a bit.



The only problem- and it's a big one- is when you have 'got 0 new tasks' multiple times in a row, which does still happen, and run dry and the reconnect time goes to 2 or 3 hours and you have nothing to crunch until then... any way to change/fix(?) that?


It depends on what mean by 'fix'.

If you are talking about MW being able to feed work to the scheduler faster, then no, not until Monday at the earliest.

If you're talking about not running dry, then you have two choices:

1.) Sit at the console all the time and pound on the update button when you are out.

2.) Run a backup project and just ride out these difficulties until a satisfactory backend fix can be implemented.

Alinator


ID: 13343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 13345 - Posted: 28 Feb 2009, 16:56:36 UTC - in response to Message 13343.  



The only problem- and it's a big one- is when you have 'got 0 new tasks' multiple times in a row, which does still happen, and run dry and the reconnect time goes to 2 or 3 hours and you have nothing to crunch until then... any way to change/fix(?) that?


It depends on what mean by 'fix'.

If you are talking about MW being able to feed work to the scheduler faster, then no, not until Monday at the earliest.

If you're talking about not running dry, then you have two choices:

1.) Sit at the console all the time and pound on the update button when you are out.

2.) Run a backup project and just ride out these difficulties until a satisfactory backend fix can be implemented.

Alinator


I think he was talking about the 3 hour delay.

Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 13345 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brickhead
Avatar

Send message
Joined: 20 Mar 08
Posts: 108
Credit: 2,607,924,860
RAC: 0
Message 13355 - Posted: 28 Feb 2009, 17:51:58 UTC - in response to Message 13265.  

So far, I've seen *none* of the 0 responses

I spoke too soon. Quite a few 0 responses hidden in the logs, and running dry on one occasion. But still a significant improvement over both the previous per-core quotas tried, as far as my crunchers are concerned.
ID: 13355 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 13356 - Posted: 28 Feb 2009, 17:58:28 UTC - in response to Message 13355.  

So far, I've seen *none* of the 0 responses

I spoke too soon. Quite a few 0 responses hidden in the logs, and running dry on one occasion. But still a significant improvement over both the previous per-core quotas tried, as far as my crunchers are concerned.


Should be even better now that the workunits should take around twice as long to crunch.
ID: 13356 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 13360 - Posted: 28 Feb 2009, 18:13:33 UTC - in response to Message 13356.  


Should be even better now that the workunits should take around twice as long to crunch.

Huh. Making us work twice as slow eh? :p


ID: 13360 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JAMC

Send message
Joined: 9 Sep 08
Posts: 96
Credit: 336,443,946
RAC: 0
Message 13366 - Posted: 28 Feb 2009, 18:37:22 UTC - in response to Message 13345.  



The only problem- and it's a big one- is when you have 'got 0 new tasks' multiple times in a row, which does still happen, and run dry and the reconnect time goes to 2 or 3 hours and you have nothing to crunch until then... any way to change/fix(?) that?


It depends on what mean by 'fix'.

If you are talking about MW being able to feed work to the scheduler faster, then no, not until Monday at the earliest.

If you're talking about not running dry, then you have two choices:

1.) Sit at the console all the time and pound on the update button when you are out.

2.) Run a backup project and just ride out these difficulties until a satisfactory backend fix can be implemented.

Alinator


I think he was talking about the 3 hour delay.


'If you are talking about MW being able to feed work to the scheduler faster...'
I'm not.

'I think he was talking about the 3 hour delay.'
I was.
ID: 13366 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Debs

Send message
Joined: 15 Jan 09
Posts: 169
Credit: 6,734,481
RAC: 0
Message 13367 - Posted: 28 Feb 2009, 18:39:44 UTC

Good to see that the idea of increasing the size of a work unit was eventually taken on board. Making them smaller always seemed crazy when they are mostly completed so fast :)
ID: 13367 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Misfit
Avatar

Send message
Joined: 27 Aug 07
Posts: 915
Credit: 1,503,319
RAC: 0
Message 13380 - Posted: 28 Feb 2009, 19:17:39 UTC - in response to Message 13360.  

Should be even better now that the workunits should take around twice as long to crunch.

Huh. Making us work twice as slow eh? :p

Ooh 2 seconds then.
me@rescam.org
ID: 13380 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 13382 - Posted: 28 Feb 2009, 19:28:46 UTC - in response to Message 13380.  

Should be even better now that the workunits should take around twice as long to crunch.

Huh. Making us work twice as slow eh? :p

Ooh 2 seconds then.


Hey now, some of the GPU apps are taking a whole 3 seconds.
ID: 13382 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 13391 - Posted: 28 Feb 2009, 21:04:37 UTC - in response to Message 13382.  

Should be even better now that the workunits should take around twice as long to crunch.

Huh. Making us work twice as slow eh? :p

Ooh 2 seconds then.


Hey now, some of the GPU apps are taking a whole 3 seconds.


Need a faster app now.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 13391 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

Message boards : Number crunching : new workunit queue size (6)

©2024 Astroinformatics Group