Welcome to MilkyWay@home

20 workunit limit

Message boards : Number crunching : 20 workunit limit
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 8 · Next

AuthorMessage
Profile Philadelphia
Avatar

Send message
Joined: 9 Nov 07
Posts: 131
Credit: 180,454
RAC: 0
Message 2343 - Posted: 18 Mar 2008, 2:54:51 UTC

At the rate we're getting them, 1 WU is a good thing.
CLICK TO HELP BUILD
ID: 2343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Honza

Send message
Joined: 28 Aug 07
Posts: 31
Credit: 86,152,236
RAC: 0
Message 2344 - Posted: 18 Mar 2008, 6:45:57 UTC - in response to Message 2330.  
Last modified: 18 Mar 2008, 6:47:19 UTC

Perhaps wasting work, or even the lax deadlines aren't that much of a problem. The server could send out WUs in progress again to new users if it hasn't gotten results after a certain amount of time. The result that arrives later will simply be ignored, but credit still granted. Since computers are fickle and the WUs are short, this could mean the difference between a few hours and a few days...

BOINC has feature to abort task that are no longer needed (quorum met for example). It is a server side feature, not client (like cancel those beyond deadline).

A smart way would be to feed fast hosts with more work a slow onces with less amount. A fast host is the one with low turn-around time, slow one is the one that sents results back late.
Note that slow one may a be quad that gets turned off or is overworked with other tasks meeting deadline. It is not only about FLOPs.

EDIT: What is a general turn-around time per Wu?
ID: 2344 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
johannes

Send message
Joined: 6 Sep 07
Posts: 1
Credit: 87,753
RAC: 0
Message 2346 - Posted: 18 Mar 2008, 7:26:51 UTC - in response to Message 2327.  

Maybe if each WU carried work from a number of different genetic seeds and ran for a while longer?

Seems like a good idea!
Would it be possible for the project to be split into several "strands" starting from different genetic seeds, and these strands be processed side by side so that progress within a strand is slower and the time-window within which a returned result is still meaningful/useful is thereby extended?
You could also distribute small batches of WUs with one WU coming from each strand.


ID: 2346 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Sailor

Send message
Joined: 10 Feb 08
Posts: 32
Credit: 13,227,191
RAC: 0
Message 2349 - Posted: 18 Mar 2008, 12:23:37 UTC - in response to Message 2312.  

If these WUs are time sensitive, then why do they have a 5 day deadline? The server has been out of work for nearly 24 hours now, and there are still over 5000 WUs out in the wind. Seems to me that if new work generation depends on the old results, the deadline should be shortened. Maybe try 12 hours and see how that goes, at least if the WU ends up on a duffer, it would time out and possibly get sent to a faster, more reliable host.




Fish


I couldnt agree more - nothing that bothers me more then when I have to read that im computing for the trash. If the project needs a shoter deadline then thats the way it has to be, I think most will agree here.
ID: 2349 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 2350 - Posted: 18 Mar 2008, 12:30:16 UTC
Last modified: 18 Mar 2008, 12:32:14 UTC

I think the deadlines were increased just because we had some longer server outages before. ;-)
When the servers went down for more than 12 hours it was no fun not being able to report finished results in time. *grin*


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.
Lovely greetings, Cori
ID: 2350 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Groundhog

Send message
Joined: 21 Nov 07
Posts: 6
Credit: 154,900
RAC: 0
Message 2354 - Posted: 18 Mar 2008, 13:11:33 UTC - in response to Message 2350.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog


ID: 2354 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 2361 - Posted: 18 Mar 2008, 18:11:49 UTC - in response to Message 2354.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog



Yup. But you'll always need at least one backup project when running out of WUs here. :-)
Lovely greetings, Cori
ID: 2361 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Philadelphia
Avatar

Send message
Joined: 9 Nov 07
Posts: 131
Credit: 180,454
RAC: 0
Message 2362 - Posted: 18 Mar 2008, 18:40:18 UTC - in response to Message 2361.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog



Yup. But you'll always need at least one backup project when running out of WUs here. :-)


I have several backup projects :)

CLICK TO HELP BUILD
ID: 2362 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 2363 - Posted: 18 Mar 2008, 18:57:26 UTC - in response to Message 2362.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog



Yup. But you'll always need at least one backup project when running out of WUs here. :-)


I have several backup projects :)

Me too. :-) That's why I thought a 12 hrs deadline might be a little tight. *grin*
Lovely greetings, Cori
ID: 2363 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Emanuel

Send message
Joined: 18 Nov 07
Posts: 280
Credit: 2,442,757
RAC: 0
Message 2364 - Posted: 18 Mar 2008, 19:17:53 UTC - in response to Message 2363.  

Me too. :-) That's why I thought a 12 hrs deadline might be a little tight. *grin*


It would be nice if you could tell Boinc to do a certain amount of WUs before switching to another project (unless it fails to get any, of course), rather than just relying on a time share and deadlines :) I don't suppose there's an advanced option, or an up-to-date custom build that allows you to do that, is there?
ID: 2364 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Philadelphia
Avatar

Send message
Joined: 9 Nov 07
Posts: 131
Credit: 180,454
RAC: 0
Message 2365 - Posted: 18 Mar 2008, 19:18:44 UTC - in response to Message 2363.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog



Yup. But you'll always need at least one backup project when running out of WUs here. :-)


I have several backup projects :)

Me too. :-) That's why I thought a 12 hrs deadline might be a little tight. *grin*


Very true.

Doesn't BOINC give WU's getting ready to expire priority over other WU's regardless of the project?

CLICK TO HELP BUILD
ID: 2365 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 2366 - Posted: 18 Mar 2008, 19:41:09 UTC - in response to Message 2365.  


EDIT: when the server is stable enough the deadlines can be decreased again of course, but 12 hours is really short if you participate in more than a few projects.


I agree on this, but then it is simple - for those with numerous projects in which they are participating simultaniously Milky Way simply is the wrong project. The same for slower machines.

Deadlines should be set to the need of the project's science, not the convienience of users.

Science first.

Greetings, Groundhog



Yup. But you'll always need at least one backup project when running out of WUs here. :-)


I have several backup projects :)

Me too. :-) That's why I thought a 12 hrs deadline might be a little tight. *grin*


Very true.

Doesn't BOINC give WU's getting ready to expire priority over other WU's regardless of the project?

It does. So-called "panic mode". *grin*
But it would be bad if BOINC would go into constant panic mode and only crunch Milkyways and you don't get your other WUs finished.

But I guess it's a bit early to figure out now what might be in some days/weeks. ;-)
Lovely greetings, Cori
ID: 2366 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Crunch3r
Volunteer developer
Avatar

Send message
Joined: 17 Feb 08
Posts: 363
Credit: 258,227,990
RAC: 0
Message 2367 - Posted: 18 Mar 2008, 20:08:09 UTC - in response to Message 2363.  


...That's why I thought a 12 hrs deadline might be a little tight.


Sorry i don't agree here.

It's all a personal preference if you'd like to attach to all available boinc projects on earth or wan't to crunch MW and leave only a few like 2 or 4 projects for backup.

It all depends on what priority you give to MW... i for myself am ready to crunch only MW and ABC+TSP as backup.

So, cut it down to 4h deadline... no just kidding 12h are fine :P







Join Support science! Joinc Team BOINC United now!
ID: 2367 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ChertseyAl
Avatar

Send message
Joined: 31 Aug 07
Posts: 66
Credit: 1,002,668
RAC: 0
Message 2368 - Posted: 18 Mar 2008, 20:28:20 UTC - in response to Message 2366.  

It does. So-called "panic mode". *grin*
But it would be bad if BOINC would go into constant panic mode and only crunch Milkyways and you don't get your other WUs finished.


Exactly the point I raised yesterday.

The current computing model doesn't really suit BOINC, it relies on rapid reporting.

Options:

1) Abandon BOINC and go stand-alone.

2) As 1 above, but farm out 'child' genetic threads to BOINC.

3) Make WUs a mix of parallel genetic seeds, increasing crunching time, but feeding more improved start points back into the matrix.

I'm actually thinking that option 1 might be best. My slow host is turning over WUs in 5 hours. This is too slow. I will probably set this host to NNT as I suspect more 'old' science is jsut that ... OLD.

Al.
ID: 2368 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Crunch3r
Volunteer developer
Avatar

Send message
Joined: 17 Feb 08
Posts: 363
Credit: 258,227,990
RAC: 0
Message 2369 - Posted: 18 Mar 2008, 20:35:41 UTC - in response to Message 2368.  

....
1) Abandon BOINC and go stand-alone.

.....

My slow host is turning over WUs in 5 hours. This is too slow. I will probably set this host to NNT as I suspect more 'old' science is jsut that ... OLD.

Al.


to 1) no ...why ? ....one could use a different boinc client that reports results immediately and single out that std. boinc clients... now how about that one ?

5h return time ? what's that a PII@300 MHz ? or do you mean the time that pases befor your boinc client contacts the server ?










Join Support science! Joinc Team BOINC United now!
ID: 2369 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Honza

Send message
Joined: 28 Aug 07
Posts: 31
Credit: 86,152,236
RAC: 0
Message 2370 - Posted: 18 Mar 2008, 20:56:29 UTC

My quads (stock speed) are doing 0.03 days of turn-around time. If there were no down times and 20 minutes limit for another scheduler connection, turn-around time would be ~8 minutes. That's it - 8 minutes or 0.006 days.

I've raised question of several islands in GECCO2008 paper accepted, so far without answer.
ID: 2370 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ChertseyAl
Avatar

Send message
Joined: 31 Aug 07
Posts: 66
Credit: 1,002,668
RAC: 0
Message 2372 - Posted: 18 Mar 2008, 21:49:33 UTC - in response to Message 2369.  

5h return time ? what's that a PII@300 MHz ? or do you mean the time that pases befor your boinc client contacts the server ?


Sorry, must have explained this on another thread that you missed.

1.8G machine, WU time 15m. 20WU max. Last WU in will take a minimum of 5h to get crunched and reported. FIFO and all that.

So every WU that machine crunches will take 5h from arrival to reporting. Agreed?

Solution: Set a buffer of 0.001 hours, so that only one WU is ever active. But that throttles the faster hosts. Also, given the 'reliability' of MW, that ain't a good strategy ;)

Now, a BOINC client that can limit the number of WUs per project (say 2 for MW), give them priority over all else, but respect the deadlines of backup projects (and we need BU projects when MW is still flaky) to avoid starving other projects would be good.

I'm really thinking that genetic models don't suit BOINC. BOINC is great for boring, grinding, tedious number-crunching, but evolving models won't work.

BTW, this has nothing to d0o with MW. MW is one of my fave projects. But, as such, I'd rather see it run for the science than for the user.

Al.
ID: 2372 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Jayargh
Avatar

Send message
Joined: 8 Oct 07
Posts: 289
Credit: 3,690,838
RAC: 0
Message 2373 - Posted: 18 Mar 2008, 22:05:35 UTC - in response to Message 2372.  

5h return time ? what's that a PII@300 MHz ? or do you mean the time that pases befor your boinc client contacts the server ?


Sorry, must have explained this on another thread that you missed.

1.8G machine, WU time 15m. 20WU max. Last WU in will take a minimum of 5h to get crunched and reported. FIFO and all that.

So every WU that machine crunches will take 5h from arrival to reporting. Agreed?

Solution: Set a buffer of 0.001 hours, so that only one WU is ever active. But that throttles the faster hosts. Also, given the 'reliability' of MW, that ain't a good strategy ;)

Now, a BOINC client that can limit the number of WUs per project (say 2 for MW), give them priority over all else, but respect the deadlines of backup projects (and we need BU projects when MW is still flaky) to avoid starving other projects would be good.

I'm really thinking that genetic models don't suit BOINC. BOINC is great for boring, grinding, tedious number-crunching, but evolving models won't work.

BTW, this has nothing to d0o with MW. MW is one of my fave projects. But, as such, I'd rather see it run for the science than for the user.

Al.


But Al ...all that being said and true ....Travis has also said we are beating the BlueGene supercomputer in both the amount of results returned and the short turnaround time so how can Boinc not be at least an adequate platform?

ID: 2373 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Cori
Avatar

Send message
Joined: 27 Aug 07
Posts: 647
Credit: 27,592,547
RAC: 0
Message 2374 - Posted: 18 Mar 2008, 22:29:38 UTC - in response to Message 2373.  
Last modified: 18 Mar 2008, 22:30:42 UTC

But Al ...all that being said and true ....Travis has also said we are beating the BlueGene supercomputer in both the amount of results returned and the short turnaround time so how can Boinc not be at least an adequate platform?

I hope that BOINC is an adequate platform! I would certainly miss Milkyway if it was no BOINC project anymore. ;-)

Btw: my turnaround time for 20 WUs is about 80 to 85 minutes on my new lappy.
So a 12 hrs deadline would be no problem if there were almost no outages and mostly always WUs available. That way I'd be happy to crunch along Milkyway exclusively on that box 24/7. *grin*
Lovely greetings, Cori
ID: 2374 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,515,987
RAC: 36,974
Message 2376 - Posted: 19 Mar 2008, 0:34:26 UTC
Last modified: 19 Mar 2008, 0:35:10 UTC

My turnaround is 50 Minutes for 20 WU's running 50/50 with another Project, less than 30 running 100% but then I invariably run out of work so I stay with 50/50 ... :)
ID: 2376 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 8 · Next

Message boards : Number crunching : 20 workunit limit

©2024 Astroinformatics Group