Message boards :
Number crunching :
8 Workunit limit
Message board moderation
Previous · 1 · 2 · 3
Author | Message |
---|---|
Send message Joined: 29 Aug 07 Posts: 486 Credit: 576,548,171 RAC: 0 |
I'm going to try and increase the max_wus_in_progress back to 20, and lower the deadline to a day for the workunits and see if this helps you guys out any. That (20) would be much better Travis than 8, the 1 Day Deadline doesn't bother me either although I'm sure theres going to be some understandable objection to it. With a 1 Day Deadline you either want to run the Project or you don't is the way I see it ... :) |
Send message Joined: 8 Oct 07 Posts: 289 Credit: 3,690,838 RAC: 0 |
I keep looking and really don't see any option for a per-core WU limit. Theres basically one option in the config file that is max_wus_in_progress which basically says how many workunits per machine are allowed. I've sent an email to Dave about what I can do about this. Travis,would it be possible to make that deadline 2 days? That way my hosts won't be running in high priority constantly....just on occasion,and internet service/server problems would be less of an issue. Right now a 372 can take 12hrs on a P4 ...so a 1 day deadline is cutting it pretty close.Also a 1 day deadline would effectively cut out all P3 types from running this project. |
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
I keep looking and really don't see any option for a per-core WU limit. Theres basically one option in the config file that is max_wus_in_progress which basically says how many workunits per machine are allowed. I've sent an email to Dave about what I can do about this. ok i've changed the deadline to 2 days. |
Send message Joined: 8 Oct 07 Posts: 289 Credit: 3,690,838 RAC: 0 |
I keep looking and really don't see any option for a per-core WU limit. Theres basically one option in the config file that is max_wus_in_progress which basically says how many workunits per machine are allowed. I've sent an email to Dave about what I can do about this. Thanks Travis :) I am sure you will still get complaints but this is now doable.;) |
Send message Joined: 31 Aug 07 Posts: 66 Credit: 1,002,668 RAC: 0 |
ok i've changed the deadline to 2 days. That makes things workable on my old P4's. But ... Do you know when you will settle down to one WU length? The mixture of 371/2/3 WUs that I'm getting makes it impossible for BOINC manager to know how much work to get. I end up with a 372 running near to completion and maybe 2 371s ready to go with estimated run times based on 372s - As a result it 'appears' that my 1 day cache is full when in reality it will be dry in 40 minutes :( It would be nice if you were only running singe WU lengths :) Al. |
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
ok i've changed the deadline to 2 days. I think that's up to Nate. It really depends what parts of the sky they're running their searches over. Different areas will have more or less stars and the background integral will have a larger or smaller area. I can't really say if we'll ever have all the WU lengths be the same again... |
Send message Joined: 28 Aug 07 Posts: 31 Credit: 86,152,236 RAC: 0 |
ok i've changed the deadline to 2 days.This is bad - it makes the project suitable only for hosts running 24/7. Imagine office boxes being turned off over the weekend - all WUs downloaded (and some partially computed) on Friday will be lost and wasted. Even those finished on Friday and not reported immediately will be lost. Sorry if I missed some part of discussion as I wasn't on discussion lately but that's my point. (I'm running some boxes 24/7 but for other 20 office boxes it is not a good choice.) BOINC Project specifications and hardware requirements |
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
ok i've changed the deadline to 2 days.This is bad - it makes the project suitable only for hosts running 24/7. Imagine office boxes being turned off over the weekend - all WUs downloaded (and some partially computed) on Friday will be lost and wasted. Even those finished on Friday and not reported immediately will be lost. Office boxes turned off over the weekend can just download new WUs when they're turned back on. If the ones in their queue aren't computed it really doesn't effect what we're doing. Really old WUs really don't have very much benefit to what we're doing and would just be wasting cycles, it's just the nature of the project. |
Send message Joined: 28 Aug 07 Posts: 31 Credit: 86,152,236 RAC: 0 |
Office boxes turned off over the weekend can just download new WUs when they're turned back on. If the ones in their queue aren't computed it really doesn't effect what we're doing. Really old WUs really don't have very much benefit to what we're doing and would just be wasting cycles, it's just the nature of the project. Thanks for quick answer. You are correct about those downloaded but not started. Only those partialy computed will be wasted...from users perspective. A 3 days deadline would overcome this problem - WUs started on Friday would have been finished on Monday. BOINC Project specifications and hardware requirements |
Send message Joined: 31 Aug 07 Posts: 66 Credit: 1,002,668 RAC: 0 |
Really old WUs really don't have very much benefit to what we're doing and would just be wasting cycles, it's just the nature of the project. Would it be possible to send cancellation messages for workunits that haven't begun crunching and aren't needed because they are 'stale'? That would of course rely on the hosts doing an update (at least with 5.x.x managers I guess). Are WUs that sneak in 1 second before the deadline actually useful? You imply that 1 day is 'good' for you, so are we just wasting resources once a WU is more than a day old? Maybe if you could guarantee (yeah, right!) that work was always available we could go for a 0.05 day cache and provided there were no really short WUs and that BOINC manager requested works frequently enough, work would be pushed through quickly and always be 'fresh'. I can't help but feel that this project doesn't really quite suit BOINC, as parameters have to be set to force the behaviour that you need. Maybe a standalone DC app would work better. Anyway, I have no axe to grind - I'm happy to crunch anything that useful to the project, and if my hardware isn't up to it I can crunch plenty of other projects. Or I can buy new hardware ;) Al. |
Send message Joined: 4 Oct 07 Posts: 43 Credit: 53,898 RAC: 0 |
I'm thinking that a good time to shoot for would be something that on average will be around the 373* series. However, I don't know if there will ever be a set length. As Travis brought up, it really depends on what part of the sky we're looking at and how complicated it is. I've been really busy the past couple weeks making the code more robust to deal with these more complicated pieces of the sky. The problem is that with each complication comes a significant increase in runtime... For example, if there is two pieces of tidal debris in the piece of sky we're looking at it increases the runtime by 75%, also if there happens to be a piece of the sky that has bad data or something that we need to ignore that also increases the runtime by some nontrivial amount (I know it seems backwards that if we need to ignore something that the runtime gets longer, but we have to calculate the effects of it). So as you can see, the runtime can ramp up very quickly. To keep the WUs similar lengths we'll need to be scaling the number of integral points by an amount that we think will compensate the other changes, but obviously it will not be an exact science. However, I think I can say that we'll do our best to keep them around the length of the 373* series, and that there should never be anything as long as the 372* series, no anything as short as the 371* series. Unfortunately, I don't think I can be more specific than this. ~Nate~ |
Send message Joined: 28 Aug 07 Posts: 31 Credit: 86,152,236 RAC: 0 |
Maybe if you could guarantee (yeah, right!) that work was always available we could go for a 0.05 day cache and provided there were no really short WUs and that BOINC manager requested works frequently enough, work would be pushed through quickly and always be 'fresh'. Report Results Immediately, a feature introduced in some older non-UCB BOINC cores like BOINC Studio core would solve the problem. It means once the result is finished and uploaded, it will trigger it's report. BOINC Project specifications and hardware requirements |
Send message Joined: 31 Aug 07 Posts: 66 Credit: 1,002,668 RAC: 0 |
ok i've changed the deadline to 2 days. Arrrggghhh! Now the new WUS are hijacking the ones I'm currently running because they kicked the host into panic mode :( It might have been better to wait until WUs with the old deadline had reported back before changing anything. Oh well. Not as bad as CosNob though ... ;) Al. |
Send message Joined: 18 Nov 07 Posts: 280 Credit: 2,442,757 RAC: 0 |
I'm thinking that a good time to shoot for would be something that on average will be around the 373* series. However, I don't know if there will ever be a set length. As Travis brought up, it really depends on what part of the sky we're looking at and how complicated it is. I've been really busy the past couple weeks making the code more robust to deal with these more complicated pieces of the sky. The problem is that with each complication comes a significant increase in runtime... For example, if there is two pieces of tidal debris in the piece of sky we're looking at it increases the runtime by 75%, also if there happens to be a piece of the sky that has bad data or something that we need to ignore that also increases the runtime by some nontrivial amount (I know it seems backwards that if we need to ignore something that the runtime gets longer, but we have to calculate the effects of it). Could you possibly auto-detect anomalously lengthy WUs in a batch and split them up into two smaller ones? |
©2024 Astroinformatics Group