Welcome to MilkyWay@home

rate of processing

Message boards : Number crunching : rate of processing
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 4934 - Posted: 21 Aug 2008, 22:42:42 UTC

As I mentioned in another thread, I attached my AMD and pulled down work. I have a significant concern that I won't make deadline on several of them due to my calculation of actual runtime. So, my question is:

Do tasks have a fairly linear progression or are there spots in the processing where the overall task progress will increase at a faster (or slower) rate?

I'm estimating about 10.5 hours for the task that I just started. A 3-day deadline will allow me 72 hours of processing. 20 x 10.5 = 230 hours.

I know DCF will take care of subsequent work requests, I just don't know if I have 10-13 tasks that I'll have no way of getting to before deadline...

Thanks,

Brian
ID: 4934 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Labbie
Avatar

Send message
Joined: 29 Aug 07
Posts: 327
Credit: 116,463,193
RAC: 0
Message 4937 - Posted: 21 Aug 2008, 23:35:25 UTC

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Calm Chaos Forum...Join Calm Chaos Now
ID: 4937 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 4938 - Posted: 21 Aug 2008, 23:43:22 UTC - in response to Message 4937.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Labbie's right Brian as far as I've seen so far ...
ID: 4938 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 4941 - Posted: 22 Aug 2008, 0:05:47 UTC - in response to Message 4937.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Well, that would get me down to around 5-6 hours per task...still leaving several tasks that I'll have no way of getting to...
ID: 4941 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 4942 - Posted: 22 Aug 2008, 0:09:18 UTC - in response to Message 4941.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Well, that would get me down to around 5-6 hours per task...still leaving several tasks that I'll have no way of getting to...


Not all Tasks are the same length, some are as short as 2 to 3 minutes, some 1.5 Hr's & some 3 Hr's on my Computers anyway, usually theres a slight mixture in the Downloads ...
ID: 4942 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Travis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
Message 4943 - Posted: 22 Aug 2008, 0:09:36 UTC - in response to Message 4941.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Well, that would get me down to around 5-6 hours per task...still leaving several tasks that I'll have no way of getting to...


It's not a huge deal if you cancel tasks with this project. The way our searches work if results don't get returned they keep on running.
ID: 4943 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile DoctorNow
Avatar

Send message
Joined: 28 Aug 07
Posts: 146
Credit: 10,692,107
RAC: 4,297
Message 4977 - Posted: 22 Aug 2008, 7:51:16 UTC - in response to Message 4942.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.



Well, that would get me down to around 5-6 hours per task...still leaving several tasks that I'll have no way of getting to...


Not all Tasks are the same length, some are as short as 2 to 3 minutes, some 1.5 Hr's & some 3 Hr's on my Computers anyway, usually theres a slight mixture in the Downloads ...

To be precise, look at the WU-names if you really want to know:
gs_371_* -> very short ones,
gs_373_* -> the middle ones,
gs_372_* -> the long ones.
Member of BOINC@Heidelberg and ATA!

My BOINCstats
ID: 4977 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 4978 - Posted: 22 Aug 2008, 11:42:18 UTC - in response to Message 4977.  


To be precise, look at the WU-names if you really want to know:
gs_371_* -> very short ones,
gs_373_* -> the middle ones,
gs_372_* -> the long ones.


Thanks...

I am noticing something very weird though. I have the project set to No New Tasks, but BOINC is still reporting completed work without me having to hit the Update button. I don't know if this is because BOINC is thinking it can't meet deadline or what is going on...

Also, as I speculated based on a quick check of a few systems where I could see their results, I'm ending up getting about 45-50 cr/hr. This is around 2.5X the amount the Cross-Project Parity Police want (15-23).
ID: 4978 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jord
Avatar

Send message
Joined: 30 Aug 07
Posts: 125
Credit: 207,206
RAC: 0
Message 4979 - Posted: 22 Aug 2008, 12:25:21 UTC - in response to Message 4978.  

I have the project set to No New Tasks, but BOINC is still reporting completed work without me having to hit the Update button.

Work will be reported (independently of the NNT option) at:

1) 24 hours after completion (was Connect every X after completion).
2) 24 hours before report is due.
3) Connect every X before report is due.
4) On a trickle up message (CPDN only so far).
5) On a request for more work.
6) On a manual update.
Jord.

The BOINC FAQ Service.
ID: 4979 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 4980 - Posted: 22 Aug 2008, 12:34:54 UTC
Last modified: 22 Aug 2008, 12:38:46 UTC

Also the Project will report every 20 Minutes on it's own, a Server Side Setting I think. That amount of time will slowly creep up though if no work is returned or requested.
ID: 4980 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 4983 - Posted: 22 Aug 2008, 13:44:54 UTC - in response to Message 4979.  
Last modified: 22 Aug 2008, 14:42:08 UTC

I have the project set to No New Tasks, but BOINC is still reporting completed work without me having to hit the Update button.

Work will be reported (independently of the NNT option) at:

1) 24 hours after completion (was Connect every X after completion).
2) 24 hours before report is due.
3) Connect every X before report is due.
4) On a trickle up message (CPDN only so far).
5) On a request for more work.
6) On a manual update.


Does that apply to 5.8.16? ;-)

Anyway, I can eliminate 4-6 automatically. Tasks were reporting less than 24 hours after completion, thus #1 shouldn't be happening. #2 was not true. I would guess perhaps it has something to do with #3 and the DCF value(???). I'll look at it tonight...

Edit: YIKES!!!!

Task duration correction factor 41.471856
ID: 4983 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 4984 - Posted: 22 Aug 2008, 13:47:46 UTC - in response to Message 4980.  

Also the Project will report every 20 Minutes on it's own, a Server Side Setting I think. That amount of time will slowly creep up though if no work is returned or requested.


That might be more of a plausible explanation. I'll have to look at the log file tonight...
ID: 4984 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jord
Avatar

Send message
Joined: 30 Aug 07
Posts: 125
Credit: 207,206
RAC: 0
Message 4988 - Posted: 22 Aug 2008, 14:46:56 UTC - in response to Message 4983.  

Does that apply to 5.8.16? ;-)

Yes, it's from 5.8.16 onwards.
Jord.

The BOINC FAQ Service.
ID: 4988 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Odd-Rod

Send message
Joined: 7 Sep 07
Posts: 444
Credit: 5,712,523
RAC: 1
Message 4995 - Posted: 22 Aug 2008, 21:32:33 UTC - in response to Message 4937.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.


Actually, what I've noticed is that the estimated time to completion keeps increasing until 50%, after which it finishes quickly.

Rod
ID: 4995 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 4999 - Posted: 22 Aug 2008, 22:38:10 UTC - in response to Message 4995.  

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.


Actually, what I've noticed is that the estimated time to completion keeps increasing until 50%, after which it finishes quickly.

Rod


I watched 3 Wu's for awhile on 1 of my Quads & the Completion Time would stay the same for a few seconds but never actually went back up & then @ 50% Progress the Wu would finish in a few seconds ...
ID: 4999 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Odd-Rod

Send message
Joined: 7 Sep 07
Posts: 444
Credit: 5,712,523
RAC: 1
Message 5005 - Posted: 23 Aug 2008, 5:41:16 UTC - in response to Message 4999.  
Last modified: 23 Aug 2008, 6:17:29 UTC

Once a WU gets to 50%, it will finish in 15 to 20 seconds. Other than that, they seem to be fairly consistent.


Actually, what I've noticed is that the estimated time to completion keeps increasing until 50%, after which it finishes quickly.

Rod


I watched 3 Wu's for awhile on 1 of my Quads & the Completion Time would stay the same for a few seconds but never actually went back up & then @ 50% Progress the Wu would finish in a few seconds ...

It's interesting that we have noticed different behaviour. I'm going to to take more careful note of a couple of WUs on different hosts and I'll post what I find. It may only be after a couple of days, so don't watch this space yet. ;)

[edit]: Well, that was a quick couple of days!
Ok, I see what it is. I use Boincview (BV) most of the time to monitor my various hosts. It has a 'To Completion' column and a 'Completion' column. The To Completion column would be equivalent to Boinc manager's (BM) To Completion, whereas BV's Completion column show the estimated total crunch time for the WU, based on the progress.

In the last few minutes the To Completion does in fact seem to only drop. But, it's not dropping fast enough - over 1 minute it may drop only 40 seconds (total guess - I haven't had time to check more carefully yet).
Because of this the Completion time rises slowly until 50%.

This in fact exactly what would be expected of a WU that finishes the first 50% much slower than the second 50%.

I hope this made sense - I edited quickly before going to work. Will reread tonight and see if even I understand it!
[/edit]
ID: 5005 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 5007 - Posted: 23 Aug 2008, 8:36:16 UTC

In the last few minutes the To Completion does in fact seem to only drop. But, it's not dropping fast enough - over 1 minute it may drop only 40 seconds (total guess - I haven't had time to check more carefully yet).
Because of this the Completion time rises slowly until 50%.


I suspect With all the hesitations in the To Completion time it could be why when the Wu gets to 50% completion it then jumps to 100% & is done ...

ID: 5007 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Odd-Rod

Send message
Joined: 7 Sep 07
Posts: 444
Credit: 5,712,523
RAC: 1
Message 5011 - Posted: 23 Aug 2008, 18:13:26 UTC - in response to Message 5005.  

Can't edit anymore so apologies for replying to my own post.

In the last few minutes the To Completion does in fact seem to only drop.

I meant 'In the last few minutes that I've been watching a WU crunch, the To Completion...'

Also, I should probably have said more specifically that I must have been referring to Completion - and not To Completion - when I mentioned it increasing in the first 50% processing of a WU.

Just for the record. Since this morning, I have yet to see To Completion increase.

However, a thought or question, not an observation of any such behaviour:
If you get one of the long WUs and your DCF is too low for it, would the To Completion not increase during processing until 50%, since the initial estimate would have been too low? Any ideas on that?

Rod


ID: 5011 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,548,171
RAC: 2,784
Message 5012 - Posted: 23 Aug 2008, 19:27:36 UTC - in response to Message 5011.  
Last modified: 23 Aug 2008, 20:10:05 UTC

owever, a thought or question, not an observation of any such behaviour:
If you get one of the long WUs and your DCF is too low for it, would the To Completion not increase during processing until 50%, since the initial estimate would have been too low? Any ideas on that?

Rod




I would think it should to some degree, but with the To Completion time as Hesitant as it is it might over-ride any DFC discrepancy so the To Completion time wouldn't rise anyway unless the DFC was so far off it almost had to rise ... IMO
ID: 5012 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 5015 - Posted: 23 Aug 2008, 20:40:12 UTC - in response to Message 4984.  
Last modified: 23 Aug 2008, 20:42:41 UTC

Also the Project will report every 20 Minutes on it's own, a Server Side Setting I think. That amount of time will slowly creep up though if no work is returned or requested.


That might be more of a plausible explanation. I'll have to look at the log file tonight...


I looked into this today... The reporting happened at no more than 4 seconds (sometimes 1 second) after the completion of the upload for each and every task. What comes to mind right offhand is that BOINC is confused by the high DCF.

However, there is one other possibility... Can return_results_immediately be triggered by a science application or, put another way, can science applications initiate a communication session on their own?
ID: 5015 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : rate of processing

©2024 Astroinformatics Group