Welcome to MilkyWay@home

Tasks slow to start

Message boards : Number crunching : Tasks slow to start
Message board moderation

To post messages, you must log in.

AuthorMessage
Aaron

Send message
Joined: 2 May 10
Posts: 6
Credit: 267,445,323
RAC: 50,227
Message 77142 - Posted: 18 May 2024, 18:35:59 UTC

First of all this is a gaming PC I built last year but quickly ran out of time to use it for its intended purpose. Now it crunches 24/7 with a weekly reboot on Saturday.

I've been looking to optimize performance and have noticed all tasks sit at 0% for exactly 45 seconds then finish normally a couple minutes later. Is this normal behavior?

Currently I'm playing around with how many cores to allocate per task and seem to be experiencing diminishing returns with actual task start time. For example... if all tasks finish around the same time the CPU sits idle until the next ones actually start. While monitoring RAM the bandwidth ranges from 1.5-4GBPS.while running and sub 1GBPS until they actually start. Any suggestions?
ID: 77142 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 618
Credit: 19,254,980
RAC: 26
Message 77143 - Posted: 18 May 2024, 19:22:43 UTC - in response to Message 77142.  
Last modified: 18 May 2024, 19:40:08 UTC

I've been looking to optimize performance and have noticed all tasks sit at 0% for exactly 45 seconds then finish normally a couple minutes later. Is this normal behavior?
Yes.


Currently I'm playing around with how many cores to allocate per task and seem to be experiencing diminishing returns with actual task start time. For example... if all tasks finish around the same time the CPU sits idle until the next ones actually start. While monitoring RAM the bandwidth ranges from 1.5-4GBPS.while running and sub 1GBPS until they actually start. Any suggestions?
Use one thread per task. This eliminates the times when the CPU sits nearly idle during the single thread start up phase of each task.

According to my calculation for my Ryzen system, the multi-threaded application (v1.86) using just two threads per task would generate a RAC of about 14k, the single-threaded (v1.86) about 22k, however I think there seems to be some CreditNew lottery involved considering the lower credits for v1.87 (RAC ~20k), so YMMV, but I expect one thread per task to be most efficient in any case. And my CPU also seems to be using about 4 Watts less when running 16x 1 thread instead of 8x 2 thread, no idea why, but it does.
ID: 77143 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 14
Message 77144 - Posted: 19 May 2024, 11:04:54 UTC - in response to Message 77142.  

I've been looking to optimize performance and have noticed all tasks sit at 0% for exactly 45 seconds then finish normally a couple minutes later. Is this normal behavior?

Currently I'm playing around with how many cores to allocate per task and seem to be experiencing diminishing returns with actual task start time. For example... if all tasks finish around the same time the CPU sits idle until the next ones actually start. While monitoring RAM the bandwidth ranges from 1.5-4GBPS.while running and sub 1GBPS until they actually start. Any suggestions?


i don't think the cores are idle just waiting for Boinc to recognize the lowest percentage of crunching that MW has programmed into the tasks, somewhere around 1 to 2%, this is also where the task checks to see if it's been run previously and has a checkpoint to start from etc etc etc. You can look at the logs if you really want to, all Boinc tasks from every Project do similar things, by looking in the stderr file in the boinc/slots directories.

I agree with Link 1 core per task seems to work for me too.
ID: 77144 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 618
Credit: 19,254,980
RAC: 26
Message 77145 - Posted: 19 May 2024, 13:36:56 UTC - in response to Message 77144.  

i don't think the cores are idle just waiting for Boinc to recognize the lowest percentage of crunching that MW has programmed into the tasks, somewhere around 1 to 2%
Yes, the cores are idle, the initial phase is single core, so if you run Milkyway with the default 16 threads per task, 1 will be used and 15 idle. There's also no progress during that phase reported by the application, it stays at 0.000% until it's done with whatever it's doing there on that single core.
ID: 77145 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 14
Message 77146 - Posted: 21 May 2024, 10:02:54 UTC - in response to Message 77145.  

i don't think the cores are idle just waiting for Boinc to recognize the lowest percentage of crunching that MW has programmed into the tasks, somewhere around 1 to 2%
Yes, the cores are idle, the initial phase is single core, so if you run Milkyway with the default 16 threads per task, 1 will be used and 15 idle. There's also no progress during that phase reported by the application, it stays at 0.000% until it's done with whatever it's doing there on that single core.


I don't see any of that because I run them all single core.
ID: 77146 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 618
Credit: 19,254,980
RAC: 26
Message 77147 - Posted: 21 May 2024, 18:33:01 UTC - in response to Message 77146.  

Yes, that eliminates the issue completely.
ID: 77147 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile KeithBriggs

Send message
Joined: 28 Apr 11
Posts: 36
Credit: 283,587,207
RAC: 22,663
Message 77148 - Posted: 29 May 2024, 13:01:18 UTC - in response to Message 77147.  

Great catch btw.

I switched from 16 cores to 2 cores per WU. Odd that the delay went from ~30 sec to ~60 sec so the other cores were doing something "during idle" when 16 were process each WU.

With 32 cores, I have 16 running now.

Maybe I'll try one core per WU later but it seems to be much more efficient per task manager.
ID: 77148 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 618
Credit: 19,254,980
RAC: 26
Message 77149 - Posted: 29 May 2024, 15:39:04 UTC - in response to Message 77148.  

Odd that the delay went from ~30 sec to ~60 sec
That's expected, now the start up phase has to share a physical CPU core with one thread from another task, before it had the whole core for itself.
ID: 77149 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile KeithBriggs

Send message
Joined: 28 Apr 11
Posts: 36
Credit: 283,587,207
RAC: 22,663
Message 77171 - Posted: 3 Jul 2024, 4:08:11 UTC

I did switch to from 2 to 1 core per work unit. Thanks again.
ID: 77171 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Tasks slow to start

©2024 Astroinformatics Group