Welcome to MilkyWay@home

Millkyway@home multi-CPU tasks

Questions and Answers : Preferences : Millkyway@home multi-CPU tasks
Message board moderation

To post messages, you must log in.

AuthorMessage
TheFanMan64

Send message
Joined: 24 Mar 20
Posts: 2
Credit: 4,216,530
RAC: 965
Message 72594 - Posted: 8 Apr 2022, 16:03:36 UTC

These multi CPU tasks are hogging all my Cores preventing single CPU tasks from Milkyway and other projects from running.
ID: 72594 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 696
Credit: 540,036,988
RAC: 86,680
Message 72597 - Posted: 8 Apr 2022, 17:16:02 UTC - in response to Message 72594.  

These multi CPU tasks are hogging all my Cores preventing single CPU tasks from Milkyway and other projects from running.

You need to limit the number of cores the N-body tasks use with a app_config.xml file.

<app_config>
<app>
<name>milkyway_nbody</name>
<max_concurrent>1</max_concurrent>
<app>
<app_version>
<app_name>milkyway_nbody</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>1</avg_ncpus>
<cmdline>--nthreads 4</cmdline>
</app_version>
</app_config>

This example would limit the host to only running a single task at a time using 4 thread or cores. You adjust the number of threads that the task spans on the cpu with the nthreads count.
ID: 72597 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,030,134
RAC: 36,348
Message 72598 - Posted: 8 Apr 2022, 17:44:02 UTC - in response to Message 72597.  

Interesting. Does changing <max_concurrent>1</max_concurrent> to <max_concurrent>4</max_concurrent>, allow me to run four 4 CPU instances?
ID: 72598 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3315
Credit: 519,946,492
RAC: 22,242
Message 72638 - Posted: 9 Apr 2022, 12:29:43 UTC - in response to Message 72598.  

Interesting. Does changing <max_concurrent>1</max_concurrent> to <max_concurrent>4</max_concurrent>, allow me to run four 4 CPU instances?


Yes

AND if you put it in your Climate Prediction Project you can set it to 1 and it will run 1 CPDN task at a time as well, essentially you can do this in every project just remember that you are doing it when you stop getting tasks from some place and find out you are only using 4 cpu cores of a 16 core pc and can't figure out why.

One other thing to think about is that the Boinc Project side has no clue you are doing this so keep your cache size in mind as Boinc will fill the cache, along with the individual project deadlines, when you get new tasks. Meaning that if you have a 16 core pc and are allowing 100% of the cpu cores to be used Boinc will get tasks for 16 cpu cores to finish within the deadline but if you are limiting it to 4 tasks at a time you may run into task deadline issues if your cache size is too big.
ID: 72638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JohnDK
Avatar

Send message
Joined: 18 Feb 10
Posts: 53
Credit: 221,574,791
RAC: 10,040
Message 72702 - Posted: 11 Apr 2022, 18:45:33 UTC

I only want to use 1 cpu per task, I'm using this in my app_config

<app>
<name>milkyway_nbody</name>
<app_version>
<app_name>milkyway_nbody</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>1</avg_ncpus>
<cmdline>--nthreads 1</cmdline>
</app_version>
</app>

Only running only 1 task at the moment and that task if using around 30% cpu, so what am I doing wrong?
ID: 72702 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 696
Credit: 540,036,988
RAC: 86,680
Message 72707 - Posted: 11 Apr 2022, 20:38:17 UTC - in response to Message 72702.  
Last modified: 11 Apr 2022, 21:06:48 UTC

What host? What cpu? How much of the cpu% is allowed in BOINC settings?

How are you measuring? What tool or utility?

1 core out of a 4 core cpu at 100% utilization is 25%

Have you restarted BOINC since you implemented your app_config.xml?

BOINC needs a restart on a brand new implementation of an app_config.xml file.
Once in place and active though, you can make changes to the file and just re-read config files in the Manager and get the changes picked up by the client.

Your tasks are using 10 threads still.
Using OpenMP 10 max threads on a system with 24 processors

HRFMguy is using 3 threads via his app_config
Using OpenMP 3 max threads on a system with 24 processors
ID: 72707 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JohnDK
Avatar

Send message
Joined: 18 Feb 10
Posts: 53
Credit: 221,574,791
RAC: 10,040
Message 72709 - Posted: 11 Apr 2022, 21:37:00 UTC

This host https://milkyway.cs.rpi.edu/milkyway/show_host_detail.php?hostid=880719

I'm allowing 10 CPUs in Boinc, I was just using task manager to see how much CPU it uses running 1 task, Rosetta use 4.x %, when suspending Rosetta nbody uses around 30%.

I did try restarting BOINC without it helping.

The solution that seems to work is editing client_state.xml and change <avg_ncpus>10.000000</avg_ncpus> to 1.000000.
ID: 72709 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 696
Credit: 540,036,988
RAC: 86,680
Message 72710 - Posted: 11 Apr 2022, 21:55:51 UTC - in response to Message 72709.  
Last modified: 11 Apr 2022, 21:57:47 UTC

Well, pretty much going against what OpenMP is used for and expecting.

The whole point is parallel processing. So limiting to only use a single thread is counter-productive.

Glad you figured out a solution you are happy with. This is what you are using now.

Using OpenMP 1 max threads on a system with 24 processors
ID: 72710 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JohnDK
Avatar

Send message
Joined: 18 Feb 10
Posts: 53
Credit: 221,574,791
RAC: 10,040
Message 72711 - Posted: 11 Apr 2022, 22:12:14 UTC

The only reason I'm trying nbody is that I was having trouble getting Separation work. And well I had 2 threads free so why not try the nbody stuff.
ID: 72711 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 696
Credit: 540,036,988
RAC: 86,680
Message 72712 - Posted: 11 Apr 2022, 22:18:26 UTC - in response to Message 72711.  

I'm not having any issues getting Separation work. I haven't run out yet.

I only carry 30 tasks in my cache, though it has dipped as low as 5 once in a while before refilling.

I run an update script every 15 minutes to top up my buffer cache.

I would recommend everyone do that now with the database rebuild difficulties and the major change in stability the project has suffered.
ID: 72712 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Preferences : Millkyway@home multi-CPU tasks

©2024 Astroinformatics Group