Welcome to MilkyWay@home

How do tasks like de_nbody_08_31_2021_v176_40k__data__13_1647295263_3642347_2 work?

Questions and Answers : Unix/Linux : How do tasks like de_nbody_08_31_2021_v176_40k__data__13_1647295263_3642347_2 work?
Message board moderation

To post messages, you must log in.

AuthorMessage
Jean-David Beyer

Send message
Joined: 14 May 22
Posts: 7
Credit: 8,077,321
RAC: 0
Message 73501 - Posted: 14 May 2022, 6:13:33 UTC

I am running a machine using Red Hat Enterprise Linux release 8.5 (Ootpa).

When my machine runs a task like de_nbody_08_31_2021_v176_40k__data__13_1647295263_3642347_2, it seems to run just fine, and sends back up to the server where it is marked valid. But if I run the top command to look at only Boinc tasks, that kind of task seems to be assigned to one CPU, but that CPU seems to run at 650% (or so) of capacity, and the other Boinc tasks go to sleep. I get the impression that it is using six to eight processors, but top does not say so. So this is not a complaint, but I would like to know what is really going on.
ID: 73501 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 13 Oct 21
Posts: 43
Credit: 225,021,704
RAC: 9,632
Message 73503 - Posted: 14 May 2022, 8:23:23 UTC - in response to Message 73501.  

You're guessing correctly, the task is probably using 8 cores. N-Body Simulation project is multicore and can be set up to use 1 to 16 cores per task via app_config.xml. By default each task will use all available cores (up to 16). BOINC will even sometimes pause other running tasks, like you've noticed. I'm currently running this project on Ubuntu 20.04 (via WSL2 on Windows 10) and both top and htop commands also give the same reading as you describe. It seems to be the (strange) way they interpret multicore usage. I found that 3-5 core setup produces highest throughputs. Here's the app_config.xml that I have to run it using 4 cores.
<app_config>
   <app_version>
      <app_name>milkyway_nbody</app_name>
      <plan_class>mt</plan_class>
      <avg_ncpus>4</avg_ncpus>
   </app_version>
</app_config>
ID: 73503 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alanb1951

Send message
Joined: 16 Mar 10
Posts: 208
Credit: 105,454,055
RAC: 36,234
Message 73504 - Posted: 14 May 2022, 8:30:09 UTC - in response to Message 73501.  
Last modified: 14 May 2022, 8:35:47 UTC

Jean-David.

N-Body tasks use the OpenMP library to co-ordinate multiple threads, and by default an N-Body task will use all the "CPUs" allocated to BOINC. You can tune this if you want to, by adding an app_config.xml file; I've done this on my Ryzen 3700 to constrain it to 4 CPU threads, and it reports
Using OpenMP 4 max threads on a system with 16 processors

[Edit: I see AndreyOR posted while I was composing this, and covered the tuning aspect...]

System calls for getting CPU usage can either be program-specific or thread-specific; the top command (in its basic form) gets performance metrics based on all threads associated with a running program so what you're seeing is to be expected.

Cheers - Al.
ID: 73504 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 14 May 22
Posts: 7
Credit: 8,077,321
RAC: 0
Message 73507 - Posted: 14 May 2022, 12:15:42 UTC - in response to Message 73503.  

You're guessing correctly, the task is probably using 8 cores. N-Body Simulation project is multicore and can be set up to use 1 to 16 cores per task via app_config.xml. By default each task will use all available cores (up to 16). BOINC will even sometimes pause other running tasks, like you've noticed. I'm currently running this project on Ubuntu 20.04 (via WSL2 on Windows 10) and both top and htop commands also give the same reading as you describe. It seems to be the (strange) way they interpret multicore usage. I found that 3-5 core setup produces highest throughputs. Here's the app_config.xml that I have to run it using 4 cores.


Thank-you. I set it to run with only 4 cores and it is now working that way.

I am not sure what the optimum is. I am now running
# perf stat -aB -e cache-references,cache-misses
to see how the processor cash is doing. So I am by no means running out of processor cache. This is unusually good number. When ClimatePrediction is running big (N216) models, the processor cache missis go up to about 50%, but now only milkyway and universe@home are the only Boinc tasks (and not much else) that are running.
Performance counter stats for 'system wide':

    10,736,725,787      cache-references                                            
       122,353,406      cache-misses              #    1.140 % of all cache refs    

      60.247759271 seconds time elapsed

ID: 73507 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3315
Credit: 519,950,233
RAC: 22,029
Message 74071 - Posted: 17 Aug 2022, 10:45:46 UTC - in response to Message 73507.  

You're guessing correctly, the task is probably using 8 cores. N-Body Simulation project is multicore and can be set up to use 1 to 16 cores per task via app_config.xml. By default each task will use all available cores (up to 16). BOINC will even sometimes pause other running tasks, like you've noticed. I'm currently running this project on Ubuntu 20.04 (via WSL2 on Windows 10) and both top and htop commands also give the same reading as you describe. It seems to be the (strange) way they interpret multicore usage. I found that 3-5 core setup produces highest throughputs. Here's the app_config.xml that I have to run it using 4 cores.


Thank-you. I set it to run with only 4 cores and it is now working that way.

I am not sure what the optimum is. I am now running
# perf stat -aB -e cache-references,cache-misses
to see how the processor cash is doing. So I am by no means running out of processor cache. This is unusually good number. When ClimatePrediction is running big (N216) models, the processor cache missis go up to about 50%, but now only milkyway and universe@home are the only Boinc tasks (and not much else) that are running.
Performance counter stats for 'system wide':

    10,736,725,787      cache-references                                            
       122,353,406      cache-misses              #    1.140 % of all cache refs    

      60.247759271 seconds time elapsed


How many tasks at one time is it running? If only one task at a time are you using the other 12 cpu cores for Universe and Climate Prediction? Or are they just sitting idle?
ID: 74071 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : How do tasks like de_nbody_08_31_2021_v176_40k__data__13_1647295263_3642347_2 work?

©2024 Astroinformatics Group