In my case, the Milkyway tasks force other tasks to stop running. Solution? Until a better solution is found to allow other tasks to continue running, forbid new tasks so that other tasks can run.

James,

TL;DR -- use app_config.xml

[Otherwise...] A couple of points about managing work, one about managing BOINC work in general and one specific to MilkyWay N-body...

The general one first because it plays into the overall situation - it's about how much of your system BOINC is allowed to use, and how:

A system needs one or two spare processor threads all the time to manage the system, handle I/O and do whatever else might be needed. I note that you seem to have constrained your system to leave one processor thread free; I'd be very inclined to free up at least one more processor thread because there's quite a lot going on besides your user program(s) (BOINC or otherwise) and every time one of those things needs to get a turn it has to suspend another process if there's a shortage of free processors :-)

There's another issue too; certain sorts of work place enough overheads on L3 cache and RAM access to result in processor threads running idle for lack of data. There are projects where it is often recommended that one uses at most half the available CPU threads, and I've had work from at least one project where an 8-thread processor was more productive running 2 tasks than 4 :-) (In certain cases there might be other non-calculation related reasons for delay, but this is getting a bit long already!)

There's also the matter of managing how much work is run at once if one has multiple BOINC projects active. This is best managed by using an app_config.xml file in each project folder to provide a project_max_concurrent constraint as to how many tasks can run for that project at the same time. (There's plenty of discussion about app_config.xml elsewhere...)

A couple of my systems are Ryzens (a 3700X and a 5600H) and I've found I get the best throughput if I only use 75% of the CPUs (12/16 for the 3700X, 9/12 for the 5600H) -- to be fair, I have a couple of home-grown BOINC monitoring daemons running on each system (so I like to keep a thread clear for when one of those needs to run) and I'm running a mix of CPU tasks from WCG and MilkyWay, and I have never experimented with an all-MilkyWay mix, but the principal is the same.

Now, regarding N-body (and, possibly, other multi-threaded BOINC project applications out there), assigning too many threads to one task may not be a good thing...

By default, a single N-Body task will use all CPU threads up to the limit set by your "Use at most ??% of the CPUs" or 16, whichever is less. Unfortunately, the more threads assigned to a single task, the more likely it is that one or more threads will be suspended at any given time to let the BOINC client in or to allow the O/S to do its thing, and as there are regular points within the N-body application where it needs to ensure all threads are [more or less] in sync there will be times when threads are waiting, not computing.

You'll get more throughput if you actually use an app_config.xml file in the MilkyWay project folder to constrain the number of threads allocated to a single task. I found that there was a slight degradation for each thread up to three or four, then it went downhill quite fast -- while I was working out an optimum for one of my systems, I found 2 three-thread tasks worked the CPUs harder than 1 six-thread task did, and I now run one 3-thread N-body at a time, intermixed with other BOINC stuff.

That remark about reducing the number of threads is equally applicable on Windows systems, by the way, and there's a fair amount of discussion about using app_config.xml files for this purpose over in Number Crunching.

Hope this helps.

Cheers - Al.

I will probably read through your information a few times to assimilate it all. The way things are set up now, when Milkyway is running, nothing else can run until all the Milkyway tasks have been completed. Easiest would be to just run one project as, if I read your text correctly at first read, fine tuning to get a balance between the projects is tedius at best.

I would like to assign one or two processors to each project, but have yet found a way to do that. It just occurred to me that if I restrict each preference to two processors, perhaps that would work. I will need to try it.

I will probably read through your information a few times to assimilate it all. The way things are set up now, when Milkyway is running, nothing else can run until all the Milkyway tasks have been completed. Easiest would be to just run one project as, if I read your text correctly at first read, fine tuning to get a balance between the projects is tedius at best.

I would like to assign one or two processors to each project, but have yet found a way to do that. It just occurred to me that if I restrict each preference to two processors, perhaps that would work. I will need to try it.

You need an app_config.xml file something like this:

<app_config>

<app_version>
<app_name>milkyway_nbody</app_name>
<max_concurrent>2</max_concurrent>
<plan_class>mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<cmdline>--nthreads 2</cmdline>
</app_version>
</app_config>

That file should be copied into Notepad in Windows or a text editor in Linux and then saved in the c:\program data
boinc\projects\...milkyway'' directory. The directory could be hidden so you will need to unhide it, google it it's really easy, then go in the Boinc Manager and click Options, read config files, for it to take effect.

The file above tells Boinc to use 2 cpu cores and a maximum of 2 cpu tasks at a time of the nbody tasks, you can increase the total number of tasks to run at one time by changing the <max_concurrent> line, but since you are having troubles using 16 cpu cores I would start with 4 tasks at a time and move up or down from there as that's only using a total of 8 cpu cores across the 4 tasks. You can also change the number of cpu cores per task but be careful as changes effect NEW tasks not existing tasks so if you have 100 tasks using 2 cores each and change it to 4 cpu cores per task that won't change until you start to get new tasks. The key is to use a small cache size of say 0.01 and 0.01 for the 2 settings, that is also inside the Boinc Manager under Options, computing preferences. Changing things in the Boinc Manager ONLY effects that one single pc, changing things on the website changes things for every pc you have.

c:\program data boinc\projects\...milkyway'' directory.

Unable to find where to put the file. Never ran anything like Wine on this computer so that type of directory does not exist.

I will just pick one so I will not deal with anything as complex as that. Tried to run LHC, even loaded the virtual box, but that is beyond my current knowledge to use. Seems there is something missing and I do not have the resources to find out how to use it even after reading the so-called manual. Too complex for me as it does nothing.

I will just run one project at a time and forget the rest of the problems that come with running more than one or a project that requires virtual box. If it is not plug and play, I will just ignore it.

James,

I will just run one project at a time and forget the rest of the problems that come with running more than one or a project that requires virtual box. If it is not plug and play, I will just ignore it.

Fair enough, it's your choice to make, but if you change your mind...

Mikey's information was good (apart from the Windows-specific path!).

You have a recent BOINC client so if it did a standard install, there will be a directory /var/lib/boinc (which may be a link to /var/lib/boinc-client in some cases!) One of the subdirectories in there will be called projects and the various projects each have a separate directory in there. Most will have a directory name that should identify the project :-)

Should you change your mind and feel the need for more guidance, let us know what sort of work mix you'd like to achieve and someone can probably offer further assistance. And if you're feeling adventurous and want to find out more about app_config.xml on your own you could always look at https://boinc.berkeley.edu/wiki/Client_configuration [if you haven't already done so :-)]

Good luck and happy crunching.

Cheers - Al.

P.S. On my Ryzen 3700X I run a mix of WCG (selected sub-projects, each separately managed via app_config.xml), Einstein (GPU work only) and MilkyWay N-body, with TN-Grid as a fallback for times when WCG is short of work. I found it easy to set up and tune, and it rarely gives me any trouble unless there's a mass shortage of work :-)

..James wrote:

I will just run one project at a time and forget the rest of the problems that come with running more than one or a project that requires virtual box. If it is not plug and play, I will just ignore it.

You can always add an app_config.xml file to limit the number of tasks each Project runs at a time but again that involves changing things in the var/init/boinc/projects folders. MOST people never do any of that and as you said just run one project at a time.

As for Virtual Box did you use the default software installer to install Boinc or did you use Synaptic Package Manager? Because if you used Synaptic one of the options when you do a search on Boinc is to install virtual box at the same time. Yes you can do it now but there are plenty of Boinc Projects that don't need it.

I found that there was a slight degradation for each thread up to three or four, then it went downhill quite fast -- while I was working out an optimum for one of my systems, I found 2 three-thread tasks worked the CPUs harder than 1 six-thread task did

How do you define "works harder?"
And what CPU did you test this on? TIA

---

(Aurum: it took me a moment to realize you were quoting from one of my earlier posts - sorry about the delay in responding.)

I found that there was a slight degradation for each thread up to three or four, then it went downhill quite fast -- while I was working out an optimum for one of my systems, I found 2 three-thread tasks worked the CPUs harder than 1 six-thread task did

How do you define "works harder?"

"Works harder" is based on instructions retired per second rather than total run time. The latter is not as useful as a throughput statistic because no two N-body tasks are guaranteed to execute [roughly] the same number of instructions...

I measured instructions/second for a task over a prolonged period to ensure I was likely to catch multiple iterations; I set the checkpoint interval quite high to avoid accidentally sampling a task during a checkpoint, as such activity is likely to involve a lot of context switches in a short time interval, and a lot of the CPU usage at that point is O/S rather than application...

If I wanted to test two N-body tasks running together, I'd sample one of them for a specified time then sample the other for the same amount of time. In general, if the tasks had the same number of threads they would perform in a reasonably similar fashion. By only allowing the same total number of threads, I was able to observe that there was a large enough improvement to make the smaller tasks more practical (if I intended to give that many threads to N-body...)

(Of course, I repeated the tests on several different tasks, and had to discard one or two tests because the tasks finished mid-test! The run-time estimates for N-body are not very accurate...)

It is likely that part (if not all) of the apparent improvement is down to threads for a specific task being somewhat less likely to get out of sync if there are less of them; if threads are being juggled around, caches and TLBs may be affected and more instructions will be likely to stall.

Incidentally, I never allow BOINC to use more than 75% of CPU threads; most of my systems have enough stuff going on in the background that leaving about 25% of threads free seems an effective level. Of course, that's my systems; others may find different settings are better (especially if they have completely different hardware platforms.

And what CPU did you test this on? TIA

The only systems I have [at present] that are likely to offer more than two or three threads to N-body are the Ryzen 3700X and a Ryzen 5600H I mentioned in my earlier post -- I tested the 3-versus-6 scenario on both with no other BOINC tasks running, and also looked [briefly] at the effect of various workloads on a single 3-thread task. Both systems have 2x16GB RAM; the 3700X has total power limited to 80W (which doesn't seem to slow it up much, if at all!)

I suspect the outcome might be [slightly] different on Intel non-server CPUs, and likewise fot any server chipsets; unfortunately, I don't have any of the latter for testing :-) Throughput may also depend on the number of memory channels and how memory is installed -- I've seen that discussed elsewhere (probably at WCG, but I may be misremembering...)

By the way, I haven't seen any CPDN work or WCG ARP1 work in a while -- I rather suspect that when either of those makes an appearance the memory will take an extra hit and N-body tasks will run less efficiently at that point :-)

Hope that answers your questions.

Cheers - Al.

Al, "instructions retired per second" is the secret sauce that makes this work so well. I had taken your word for it before and have all my computers running 3-thread nbodys. Thanks much for doing this useful work and explaining it so thoroughly.
I once tried to install PERF to measure instructions retired but it gave me four ways to install it and I didn't get it working. It's the only Linux program I've ever seen suggest multiple ways to install.
CPDN has issued a slug of WAH but it's windoze only. Their new guy is planning on putting out much OpenIFS for Linux this fall:
https://www.cpdn.org/forum_thread.php?id=9149&postid=69160#69160

---

Al, "instructions retired per second" is the secret sauce that makes this work so well. I had taken your word for it before and have all my computers running 3-thread nbodys. Thanks much for doing this useful work and explaining it so thoroughly.
I once tried to install PERF to measure instructions retired but it gave me four ways to install it and I didn't get it working. It's the only Linux program I've ever seen suggest multiple ways to install.
CPDN has issued a slug of WAH but it's windoze only. Their new guy is planning on putting out much OpenIFS for Linux this fall:
https://www.cpdn.org/forum_thread.php?id=9149&postid=69160#69160

"PERF" as in Ithena Measurements Perf tasks? If so it's like any other Boinc project you choose what kind of tasks you want to run, Perf is for Windows only while Ooni tasks are Linux only and the Cnode tasks are for both I think. There is also a Project Ithena computation that has Hex tasks https://comp.ithena.net/usr/

CPDN has issued a slug of WAH but it's windoze only. Their new guy is planning on putting out much OpenIFS for Linux this fall:
https://www.cpdn.org/forum_thread.php?id=9149&postid=69160#69160

WOO HOO!!!

mikey - for information:

"PERF" as in Ithena Measurements Perf tasks? If so it's like any other Boinc project you choose what kind of tasks you want to run, Perf is for Windows only while Ooni tasks are Linux only and the Cnode tasks are for both I think. There is also a Project Ithena computation that has Hex tasks https://comp.ithena.net/usr/

Nope - we're talking Linux, and perf is one of a set of kernel-specific system tools; it offers various different ways of looking at system performance...

Cheers - Al.

mikey - for information:

"PERF" as in Ithena Measurements Perf tasks? If so it's like any other Boinc project you choose what kind of tasks you want to run, Perf is for Windows only while Ooni tasks are Linux only and the Cnode tasks are for both I think. There is also a Project Ithena computation that has Hex tasks https://comp.ithena.net/usr/

Nope - we're talking Linux, and perf is one of a set of kernel-specific system tools; it offers various different ways of looking at system performance...

Cheers - Al.

Aurum,

Al, "instructions retired per second" is the secret sauce that makes this work so well. I had taken your word for it before and have all my computers running 3-thread nbodys. Thanks much for doing this useful work and explaining it so thoroughly.

Glad to have been of help!

I once tried to install PERF to measure instructions retired but it gave me four ways to install it and I didn't get it working. It's the only Linux program I've ever seen suggest multiple ways to install.

Regarding installing perf (and other kernel-specific tools): I'm on Ubuntu rather than Mint, and all I needed to do was tell Synaptic Package Manager (which I use rather than running apt or dpkg from a console!) to install linux-tools-generic... This makes sure that whenever a new kernel version is installed via Software Updater a new version of perf (and friends) gets pulled in to match the new kernel -- the older versions will remain until the corresponding kernels are uninstalled.

The alternative is to install explicitly by kernel version -- that can be a bit of a pain :-) as one ought to match both the version and the flavour (which is generic in my case and yours...

I'm not familiar enough with Mint to know whether it structures its packages in the same way; sorry about that...

CPDN has issued a slug of WAH but it's windoze only. Their new guy is planning on putting out much OpenIFS for Linux this fall:
https://www.cpdn.org/forum_thread.php?id=9149&postid=69160#69160

Yup, I've been following that saga and am looking forward to running some proper 64-bit CPDN work...

Cheers - Al.

mikey - for information:

"PERF" as in Ithena Measurements Perf tasks? If so it's like any other Boinc project you choose what kind of tasks you want to run, Perf is for Windows only while Ooni tasks are Linux only and the Cnode tasks are for both I think. There is also a Project Ithena computation that has Hex tasks https://comp.ithena.net/usr/

Nope - we're talking Linux, and perf is one of a set of kernel-specific system tools; it offers various different ways of looking at system performance...

Cheers - Al.

Just for information, 'Perf' work units can run on both Linux and Windows.

I run them on my Linux computers along with CNode and OONI Probe.
The only problem is I have to often hit update for OONI and Perf to download as CNode is the default. Project Admin has not worked out how to download all at once unfortunately.

Conan

---

mikey - for information:

"PERF" as in Ithena Measurements Perf tasks? If so it's like any other Boinc project you choose what kind of tasks you want to run, Perf is for Windows only while Ooni tasks are Linux only and the Cnode tasks are for both I think. There is also a Project Ithena computation that has Hex tasks https://comp.ithena.net/usr/

Nope - we're talking Linux, and perf is one of a set of kernel-specific system tools; it offers various different ways of looking at system performance...

Cheers - Al.

Just for information, 'Perf' work units can run on both Linux and Windows.

I run them on my Linux computers along with CNode and OONI Probe.
The only problem is I have to often hit update for OONI and Perf to download as CNode is the default. Project Admin has not worked out how to download all at once unfortunately.

Conan

Thanks for that and yes I too wish it would just download all the types of tasks for the pc asking for the tasks!!