Please read in the other topic for announcement of n-body 1.52 what I wrote (one month ago and now again), nobody ever answered me but they are not working on my Mac : running on 1 core out of 8, blocking the other boinc apps to run (pretending it is "mt"), and in 1.52 they would not even complete normally (AND they started running again now when my setup does not allow nbody)...

http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=3819&postid=64164


EDIT : there is a change now, after 15mn of running they actually start to run on all 8 cores and really using 100% of my CPU (instead of 1/8th).


EDIT2 : they don't completely run 100%, it's more like 90% of available CPU, letting a 10% idle.


EDIT3 : OK so the task did finish on estimated time and was sent successfully back to the servers, so things look better with this 1.54, even though not perfect (running 15mn with 1/8 of CPU and not running a full 100% for the remaining time).

I've got one that's seems to be causing problems. This one has been running for 78 hours and it's been at 100% since about 9 hours and it never did save a checkpoint. The original estimate was for about 1.5 hours. BOINC indicates it is using 8 cpus, but the activity monitor shows it is only using 1.3% cpu with 2 threads. I've seen some with the previous version of N-body work like this, so I'll probably watch it for a little while longer to see if it finishes.

I'm running a Mac Mini with OS 10.10.5

The work unit is de_nbody_9_09_15_orphan_sim_0_1437561602_56033_0

This app is supposed to use X cores, but it seems, regardless the CPU's type, it uses only up to 25 % of the CPU during 50 % of the WU's lenght, and then up to 100 % for the remaining 50 % of the time.

Hey,


That would be an awesome thing to get working. We have been tossing ideas around about how to do it. But it is still, unfortunately, a work in progress.

Also, right now we want to focus on making sure the application can actually return results before tinkering with the code again. But, again, we have that on our to do list!

Cheers,
Sidd

Interesting thoughts. I wonder if Sidd can enlighten us where the parallelisation 'sweet spot' is, or if they'd like us to try and find it. Considering the MT phase only, I'd imagine that there comes a point where the overhead of managing and synchronising multiple threads exceeds the benefit - but I wouldn't know whether the tipping point is above or below 15 threads.

I'm currently verifying that the Application configuration tools available in BOINC v7.4.36 allow thread control - initially, limiting the active thread count to 3, so that other projects can continue to make progress on one core while nbody runs.

The next test is to run a bundle of tasks to the first checkpoint with an app_config thread limit of one, with the intention of changing app_config when they've all been prepped, and running the MT phase with a 3-thread app_config. Very labour intensive, and not amenable to scripted automation, but might be an interesting proof-of concept. If it works, the project might consider splitting the app at the end of the initialisation phase.

a) Send out a single threaded task to perform initialisation
b) Return the initialisation data generated as an output file
c) Send out the initialistion data as an input file to a new, multithreaded, simulation task.

What I would like to know is if the initialization and actual computation of a run can be split for more effective use of available resources. Instead of doing it like this:
Download work units
Initialize one on one thread (blocking all other cores/threads from useful work)
Process the workunit
Send in completed work
Repeat

I propose the work be done like this (2 methods):
Batch mode
1.Download workunits
2.Initialize a number of workunits simultaneously. Each unit that is initialized has its state saved to await open resources, and forms a work queue in the order its prep completes. This way the execution resources arent sitting there doing nothing.
3A.When there are sufficient workunits ready, the app switches to multi-threaded mode, grabs an initialized unit, and begins processing.
3B. When complete, the unit is turned in and another initialized unit is pulled from the client side work queue for processing.
4.When the initialized work queue is exhausted, it switches back to initialization mode and preps another batch of work.

Stream mode
1.Download workunits
2. Initialize a workunit, then begin processing the moment it is done using the specified thread like how its currently done, but minus 1 thread.
3. While the ready unit is getting chewed through, the open thread is used to prep another unit for processing. Units that are ready before resources open up have their states saved to disk. The open thread is then used to ready another unit.
4. As each unit completes, a ready unit is slotted in and begins processing. This keeps going until the work dries up (either from nothing coming from the project, or the user has set no new work for the project).

In either case, 2 programs are necessary. One to initialize, and one to actually process.

Now, I'll admit I know very little about programming and thus dont know the viability of switching back and forth between prepping a batch of workunits and chewing through them one at a time when they're ready.

Therefore, for now, we removed all multithreading of the initialization of the dwarf galaxy. This is why it will run only on one thread until the initialization completes. However, after performing a speed profile on the code, it was determined that a majority of the run time of the previous code was spent on a single function (thanks to Roland Judd for catching that!). We optimized this function, leading to a great decrease in the run time.

Cheers,
Sidd

To put some figures on that. I'm running on an i5 laptop (2 cores, 4 threads). Current task was estimated at 5 hours 26 mins. The single-threaded initialisation phase lasted for 18 minutes. (I think the initialisation lasted roughly the same time for the previous task, estimated at 50 minutes, but I didn't have Process Explorer open to monitor). Would I be right in assuming that the initialisation would be expected to have a constant duration for any given host, no matter how long the expected task duration?

Hey,

We had multithreaded the assignments of radii and velocities to bodies. Both of these were done through rejection sampling, using random numbers. However, when that code ran with multiple threads the assignment of radii and velocity were different between runs even with the same random number seed and parameters. This was because which thread ran in which order was indeterminate, meaning which body was assign what radii and velocity was unpredictable. This was a very nasty bug, made nastier because it did not present itself. Runs would complete normally. However, because of the indeterminate nature of the algorithm, a poorer likelihood was reported than would be expected with a set of parameters, even if they were close. Therefore, overall, it led to poor convergence.

Therefore, for now, we removed all multithreading of the initialization of the dwarf galaxy. This is why it will run only on one thread until the initialization completes. However, after performing a speed profile on the code, it was determined that a majority of the run time of the previous code was spent on a single function (thanks to Roland Judd for catching that!). We optimized this function, leading to a great decrease in the run time.

Cheers,
Sidd