Welcome to MilkyWay@home

A simple OS process accounting error??

Message boards : Number crunching : A simple OS process accounting error??
Message board moderation

To post messages, you must log in.

AuthorMessage
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56842 - Posted: 12 Jan 2013, 3:12:16 UTC

I'm new to Milkeyway at home so this question may be covered somewhere else (sorry).

While watching my 4 WU's running, (sharing CPU's 1,3,5,and 7), I often see in Task manager 2 of the WU's running above the 13% (o.k.12.5% for 8 cpu's) that should be the maximum allowed usage (i.e. 1 cpu). I have seen WU's above 30% usage. Is this a simple OS accounting error ... or are WU's actually running on multiple cpu's? This SEEMS to be most noticeable with nbody_1.04 WU's.

Thanks

Ed Frybarger
ID: 56842 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56843 - Posted: 12 Jan 2013, 8:08:23 UTC
Last modified: 12 Jan 2013, 8:24:31 UTC

You are not imagining things, and yes the nBody application is specifically designed to be CPU multi-threaded (MT). Therefore, it is possible for it use more than 1/x percent of the CPU where x is the number of cores the processor has.

That being said, currently the 1.04 version is new and is being tested and debugged. As such it's currently setup in the context of BOINC itself to run in single threaded mode. This is why you have other projects tasks running at the same time an nBody task(s).

OK, so now for the confusing part. On Windows versions Vista and higher, the way the OS treats and handles MT programs is different than the way XP and earlier versions do. So in this case, a BOINC control component which specifies how the application is supposed to use the CPU is not in place. Without that component, BOINC defaults to treating the application as if it was a single threaded (ST) one. So generally speaking the app will only use 1/x percent of the CPU on average.

However, since Vista and higher is more aware of and sophisticated in dealing with MT programs, it has the ability to make some choices scheduling process run time on the CPU independent of and transparently to the user level programs running on the machine. This is why you are seeing the periodic 'spikes' above 12.5 percent on your 8 core machine.

So in other words, even though BOINC is treating the application as ST and requesting it to be run that way, Windows knows it in fact has MT capability, overrides the BOINC request and runs it MT briefly at times when it feels it should to maintain overall system performance criteria as defined by Microsoft.
ID: 56843 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56849 - Posted: 12 Jan 2013, 16:31:44 UTC - in response to Message 56843.  

O.K. so for the sake of WU throughput is it better to have BOINC schedule the computer at 100% or should I leave a CPU or 2 "idle" so Nbody can do its MT more efficiently??

Ed F
ID: 56849 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56852 - Posted: 12 Jan 2013, 19:11:49 UTC - in response to Message 56849.  
Last modified: 12 Jan 2013, 19:15:09 UTC

O.K. so for the sake of WU throughput is it better to have BOINC schedule the computer at 100% or should I leave a CPU or 2 "idle" so Nbody can do its MT more efficiently??

Ed F


Richard pointed out to me that you are running BOINC 6.12.34, which I overlooked when you first posted.

Given that, obviously having nBody running MT (especially more than one of them) when other tasks are running doesn't do much of anything to improve the overall efficiency of any of them.

That being said, the upside is 6.12.34 seems to be immune to what appears to be the number one failure for Windows hosts which can get the app to run at all.

Since this really is a beta test run for nBody, my suggestion would be to just leave well enough alone for now unless you are really comfortable with doing hand to hand combat with the BOINC anonymous platform (and all of its quirks) and willing to devote some time to more closely monitoring what the host is doing.

You know, sort of like the doctor's creed, "First and Foremost, Do No Harm". ;-)
ID: 56852 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56857 - Posted: 13 Jan 2013, 1:40:18 UTC - in response to Message 56852.  

As I indicated earlier .. I'm a newbie at M@H I don't see a reference to a "Richard" in the intro web description. Who is he?

As long as I'm at it ...

Where can I get more "theory of operation" info etc. for the various M@H programs?

Ed F
ID: 56857 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 56858 - Posted: 13 Jan 2013, 9:44:19 UTC - in response to Message 56857.  

As I indicated earlier .. I'm a newbie at M@H I don't see a reference to a "Richard" in the intro web description. Who is he?

I'm a volunteer like you. Alinator and I have worked together on various other BOINC projects, on and off, down the years. You can see some of our attempts to dissect this new application and get it to run as planned in the news threads, specifically Nbody 1.04
ID: 56858 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56881 - Posted: 14 Jan 2013, 13:25:17 UTC - in response to Message 56857.  

<snip>Where can I get more "theory of operation" info etc. for the various M@H programs?

Ed F


By doing what the rest of us do.

Leverage your existing knowledge, follow along in the various technically oriented project message boards, as well as the main BOINC project Wikis, boards, and mail lists.

Then ask questions when you see things you don't understand. ;-)
ID: 56881 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56892 - Posted: 15 Jan 2013, 14:07:28 UTC - in response to Message 56881.  
Last modified: 15 Jan 2013, 14:32:11 UTC

I see Nbody runs in MT mode if allowed BUT it only consumes 4 CPU's not all 8. Is that a trick of HT , a testing design limit, a consequence of running MT in "old" boinc non-MT mode, or is it a configuration prob on my part?

Ed F

[edit] ... ignore this ... looks like just a configuration thing of mine ...
ID: 56892 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56899 - Posted: 15 Jan 2013, 20:41:09 UTC - in response to Message 56892.  

O.K. I still don't know ... I have 5 CPU;'s free and nbody is only using 4 of them and 1 is idle ...

Ed F
ID: 56899 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56903 - Posted: 16 Jan 2013, 0:59:17 UTC - in response to Message 56899.  
Last modified: 16 Jan 2013, 1:13:14 UTC

Hmmm...

Try double checking your Computing Preferences again. There is two of them which can effect MT behavior. The 'Number of Processors' to Use, and 'Percentage of Processors' to Use on multiprocessor machines.

Yes I know... This is brilliantly confusing as to why you need two controls to manage essentially the same thing, but hey, what do I know? ;-)

Try setting the number of processors to the total number of cores (including virtual ones), and the percentage to 100 and see if that gets them all going.

<edit> IIRC, there is a limit to the number of processors (something like 32), but that would typically only impact or apply to large servers. If you set it to something higher than what you have in the machine it should just use them all. The default value used to be 4, but that could have changed since I went in and changed my prefs (way back in the dark ages). :-)
ID: 56903 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
EdwardPF

Send message
Joined: 8 Apr 10
Posts: 25
Credit: 268,525
RAC: 0
Message 56907 - Posted: 16 Jan 2013, 5:28:24 UTC - in response to Message 56903.  

The two settings are (currently) set at 50% (schedule 4 cores) and 100% (all the time).

The first one should be telling BOINC how many WU's to allow to run - In my case that means that 4 WU's were running ... one of them MT so I Should have seen 100% CPU usage IFF the MT WU could use 5 or more cores. I only saw 88% busy ... I.E three ST WU's and one 4 thread WU (the nbody).

I set boinc to accept no new WU's and after the three ST WU's finished and the one MT WU was running I set the "use at most" to 100% (which shouldn't have affected anything) ... I made sure the "nbody" WU had affinity set to "all" ... and sure enough ... the computer was still 50% busy.

My conclusion ... "nbody" uses 4 sub threads and no more.

(Of course I may be wrong)

Ed F
ID: 56907 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56911 - Posted: 16 Jan 2013, 11:05:40 UTC - in response to Message 56907.  

Hmmm...

Interesting.

Your interpretation about how the processor controls work may be correct (I've been out of the BOINC dev loop for a while), but I'm pretty sure you have to be running something higher than 6.12.34 for the CC to be able to use the percentage one. IIRC, that was being tested in some of the later 'alpha' 6x's but was earmarked as a 7x feature.

In any event, I don't have any hosts at the moment with more than 4 cores, and Richard isn't running 6x on his big Intel. So unless he already knows the answer, I guess we need input from someone with a host closer to what you have than my are. :-(
ID: 56911 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3315
Credit: 519,950,357
RAC: 21,982
Message 56918 - Posted: 16 Jan 2013, 12:27:10 UTC - in response to Message 56911.  
Last modified: 16 Jan 2013, 12:31:32 UTC

Hmmm...

Interesting.

Your interpretation about how the processor controls work may be correct (I've been out of the BOINC dev loop for a while), but I'm pretty sure you have to be running something higher than 6.12.34 for the CC to be able to use the percentage one. IIRC, that was being tested in some of the later 'alpha' 6x's but was earmarked as a 7x feature.

In any event, I don't have any hosts at the moment with more than 4 cores, and Richard isn't running 6x on his big Intel. So unless he already knows the answer, I guess we need input from someone with a host closer to what you have than my are. :-(


I am pretty sure the 50% is limiting his cores to only half of the total cores. I am using Boinc 7.0.42 on this laptop and have it too set to the same settings, not here at MW though, and it behaves like I said...it only uses UPTO 50% of the available cpu cores and uses them 100% of the time.
The settings say:
On multiprocessor systems, use at most [50] percent of the processors
Use at most [100] % CPU time

These were found in the Boinc Manager and I do not think this has changed over the versions. But it must be something in Boinc itself that says 'aha a multi-threaded workunit' and only crunches 3 units while still using all 4 cores, in his case.
ID: 56918 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56921 - Posted: 16 Jan 2013, 12:54:05 UTC - in response to Message 56918.  

Thanks for the input, Mikey.

Like I said I've been out of the loop for a while, and even then only got dragged into using 7x (kicking and screaming) recently, since I wasn't all that thrilled at the time with other aspects about the way it worked (still aren't, but that's a different story). ;-)
ID: 56921 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 56922 - Posted: 16 Jan 2013, 13:26:47 UTC

The wording on the BOINC preferences page for your account tells the story:

On multiprocessors, use at most: 100% of the processors
Enforced by version 6.1+

Because it's a BOINC (not project) preference, it applies globally across all projects, and has been available since (effectively) the transition from BOINC v5 to v6.
ID: 56922 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alinator

Send message
Joined: 7 Jun 08
Posts: 464
Credit: 56,639,936
RAC: 0
Message 56923 - Posted: 16 Jan 2013, 14:37:59 UTC - in response to Message 56922.  

LOL... Like I said you forget sooooo many details when you've been away! ;-)

So then if you set them to the total number of cores you have and/or with 100% utilization, then all of them should be doing something BOINC related unless there is some specific project, task, or user override in effect, correct?
ID: 56923 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 56924 - Posted: 16 Jan 2013, 14:52:31 UTC - in response to Message 56923.  

LOL... Like I said you forget sooooo many details when you've been away! ;-)

So then if you set them to the total number of cores you have and/or with 100% utilization, then all of them should be doing something BOINC related unless there is some specific project, task, or user override in effect, correct?

Well, it's a BOINC scheduling directive, but yes - BOINC will try to schedule tasks to add up to your total number of cores (plus a bit of allowed overcommit, to accommodate non compute intensive and GPU tasks).

But BOINC has no way of adapting to science applications which use more (or less) CPU power than the admins have declared in the <app_version> and <plan_class>.
ID: 56924 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : A simple OS process accounting error??

©2024 Astroinformatics Group