Welcome to MilkyWay@home

Admin Updates Discussion

Message boards : News : Admin Updates Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

AuthorMessage
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 76967 - Posted: 9 Mar 2024, 16:52:18 UTC

@Kevin: will the new N-Body Simulation with Orbit Fitting replace the old N-Body Simulation? As far as I see in the results it's calculating <search_likelihood>, <search_likelihood_EMD>, <search_likelihood_Mass> and <search_likelihood_Beta> like the old app did and adds on top of that <search_likelihood_BetaAvg>, <search_likelihood_VelAvg> and <search_likelihood_Dist>. So the old app does not seem necessary unless it's working on completely different data.
ID: 76967 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevin Roux
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 9 Aug 22
Posts: 69
Credit: 1,155,948
RAC: 8,038
Message 76969 - Posted: 10 Mar 2024, 22:05:09 UTC - in response to Message 76967.  

@Kevin: will the new N-Body Simulation with Orbit Fitting replace the old N-Body Simulation? As far as I see in the results it's calculating , , and like the old app did and adds on top of that , and . So the old app does not seem necessary unless it's working on completely different data.


The new app is running the newest version of our n-body code. We want to fit the orbit of the Orphan-Chenab Stream that a past grad student had optimized using MW@home before. The old app currently has no active runs but still has all those workunits that need to be crunch. The old application is running a completely different set of data as it is fitting Palomar 5. There will be future runs that will not be fitting orbits but looking at the affects of self-interacting dark matter on streams. The reason we made a new application is because the new code had changes to the way it calculates likelihood scores so it would be unable to validate the current runs on the old application. Making the new application was just the easiest way to get new runs up without affecting the old ones and making sure those workunits that needed validation would get validated.
ID: 76969 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 76981 - Posted: 23 Mar 2024, 20:13:20 UTC
Last modified: 23 Mar 2024, 20:19:37 UTC

Regarding the new application, perhaps you need to check how it handles "Number of particles in bins is very small compared to total. (0 << 40000). Skipping distance calculation". The old application finished such WU in one second or so, the new one runs it for many hours and as far as I can tell, there's no useful result from all that computing, the likelihood values are always identical. Like this one for example (WU 968109276):

<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
mber of particles in bins is very small compared to total. (0 << 40000). Skipping distance calculation
Running MilkyWay@home Nbody v1.86

(...)

Number of particles in bins is very small compared to total. (0 << 40000). Skipping distance calculation
Running MilkyWay@home Nbody v1.86
Number of particles in bins is very small compared to total. (0 << 40000). Skipping distance calculation
<search_likelihood>-9999999.900000000372529</search_likelihood>
<search_likelihood_EMD>-9999999.900000000372529</search_likelihood_EMD>
<search_likelihood_Mass>-0.000000000000000</search_likelihood_Mass>
<search_likelihood_Beta>-0.000000000000000</search_likelihood_Beta>
<search_likelihood_BetaAvg>-0.000000000000000</search_likelihood_BetaAvg>
<search_likelihood_VelAvg>-0.000000000000000</search_likelihood_VelAvg>
<search_likelihood_Dist>-0.000000000000000</search_likelihood_Dist>
strftime() failed called boinc_finish(0)

</stderr_txt>
]]>

This WU used 16 hours 46 min 19 sec CPU time on an Core i3-12100 and 2 days 7 hours 19 min 59 sec CPU time on an Core i3-3240. Have seen few more such WUs with different run times, they are always from "data__5" batch, for example also WU 968218051 or WU 968168663.
ID: 76981 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Marcin
Avatar

Send message
Joined: 18 Jun 09
Posts: 35
Credit: 11,219,028
RAC: 10,910
Message 76982 - Posted: 24 Mar 2024, 13:21:24 UTC

i assume there are still no plans as of today to support the Apple silicon arm architecture ?
ID: 76982 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 76990 - Posted: 30 Mar 2024, 11:47:58 UTC

Kevin Roux wrote:
Working on
- Will look into getting number of CPUs used as an option in Preferences
When looking into it, perhaps you can also check if it's possible to send the single thread application in case someone chooses to run Milkyway on a single core. I have not tested it, but probably it will run faster than the MT application forced to run on a single core. At least in theory it should. I mean, you compiled a single core version, it's there, there's just no way to get it automatically from the server. Or maybe a selection between the ST app and the MT app could be a simple way to get at least some options in case making it possible to select any number of cores needs too much time.
ID: 76990 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevin Roux
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 9 Aug 22
Posts: 69
Credit: 1,155,948
RAC: 8,038
Message 76997 - Posted: 2 Apr 2024, 19:14:51 UTC - in response to Message 76982.  

i assume there are still no plans as of today to support the Apple silicon arm architecture ?


It is something I have planned for the summer when I get a MacBook so I can build/test the binary for Apple support.
These are not solid plans however. There are always things that come up that might have to take priority.
ID: 76997 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevin Roux
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 9 Aug 22
Posts: 69
Credit: 1,155,948
RAC: 8,038
Message 76998 - Posted: 2 Apr 2024, 19:16:44 UTC - in response to Message 76990.  

Kevin Roux wrote:
Working on
- Will look into getting number of CPUs used as an option in Preferences
When looking into it, perhaps you can also check if it's possible to send the single thread application in case someone chooses to run Milkyway on a single core. I have not tested it, but probably it will run faster than the MT application forced to run on a single core. At least in theory it should. I mean, you compiled a single core version, it's there, there's just no way to get it automatically from the server. Or maybe a selection between the ST app and the MT app could be a simple way to get at least some options in case making it possible to select any number of cores needs too much time.


Thank you for the information. I wasn't sure if this was something people wanted but will look into implementing that was well when I work on implementing choosing the number of cores more easily.
ID: 76998 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GWGeorge007
Avatar

Send message
Joined: 6 Jan 18
Posts: 6
Credit: 69,153,963
RAC: 83,767
Message 77008 - Posted: 4 Apr 2024, 19:43:15 UTC
Last modified: 4 Apr 2024, 19:49:13 UTC

My MT tasks for Orbit Fitting are showing errors and invalids a lot more recently, almost 1:1. Any particular reason? I'm about to switch off Milkyway and do something else.
[/url]
George
ID: 77008 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 543,112,373
RAC: 143,585
Message 77009 - Posted: 4 Apr 2024, 19:46:46 UTC - in response to Message 77008.  

Sure, go ahead. Everyone is in the same boat.

Haven't received any response from admin Kevin either when I alerted him to the issue early yesterday.

The tasks are slowly being eliminated from issuance again after hitting the max 4 errors allowed.
ID: 77009 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 543,112,373
RAC: 143,585
Message 77011 - Posted: 4 Apr 2024, 22:25:39 UTC - in response to Message 77009.  

You can also just switch off Orbit-Fitting tasks in Preferences. The normal N-body tasks are fine.
ID: 77011 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alanb1951

Send message
Joined: 16 Mar 10
Posts: 210
Credit: 105,914,814
RAC: 25,174
Message 77013 - Posted: 5 Apr 2024, 4:04:14 UTC
Last modified: 5 Apr 2024, 4:28:46 UTC

I've not checked every failed Orbit task, but the large number I did check were all from the 03_29_2024 data feed and all seemed to have task names without "orbit_fitting" -- oops! :-)

The same seems to be the case for the 04_03_2024 data set... However, I am still processing retries for earlier, valid, tasks

Cheers - Al.

[Edited to fix a date typo (I often mess up U.S.-style dates...)]

[Edited again to note that I still get some older, viable, tasks...]
ID: 77013 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GWGeorge007
Avatar

Send message
Joined: 6 Jan 18
Posts: 6
Credit: 69,153,963
RAC: 83,767
Message 77049 - Posted: 8 Apr 2024, 4:54:03 UTC - in response to Message 77013.  

I've not checked every failed Orbit task, but the large number I did check were all from the 03_29_2024 data feed and all seemed to have task names without "orbit_fitting" -- oops! :-)

The same seems to be the case for the 04_03_2024 data set... However, I am still processing retries for earlier, valid, tasks

Cheers - Al.

[Edited to fix a date typo (I often mess up U.S.-style dates...)]

[Edited again to note that I still get some older, viable, tasks...]

Hi alanb1951,
Please let us know when you have resolved the errors and we can begin testing/BOINC'ing the "orbit_fitting" tasks again.
Thanks!
George
ID: 77049 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 13 Oct 21
Posts: 44
Credit: 225,160,630
RAC: 7,713
Message 77060 - Posted: 13 Apr 2024, 6:21:26 UTC

How does one prevent the single-thread executable from downloading and running? I have preferences set to run both applications with no limit on CPUs and Jobs. I get a mix of single and multi-threaded tasks.
ID: 77060 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Sutaru Tsureku

Send message
Joined: 30 Apr 09
Posts: 100
Credit: 29,871,882
RAC: 757
Message 77061 - Posted: 13 Apr 2024, 10:28:25 UTC
Last modified: 13 Apr 2024, 10:36:16 UTC

I guess the following settings should be for just the multi thread app on a 4 thread CPU:

Maximale Anzahl Aufgaben 1 (max. tasks)
Maximale Anzahl CPUs 4 (max. threads)

I test this now.
I was surprised as I looked in BOINC that there is now a new single thread app. And I never asked for it. ;)

The standard settings should be max. tasks 1 and max. threads no limit and not at both no limit.


EDIT: Oh I saw now. this is my 100. forum message. :D
ID: 77061 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 77062 - Posted: 13 Apr 2024, 13:37:57 UTC - in response to Message 77061.  
Last modified: 13 Apr 2024, 13:54:00 UTC

I guess the following settings should be for just the multi thread app on a 4 thread CPU:

Maximale Anzahl Aufgaben 1 (max. tasks)
Maximale Anzahl CPUs 4 (max. threads)

I test this now.
Is it working as expected? Because "Max # of jobs for this project" isn't very specific, is it like max_concurrent in the app_info.xml or is it max task on the computer from this project, i.e. something like WCG has for each of their subprojects? Also "Max # of CPUs for this project" is pretty much wrong if it's actually "Max # of threads per task for this project".


The standard settings should be max. tasks 1 and max. threads no limit and not at both no limit.
The standard settings should be no limit for number of max. concurrent tasks and 16 for number of threads, i.e. what we had until now, than no one would complain about changes they didn't ask for. ;-)

And than of course, the ST application should only be send to users, who set 1 as limit for threads (or have a single core CPU or allow BOINC to use just one core) and everyone else should get only MT application. The issue with the current implementation is that it's "max" x threads and not like for example on PrimeGrid "exactly" x threads.

On my new Ryzen with 16 threads, on which I set "--nthreads 2" in the app_info.xml, so far BOINC decided to skip all single thread tasks and when it finishes a 2-core task, it starts again a 2-core tasks instead of 2 single core tasks which actually should run first. Let's see what it will do when it runs out of 2 core tasks.
ID: 77062 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Magenta

Send message
Joined: 14 Oct 23
Posts: 3
Credit: 71,127
RAC: 1,501
Message 77063 - Posted: 13 Apr 2024, 14:09:32 UTC

I think for simplicity we should assume that people want either single threaded only or multi threaded only, possibly with a particular number of threads. Moreover, is the single threaded approach so different that we even need a separate application for that?

I'm with the single threaded group and all is well after I got rid of the mt stuff.

The defaults for these new project specific options should be unlimited in both cases as they are now. Of course the global limits still apply.
ID: 77063 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 77064 - Posted: 13 Apr 2024, 15:45:46 UTC - in response to Message 77063.  
Last modified: 13 Apr 2024, 15:51:22 UTC

I think for simplicity we should assume that people want either single threaded only or multi threaded only, possibly with a particular number of threads.
Yes, that's exactly what we have asked for.


Moreover, is the single threaded approach so different that we even need a separate application for that?
In theory it should give better performance for those who want just one thread per task and since they compiled it, they might as well send it to those who want just one thread per task.

Meanwhile my BOINC client decided to start crunching the single core tasks... sometimes one is "waiting to run" in case one 2-threads-task is started after finishing one single-core-task. Not as huge issue as some have observed in the past when trying to mix Milkyway's 16-thread tasks with other projects.
ID: 77064 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 77065 - Posted: 13 Apr 2024, 16:08:04 UTC - in response to Message 77062.  
Last modified: 13 Apr 2024, 16:19:10 UTC

Because "Max # of jobs for this project" isn't very specific, is it like max_concurrent in the app_info.xml or is it max task on the computer from this project, i.e. something like WCG has for each of their subprojects? Also "Max # of CPUs for this project" is pretty much wrong if it's actually "Max # of threads per task for this project".
Just checked, Primegrid is using the words "Max # of simultaneous PrimeGrid tasks" and "Multi-threading: Max # of threads for each task" for those two and AFAIK the applications use fewer threads only on machines, which don't have or are not allowed to use the amount specified as maximum.
They have however only MT applictions and send them also to those who allow only one thread per task, maybe it's not possible to stop the BOINC server to send the ST app to those who allow more threads per task? Until now it was not possible here to get the MT application if BOINC was only allowed to use one CPU core as reported here.
When looking into the difference in instructions between the Linux and Windows executable, perhaps it's not a bad idea to check wether the single thread application is indeed faster than the MT running on a single core once all modern instruction sets are in use. If it's not, than simply let the MT app run on one core if the user wants that.
ID: 77065 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,923,609
RAC: 4,847
Message 77066 - Posted: 13 Apr 2024, 18:39:42 UTC - in response to Message 77064.  

Meanwhile my BOINC client decided to start crunching the single core tasks... sometimes one is "waiting to run" in case one 2-threads-task is started after finishing one single-core-task. Not as huge issue as some have observed in the past when trying to mix Milkyway's 16-thread tasks with other projects.
OK, now there are 4 ST tasks waiting to run while 8 2-thread tasks are running. Doesn't make sense at all.
ID: 77066 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
xii5ku

Send message
Joined: 1 Jan 17
Posts: 32
Credit: 98,549,956
RAC: 308,125
Message 77067 - Posted: 14 Apr 2024, 7:17:27 UTC - in response to Message 77066.  
Last modified: 14 Apr 2024, 7:21:18 UTC

Link wrote:
now there are 4 ST tasks waiting to run while 8 2-thread tasks are running. Doesn't make sense at all.
In my experience, the BOINC client often resorts to put various tasks into waiting state — or starts tasks from the queue of downloaded tasks in an order which is unexpected to the user — whenever it is faced with a mixture of tasks with various thread counts in its buffer. Obviously the client has to have a scheduling algorithm which aims among else for good host utilization. The outcome of this algorithm may not always be what the user believes to be expected or optimal.

--------

On the new settings: This is what I *think* what they do:

"Max # jobs" — means "limit of tasks in progress" (or more precisely said, limit of MilkyWay@Home tasks in progress)

"Max # CPUs" — means "thread count per task" (or more precisely said, thread count per MilkyWay@Home task)

"Max # CPUs = "No limit" — The server assigns a mixture of tasks for the single-threaded app_version and for the multithreaded app_version to the host. The choice of thread count of MT tasks is left to the application. (The MT app_version behaves as before the introduction of the Max # CPUs preferences setting). Standard(?) BOINC server behaviour is to watch the computing performance of each app_version on the host, and from some point on assign only tasks of the better performing app_version to the host.

"Max # CPUs = 1" — The server assigns only tasks for the single-threaded app_version to the host.

"Max # CPUs = 2…256" — The server assigns only tasks for the multi-threaded app_version to the host. Furthermore, the application's built in upper limit of thread count per task is overridden by this setting.

(Again, that's only what I *think* how it works.)
On top of that, the user can still override the thread count of the multithreaded app_version by means of app_config.xml, or/and enforce the use of desired application binaries by means of app_info.xml.
ID: 77067 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next

Message boards : News : Admin Updates Discussion

©2024 Astroinformatics Group