Welcome to MilkyWay@home

New runs of MilkyWay Nbody out

Message boards : News : New runs of MilkyWay Nbody out
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 542,901,527
RAC: 152,500
Message 68638 - Posted: 2 May 2019, 7:30:49 UTC - in response to Message 68634.  

Yikes! A lot of the older server configurations for MilkyWay@home preferences were dropped. Hope you kept the older server code configuration files around for reference.
ID: 68638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68647 - Posted: 2 May 2019, 19:00:08 UTC - in response to Message 68638.  

I'm browsing through the old code used in the previous server, however, I cannot seem to find the old preference settings there either. Aside from updating the code to a new version of PHP, there aren't any visible differences that I can find between the project preferences between the old and new server code. How long ago were the preferences to opt out of Nbody or Separation last available? I apologize that this is taking much longer than I anticipated.
-Eric
ID: 68647 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 542,901,527
RAC: 152,500
Message 68651 - Posted: 2 May 2019, 19:47:19 UTC - in response to Message 68647.  
Last modified: 2 May 2019, 19:50:28 UTC

I'm positive the choices were there before the server code update. I would say a configure file did not get included when the new server code was compiled and installed.

I thought of using the Wayback Machine but the only snapshot is from yesterday and from 30 January. That one would have been before the server update I think. But of course the project was unavailable with the database offline on that date.

Could you ask for help from the BOINC developers on their forum?
https://boinc.berkeley.edu/forum_forum.php?id=5

or the BOINC developers list
https://groups.google.com/a/ssl.berkeley.edu/forum/#!forum/boinc_projects
ID: 68651 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
marmot
Avatar

Send message
Joined: 12 Dec 15
Posts: 53
Credit: 132,581,364
RAC: 35,130
Message 68655 - Posted: 2 May 2019, 21:45:06 UTC - in response to Message 68647.  
Last modified: 2 May 2019, 21:53:46 UTC

I'm browsing through the old code used in the previous server, however, I cannot seem to find the old preference settings there either. Aside from updating the code to a new version of PHP, there aren't any visible differences that I can find between the project preferences between the old and new server code. How long ago were the preferences to opt out of Nbody or Separation last available? I apologize that this is taking much longer than I anticipated.
-Eric



Newer BOINC server preferences coding offers pull-down menu choices for maximum number of units to download and desired number of cores for MT work units.
See preferences over at LHC@home or Cosmology@home for examples.

I also remember being able to opt out of N body work units from sometime after the 280x arrived... early April.

All work unit varieties should be optional.

To deal with guaranteed N Body coming at a machine that had opted out, won't we now have to use an app_config with:

<app_version>
<app_name>milkyway_nbody</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>1.0</avg_ncpus>
<cmdline>--nthreads 1</cmdline>
</app_version>


to reduce core count to 1 (or 2)?
ID: 68655 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 588
Credit: 18,915,455
RAC: 5,069
Message 68659 - Posted: 3 May 2019, 7:04:01 UTC - in response to Message 68647.  

Eric Mendelsohn wrote:
How long ago were the preferences to opt out of Nbody or Separation last available?

They were available until the day when the server software was upgraded.


marmot wrote:
To deal with guaranteed N Body coming at a machine that had opted out, won't we now have to use an app_config with:
(...)
to reduce core count to 1 (or 2)?

No, the new n-Body application is single core anyway.
ID: 68659 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
marmot
Avatar

Send message
Joined: 12 Dec 15
Posts: 53
Credit: 132,581,364
RAC: 35,130
Message 68665 - Posted: 3 May 2019, 17:29:20 UTC - in response to Message 68659.  
Last modified: 3 May 2019, 17:29:57 UTC

No, the new n-Body application is single core anyway.


Without preferences choice menus available, BOINC will default to 1 thread and if we want MT then we'll have to use our own app_config.xml to choose thread counts.
That's more work for the end user and a step backwards...

client_state.xml still showed both 8 thread and 1 thread even after updating Milkyway in BOINCMgr
<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>154</version_num>
<platform>windows_x86_64</platform>
<avg_ncpus>8.000000</avg_ncpus>
<max_ncpus>8.000000</max_ncpus>
<flops>4779684632.114775</flops>
<plan_class>mt</plan_class>
<api_version>6.13.0</api_version>
<cmdline>--nthreads 8</cmdline>
<file_ref>
<file_name>milkyway_nbody_1.54_windows_x86_64__mt.exe</file_name>
<main_program/>
</file_ref>
</app_version>

<app_version>
<app_name>milkyway_nbody</app_name>
<version_num>154</version_num>
<platform>windows_x86_64</platform>
<avg_ncpus>1.000000</avg_ncpus>
<max_ncpus>1.000000</max_ncpus>
<flops>3732961173.280515</flops>
<api_version>6.13.0</api_version>
<file_ref>
<file_name>milkyway_nbody_1.54_windows_x86_64.exe</file_name>
<main_program/>
</file_ref>
</app_version>


EDIT: Resetting the project made those entries disappear.
ID: 68665 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68671 - Posted: 3 May 2019, 20:58:47 UTC - in response to Message 68665.  

The main reason we are not running MT applications for milkyway_nbody anymore is because the progenitor dwarf galaxy generation is still a single-threaded process. Let's say you you wanted to run 8 cores on the multi-threaded version of milkyway_nbody. While the progenitor is being generated by a single core, the other seven sit idly by until the progenitor is created and placed in the simulation, which takes a considerable amount of time when using a two-component Plummer sphere. In that time creating the dwarf galaxy, the other seven cores could be creating progenitors for seven other single-threaded runs. Because of this, Jake and I agreed that only running single-threaded nbody applications would be a more efficient use of everyone's time and computing power.

On the topic re-adding the "opt out" preferences for Nbody and Separation, I came across this code among the old server's include files:

if ($app_types->cpu) {
    $project_pref_descs[] = new PREF_BOOL (
        tra("Use CPU")
            ."<br><span class=note>"
            .tra("Enforced by version 6.10+")
            ."</span>",
        "no_cpu",
        false,
        true
    );
}
if ($app_types->ati) {
    $project_pref_descs[] = new PREF_BOOL (
        tra("Use ATI GPU")
            ."<br><span class=note>"
            .tra("Enforced by version 6.10+")
            ."</span>",
        "no_ati",
        false,
        true
    );
}
if ($app_types->cuda) {
    $project_pref_descs[] = new PREF_BOOL (
        tra("Use NVIDIA GPU")
            ."<br><span class=note>"
            .tra("Enforced by version 6.10+")
            ."</span>",
        "no_cuda",
        false,
        true
    );
}
if ($app_types->intel_gpu) {
    $project_pref_descs[] = new PREF_BOOL (
        tra("Use Intel GPU")
            ."<br><span class=note>"
            .tra("Enforced by version 7.2+")
            ."</span>",
        "no_intel_gpu",
        false,
        true
    );
}


Does this look the like place where the Nbody opt-out preference should be located? I've searched this and many other files, but I still cannot find the original code that generated that preference in the old server. If I cannot find the old preferences, I will need to recreate them from scratch. This code will at least give me some place to start.

Again, thank you for all of your help.

-Eric
ID: 68671 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 542,901,527
RAC: 152,500
Message 68674 - Posted: 4 May 2019, 0:27:18 UTC - in response to Message 68671.  
Last modified: 4 May 2019, 0:28:11 UTC

That's the standard code snippet for choosing which gpu type to use on a project. That isn't the code snippet for choosing the type of cpu apps to run. This is an example of what the project preferences looks like typically from GPUGrid.

There normally is a section below the gpu type choice which allows selection of the various cpu and gpu applications to run also.


Resource share 
Determines the proportion of your computer's resources allocated to this project. Example: if you participate in two BOINC projects with resource shares of 100 and 200, the first will get 1/3 of your resources and the second will get 2/3.	25
Use CPU 
Enforced by version 6.10+	no
Use ATI GPU 
Enforced by version 6.10+	no
Use NVIDIA GPU 
Enforced by version 6.10+	yes
Run test applications? 
This helps us develop applications, but may cause jobs to fail on your computer	no
Is it OK for GPUGRID and your team (if any) to email you?	yes
Should GPUGRID show your computers on its web site?	yes
Default computer location	---
Maximum CPU % for graphics
0 ... 100	20
Run only the selected applications	ACEMD short runs (2-3 hours on fastest card): yes
ACEMD long runs (8-12 hours on fastest GPU): yes
ACEMD Beta: no
Quantum Chemistry (CPU): no
Quantum Chemistry (CPU, beta): no
Python Runtime : no
If no work for selected applications is available, accept work from other applications?	no
Use Graphics Processing Unit (GPU) if available	yes
Use Central Processing Unit (CPU)	yes

ID: 68674 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
marmot
Avatar

Send message
Joined: 12 Dec 15
Posts: 53
Credit: 132,581,364
RAC: 35,130
Message 68677 - Posted: 4 May 2019, 2:59:10 UTC - in response to Message 68671.  

While the progenitor is being generated by a single core, the other seven sit idly by until the progenitor is created and placed in the simulation, which takes a considerable amount of time when using a two-component Plummer sphere.



considerable amount of time

Just curious, what percentage of an average total run time would the progenation process take? YAFU 16t/32t would take about a couple hours of single thread database work during a 120-180 hours run; which I didn't mind. 1% - 2% is tolerable. Albeit, in the winter I overbook 4 of 32 cores to protect heat generation.

Thanks for taking time to give an explanation about your decision process.
ID: 68677 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68685 - Posted: 4 May 2019, 8:48:11 UTC - in response to Message 68677.  

Since the times for an nbody simulation can vary drastically depending on the 5 input parameters for our dwarf galaxy, it's hard for me to pin a percentage for dwarf progenitor generation. However, for the two-component 40,000-body progenitors we are currently running in our simulations, I want to say it takes about 1-3 minutes to generate the progenitor. So, if you were to run eight 8-threaded processes one at a time versus eight single-threaded processes simultaneously, the latter process would on average finish those eight runs 14 minutes faster than the former. But considering that these runs take between 3 and 24 hours to time evolve the system, an argument could be made that this time percentage for dwarf generation is insignificant. But since it takes about a month for runs to converge, those additional minutes could give us results a few days earlier.

Thank you for taking interest. If you have any other questions, please feel free to ask.

-Eric
ID: 68685 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile TimeRanger

Send message
Joined: 31 Oct 10
Posts: 83
Credit: 38,632,375
RAC: 0
Message 68700 - Posted: 6 May 2019, 10:56:12 UTC - in response to Message 68647.  

I'm browsing through the old code used in the previous server, however, I cannot seem to find the old preference settings there either. Aside from updating the code to a new version of PHP, there aren't any visible differences that I can find between the project preferences between the old and new server code. How long ago were the preferences to opt out of Nbody or Separation last available? I apologize that this is taking much longer than I anticipated.
-Eric

I honestly don't know - I stopped running N-Body when they were MT and taking days for this old machine to run - it was at least a year ago that i opted out.
ID: 68700 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68703 - Posted: 7 May 2019, 19:18:21 UTC - in response to Message 68674.  

Thanks for GPUGrid preferences, Mr. Myers. It really helped. I believe I've found the main problem with why the preferences were lost. In the project_specific_prefs.inc file used in BOINC, there is a line of code which is necessary to show the preferences:
define ('APP_SELECT_PREFS', true);

When we were copying over the files to the new server, this line and a few others like it were omitted. After pasting the missing "define" lines back in the inc file, I loaded up the project preferences site to see if it was fixed, however, the preferences were still not visible to anyone. I feel like there's a step I'm missing when updating the website code. Does anyone have any suggestions?

-Eric
ID: 68703 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 542,901,527
RAC: 152,500
Message 68704 - Posted: 7 May 2019, 20:24:06 UTC - in response to Message 68703.  

Other than asking for help from other administrators like I suggested in https://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=4439&postid=68651
I can't offer much further help as I have never compiled the server code before, just the client and manager.
ID: 68704 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile JohnMD
Avatar

Send message
Joined: 11 Jul 08
Posts: 13
Credit: 10,015,444
RAC: 0
Message 68706 - Posted: 8 May 2019, 0:18:02 UTC - in response to Message 68703.  

Thanks for GPUGrid preferences, Mr. Myers. It really helped. I believe I've found the main problem with why the preferences were lost. In the project_specific_prefs.inc file used in BOINC, there is a line of code which is necessary to show the preferences:
define ('APP_SELECT_PREFS', true);

When we were copying over the files to the new server, this line and a few others like it were omitted. After pasting the missing "define" lines back in the inc file, I loaded up the project preferences site to see if it was fixed, however, the preferences were still not visible to anyone. I feel like there's a step I'm missing when updating the website code. Does anyone have any suggestions?

-Eric

Hello Eric.
40 years ago I was a system programmer charged with a local upgrade of IBM VM.
We had a range of changes and additions to the previous version of the operating system - particularly resource accounting.
One did not change the supplied application/operating system; one created source update files(add/del/repl)
All these update files were checked and - if necessary - adjusted to the new upgrade.
40 years ago there was no recognised IT education - have we learnt anything since ?
ID: 68706 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Henk Haneveld

Send message
Joined: 15 Aug 14
Posts: 6
Credit: 1,619,718
RAC: 269
Message 68708 - Posted: 8 May 2019, 11:28:58 UTC - in response to Message 68703.  

Thanks for GPUGrid preferences, Mr. Myers. It really helped. I believe I've found the main problem with why the preferences were lost. In the project_specific_prefs.inc file used in BOINC, there is a line of code which is necessary to show the preferences:
define ('APP_SELECT_PREFS', true);

When we were copying over the files to the new server, this line and a few others like it were omitted. After pasting the missing "define" lines back in the inc file, I loaded up the project preferences site to see if it was fixed, however, the preferences were still not visible to anyone. I feel like there's a step I'm missing when updating the website code. Does anyone have any suggestions?

-Eric

Is this what you are looking for
https://boinc.berkeley.edu/trac/wiki/AppFiltering
ID: 68708 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68712 - Posted: 9 May 2019, 15:16:31 UTC

Great news, everyone! The app selection preferences are available again in the Project Preferences! You can now choose to opt out of Nbody or Separation. Thank you all for your help in getting this feature back!
-Eric
ID: 68712 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 708
Credit: 542,901,527
RAC: 152,500
Message 68713 - Posted: 9 May 2019, 20:46:30 UTC - in response to Message 68712.  

Thanks Eric for the effort to keep digging into the code to fix the preferences the way they were before the upgrade.
ID: 68713 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
marmot
Avatar

Send message
Joined: 12 Dec 15
Posts: 53
Credit: 132,581,364
RAC: 35,130
Message 68715 - Posted: 10 May 2019, 9:50:44 UTC - in response to Message 68712.  

Great news, everyone! The app selection preferences are available again in the Project Preferences! You can now choose to opt out of Nbody or Separation. Thank you all for your help in getting this feature back!
-Eric


Good job!
Thanks
ID: 68715 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile TimeRanger

Send message
Joined: 31 Oct 10
Posts: 83
Credit: 38,632,375
RAC: 0
Message 68725 - Posted: 13 May 2019, 9:50:24 UTC - in response to Message 68712.  

Great news, everyone! The app selection preferences are available again in the Project Preferences! You can now choose to opt out of Nbody or Separation. Thank you all for your help in getting this feature back!
-Eric


Right after you posted that, I opted out of N-Body, updated Boinc/Milkyway but for some reason, I am still receiving N-Bodys ??
ID: 68725 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Mendelsohn
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 21 Aug 18
Posts: 59
Credit: 5,350,675
RAC: 0
Message 68726 - Posted: 13 May 2019, 16:02:02 UTC - in response to Message 68725.  

Right after you posted that, I opted out of N-Body, updated Boinc/Milkyway but for some reason, I am still receiving N-Bodys ??


I believe that's because your queue still has nbody workunits. I think if you give it a few days to finish the nbodies in the queue, you will not receive anymore workunits of that type. I advise that you do not abort the nbody workunits currently in your queue, as that will deny credit to those who are working on tasks validating said workunit. If you are still receiving nbody workunits after a week, please let me know.

-Eric
ID: 68726 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · Next

Message boards : News : New runs of MilkyWay Nbody out

©2024 Astroinformatics Group