Welcome to MilkyWay@home

Setting up multiple instances of BOINC in Windows

Message boards : Number crunching : Setting up multiple instances of BOINC in Windows
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73766 - Posted: 1 Jun 2022, 16:51:03 UTC

I would like to experiment with this to see if it's possible to run GPU separation with no CPU separation, on the same computer. I have a hunch this is possible with multiple instances.
https://www.overclock.net/threads/guide-setting-up-multiple-boinc-instances.1628924/
The link, from April 2017, gives guidance on this, but requires editing the cc_config.xml file. Sadly, I don't have any of those. With all the boinc upgrades since then, is there a stunt double file that replaced it?

Some of the web discussions mention hundreds of boinc clients, on multiple computers, running many projects. And also bunkering.

My much more simplified goal is to dedicate one client to each of these MW classes: n body CPU, NVIDIA GPU separation, and AMD GPU separation.

We have over 10,000 users in the last 24 hours, so probably at least one of you is doing this, in a Windows 10 environment. Any success stories out there?
ID: 73766 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
captainjack

Send message
Joined: 22 Jun 13
Posts: 44
Credit: 64,258,609
RAC: 0
Message 73767 - Posted: 1 Jun 2022, 20:21:54 UTC - in response to Message 73766.  

HRFMguy,

It's pretty easy to create a cc_config.xml file using these instructions.

https://boinc.berkeley.edu/wiki/client_configuration

At one time, I followed the overclock instructions to run 2 BOINC clients and they work like a charm.

Best of luck.
ID: 73767 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73768 - Posted: 1 Jun 2022, 23:23:00 UTC - in response to Message 73766.  
Last modified: 1 Jun 2022, 23:28:22 UTC

I would like to experiment with this to see if it's possible to run GPU separation with no CPU separation, on the same computer. I have a hunch this is possible with multiple instances.
https://www.overclock.net/threads/guide-setting-up-multiple-boinc-instances.1628924/
The link, from April 2017, gives guidance on this, but requires editing the cc_config.xml file. Sadly, I don't have any of those. With all the boinc upgrades since then, is there a stunt double file that replaced it?

Some of the web discussions mention hundreds of boinc clients, on multiple computers, running many projects. And also bunkering.

My much more simplified goal is to dedicate one client to each of these MW classes: n body CPU, NVIDIA GPU separation, and AMD GPU separation.

We have over 10,000 users in the last 24 hours, so probably at least one of you is doing this, in a Windows 10 environment. Any success stories out there?


If you do an <exclude_gpu> line it should work because then the project won't see your gpu as available to MIlkyway, if for instance you have an AMD and an Nvidia in the same machine you can even exclude either or both of them but then you are back to getting gpu tasks.

If you are interested in ALOT of credits for your gpu crunching them the math project Collatz pays the most creds per gpu task BY FAR over any other gpu project, but the SRBase TF tasks don't do too bad either. In Collatz you can even tweak the gpu to cut at least 1/3 off of the standard gpu times but be sure to use a zero resource buffer until you get it right as you could trash alot of tasks while testing.

I run this one here at MilkyWay but only turn on the n-body tasks:

<app_config>

<app>
<name>milkyway</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>0.5</cpu_usage>
</gpu_versions>
</app>
<app_version>
<app_name>milkyway_nbody</app_name>
<max_concurrent>6</max_concurrent>
<plan_class>mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<cmdline>--nthreads 2</cmdline>
</app_version>
</app_config>
ID: 73768 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 715
Credit: 556,845,053
RAC: 43,801
Message 73769 - Posted: 2 Jun 2022, 7:21:18 UTC
Last modified: 2 Jun 2022, 7:23:55 UTC

To create a default cc_config.xml is extremely easy with the Manager.

Just change one logging option here: Options >>Event Log Options and toggle on ANY option not already checked and Save.

This generates a fully populated cc_config.xml file in the BOINC data directory with the default settings.

Then either leave the non-standard logging option enabled or toggle it back off and Save it again.

I recommend using the sched_op_debug option as it should be a default in my opinion.

Then you can change this option in cc_config.xml

<allow_multiple_clients>0</allow_multiple_clients> to

<allow_multiple_clients>1</allow_multiple_clients>
ID: 73769 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73770 - Posted: 2 Jun 2022, 11:50:58 UTC - in response to Message 73769.  

To create a default cc_config.xml is extremely easy with the Manager.

Just change one logging option here: Options >>Event Log Options and toggle on ANY option not already checked and Save.

This generates a fully populated cc_config.xml file in the BOINC data directory with the default settings.

Then either leave the non-standard logging option enabled or toggle it back off and Save it again.

I recommend using the sched_op_debug option as it should be a default in my opinion.

Then you can change this option in cc_config.xml

<allow_multiple_clients>0</allow_multiple_clients> to

<allow_multiple_clients>1</allow_multiple_clients>


Thanks Keith I had no clue you needed to do that and it could be why when I've tried it a few times it never really worked for me!!
ID: 73770 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73771 - Posted: 2 Jun 2022, 17:28:29 UTC

Ok, got the cc_config setup, and tweaked per Keith. Boinc2 folder is created and got the expected folder and 12 files mentioned, per the overclock destructions. Got the boinc2 shortcut created and edited, with the newly created gui_rpc_auth.cfg password, again per the overclock destructions.

The issue I see now is that the boinc2 and original boinc managers are displaying identical tasks, progress, elapsed time, etc. Ditto the 2 event logs.

In the second boinc manager, I selected file>select computer and for the host name used DESKTOP-8HQ6IR0. This is the computer I'm running on. Password is blank.

Both boinc managers say connected to localhost(7.16.20)

If I change options>computing preferences in one, it also changes in the other. I don't think I have 2 unique clients, yet.

The original c:>programdata>boinc folder has 29 items. The new boinc2 folder only has 13 items. Is this ok?

Anybody have a clue? Pretty sure I followed the overclock destruction correctly.

Do I need a separate installation directory? This line confused me: Update the installation directory (if different) and the BOINC2 directory to wherever your BOINC folder is located and how it is named. BOINC2 if you're following along with me.
ID: 73771 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle

Send message
Joined: 12 Dec 11
Posts: 8
Credit: 292,327,420
RAC: 18,297
Message 73772 - Posted: 2 Jun 2022, 18:28:00 UTC - in response to Message 73771.  

Sometimes project servers are too "smart" when you attach an additional client from the same box that has already attached a client.
They merge both entries.

You need to force the server to create a fresh computer ID in it's DB.
In most cases it's enough to fake "<ncpus>x</ncpus>" in cc_config.xml.
Reload config files and contact the project from both clients.

Once you got a fresh host ID for one client you can remove the fake setting.
Check the local logs and the project's computer list for the new IDs.
ID: 73772 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73773 - Posted: 2 Jun 2022, 20:44:43 UTC

original ncpus was set to -1, then changed to 2, then 0, then 5. all 4 test cases allowed duplicate clients.

here is a snippet from the options section of cc_config.xml:
<options>
<allow_multiple_clients>1</allow_multiple_clients>
<max_file_xfers>10</max_file_xfers>
<max_file_xfers_per_project>10</max_file_xfers_per_project>
<ncpus>5</ncpus>
</options>

all other options were left unchanged.

I guess the real question is how to force the second boinc manager to get a new and different computer ID.

This link mentions that an 'out of sequence RPC number' will force a new ID in the server database. https://boinc.berkeley.edu/wiki/Host_identification_and_merging

There is also a create_account.xml in the original programdata>boinc folder, but not in the boinc2 folder. should it be copoied to the boinc2 folder???

local logs show the same computer ID.
ID: 73773 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 715
Credit: 556,845,053
RAC: 43,801
Message 73774 - Posted: 2 Jun 2022, 23:45:43 UTC - in response to Message 73773.  
Last modified: 2 Jun 2022, 23:58:18 UTC

The easiest way to get a new hostID is to edit the client_state.xml file for the 2nd computer that you want to be independent of the original computer.

The client_state file keeps track of the number of times the host contacts the project in the rpc_seqno value. Each project has a value for the number of times the host has contacted that project. You can see that value at the project also in the Host Details page.

Make sure you are editing the Milkyway section of the file.

Edit that value to some value LESS than the original host and that will make the projects puke a new hostID right quick.

For example change <rpc_seqno>86079</rpc_seqno> to <rpc_seqno>10</rpc_seqno>

and that will create a new hostID.

FYI the rpc_seqno value is also the way to update a newly configured or updated hardware host to keep the original hostID and creation date and all the credits assigned to the older original host.

You edit the client_state.xml file and change the hostID of all attached projects back to the original hostID's that you want to preserve. All you do is increase the values of rpc_seqno by at least one or more and the project will accept the newly configured host as the original host. Best to do this as soon as possible and not crunch much work on the new host as when you revert to the older hostID, you throw away all the credits on the new host you crunched.

This method is also the only way to preseve the orignal hostID when converting from Windows to Linux or vice versa. The traditional Merge function in BOINC is incapable of merging hosts with different OS'
It also always merges the older hostID into the newer hostID. It won't do the reverse unfortunately. Why the rpc_seqno method is the best way to achieve New Host >> Old Host.
ID: 73774 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 715
Credit: 556,845,053
RAC: 43,801
Message 73775 - Posted: 2 Jun 2022, 23:57:03 UTC
Last modified: 3 Jun 2022, 0:02:33 UTC

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

The only necessary files and folders in the copied BOINC directories are the 5 main executables. boinc, boincmgr,boinccmd, boincscr and switcher. Also the sub-folders should be created also, projects, docs, skins and locales. Don't have to put anything into them, the projects will fill them as necessary.

When the client is started up for the first time you will be presented with the standard initial welcome and asked to Join a project. Once you join a project, all the necesary support files will be pulled from the project server.

If you do this on the first cloned BOINC folder, you can simply zip it up and copy and expand it into the rest of the cloned BOINC's.

If I remember correctly I once had twenty cloned BOINC folders running the special app back at Seti during the Seti Germany WoW contest in past years.
ID: 73775 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73776 - Posted: 3 Jun 2022, 0:31:17 UTC - in response to Message 73774.  

Keith, do your comments apply to the case where you are running 2 clients on the same computer? My boinc2 client state is only 4 pages long, and does not have a project section, which is where the rpc_seqno resides. The original boinc client state.xml is 29 pages and does include the rpc_seqno.
ID: 73776 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73777 - Posted: 3 Jun 2022, 0:36:34 UTC - in response to Message 73775.  

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

The only necessary files and folders in the copied BOINC directories are the 5 main executables. boinc, boincmgr,boinccmd, boincscr and switcher. Also the sub-folders should be created also, projects, docs, skins and locales. Don't have to put anything into them, the projects will fill them as necessary.
This was not clear in the overclock instructions.


looks like our postings overlapped.
ID: 73777 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73778 - Posted: 3 Jun 2022, 0:46:33 UTC - in response to Message 73775.  

is switcher the same as boincsvcctrl.exe? or boinctray.exe. those are the only other executables in the program files>boinc folder.
ID: 73778 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73779 - Posted: 3 Jun 2022, 2:23:23 UTC - in response to Message 73775.  

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

If I remember correctly I once had twenty cloned BOINC folders running the special app back at Seti during the Seti Germany WoW contest in past years.


NO WONDER no one else ever won!!! :-))

Why is that faster than running one client on all the cpu's instead? Or did you run into task and/or other project limitations that this got around?
ID: 73779 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Oct 16
Posts: 167
Credit: 1,008,062,758
RAC: 155
Message 73780 - Posted: 3 Jun 2022, 2:29:14 UTC - in response to Message 73777.  
Last modified: 3 Jun 2022, 2:31:42 UTC

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

The only necessary files and folders in the copied BOINC directories are the 5 main executables. boinc, boincmgr,boinccmd, boincscr and switcher. Also the sub-folders should be created also, projects, docs, skins and locales. Don't have to put anything into them, the projects will fill them as necessary.
This was not clear in the overclock instructions.


looks like our postings overlapped.


Uhhh yes it was

Right in the command to start the client points to the different directory
"c:\Program Files\BOINC\boinc.exe" --allow_multiple_clients --redirectio --detach_console --gui_rpc_port 31418 --dir C:\ProgramData\BOINC2

The only files that I create when setting up instances are cc_config.xml (copied from the 1st client) and gui_rpc_auth.cfg (also copied) and the 2nd one isn't a requirement if you want to use the random password from BOINC.

Make BOINC 2/3/4/etc folder
Copy 1 or 2 files.
Run startup command

Thats all that is required to start a 2nd client.
ID: 73780 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 73781 - Posted: 3 Jun 2022, 2:29:31 UTC - in response to Message 73775.  

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

Don't have to put anything into them, the projects will fill them as necessary.

When the client is started up for the first time you will be presented with the standard initial welcome and asked to Join a project. Once you join a project, all the necesary support files will be pulled from the project server.

If you do this on the first cloned BOINC folder, you can simply zip it up and copy and expand it into the rest of the cloned BOINC's.


I keep all the projects on a usb stick, no zipping, and then just copy them into the Boinc folder before I attach to anything and that works for me, I keep one folder on the usb drive with all the projects and then have folders for Mac only projects and Android only projects because most projects don't have tasks for them. Yes the main folder has ie Gaia in it and it's Linux only so when I try to get tasks on a Windows pc it tells me NO. It saves me ALOT of typing to get all the different little known projects up and running if I want too.
ID: 73781 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73782 - Posted: 3 Jun 2022, 3:25:24 UTC - in response to Message 73780.  

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

The only necessary files and folders in the copied BOINC directories are the 5 main executables. boinc, boincmgr,boinccmd, boincscr and switcher. Also the sub-folders should be created also, projects, docs, skins and locales. Don't have to put anything into them, the projects will fill them as necessary.
This was not clear in the overclock instructions.


looks like our postings overlapped.


Uhhh yes it was

Right in the command to start the client points to the different directory
"c:\Program Files\BOINC\boinc.exe" --allow_multiple_clients --redirectio --detach_console --gui_rpc_port 31418 --dir C:\ProgramData\BOINC2

The only files that I create when setting up instances are cc_config.xml (copied from the 1st client) and gui_rpc_auth.cfg (also copied) and the 2nd one isn't a requirement if you want to use the random password from BOINC.

Make BOINC 2/3/4/etc folder
Copy 1 or 2 files.
Run startup command

Thats all that is required to start a 2nd client.
mmonnin, this is the contents of my batch file:
"c:\Program Files\BOINC\boinc.exe" --allow_multiple_clients --redirectio --detach_console --gui_rpc_port 31418 --dir C:\ProgramData\BOINC2
exit

it seems to match yours 1 for 1.

So it looks like i did do that correctly. What I missed was copying the cc_config.xml over to boinc2. Was that clear in the overclock instructions? I did get the gui_rpc_auth.cfg correct.
ID: 73782 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile HRFMguy

Send message
Joined: 12 Nov 21
Posts: 236
Credit: 575,038,236
RAC: 0
Message 73783 - Posted: 3 Jun 2022, 3:35:29 UTC

mmonnin,

Will this this multiple instances on boinc in windows do what i wanted to do in the original post?

"My much more simplified goal is to dedicate one client to each of these MW classes: n body CPU, NVIDIA GPU separation, and AMD GPU separation."

The issue is, I get unwanted CPU separation tasks when I run NVIDIA and AMD separation. Will your scheme stop that?
ID: 73783 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle

Send message
Joined: 12 Dec 11
Posts: 8
Credit: 292,327,420
RAC: 18,297
Message 73784 - Posted: 3 Jun 2022, 3:50:25 UTC - in response to Message 73783.  

The issue is, I get unwanted CPU separation tasks when I run NVIDIA and AMD separation.

If all 3 BOINC clients run fine on your box you need to attach CPU and GPU clients to different venues at the MW server.
Then set the venue options according to your needs, e.g. switch "use CPU" off for "GPU only" clients if you don't want "CPU separation" tasks.
ID: 73784 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 715
Credit: 556,845,053
RAC: 43,801
Message 73785 - Posted: 3 Jun 2022, 7:30:05 UTC - in response to Message 73779.  

Each running client should be maintained in its own directory. So only need to make multiple BOINC folders with all the normal default files. BOINC1, BOINC2, BOINC3, BOINC4 etc. etc.

If I remember correctly I once had twenty cloned BOINC folders running the special app back at Seti during the Seti Germany WoW contest in past years.


NO WONDER no one else ever won!!! :-))

Why is that faster than running one client on all the cpu's instead? Or did you run into task and/or other project limitations that this got around?

Hah ha hah LOL. I don't think I was the team member with the most clients running at one time. Think that honor went to Juan BFP.

We did it to frontload or bunker the results for the contest so we could drop a couple of bunkered clients results every couple of days to keep all the other teams hopping to catch up.

We didn't have any issues getting work, we probably had 20-50K or more tasks in play for each team member. We also had our Pandora client that allowed us to get as much work as we wanted.

Way past the normal BOINC 1000 task limit per client. If I remember correctly Ville was still returning work 5 months past the project work generation shutdown.

Also we had our Petri special app that could crunch a task in less than 30 seconds on a 1080 TI. So we needed lots of work cached so the gpus would not starve.

Gpus will always be faster than a cpu. No matter the number of cores you might engage. Remember a cpu core does one task serially. A gpu does dozens or hundreds of parallel computations on a single task in 1/10 or 1/100 the time to crunch it on a cpu. A single gpu just has more horsepower than a single cpu core. And if you pick an appropriate motherboard, it can host 3-7 gpus in one host PC.
ID: 73785 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Setting up multiple instances of BOINC in Windows

©2024 Astroinformatics Group