Welcome to MilkyWay@home

Run Multiple WU's on Your GPU

Message boards : Number crunching : Run Multiple WU's on Your GPU
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 11 · Next

AuthorMessage
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69187 - Posted: 29 Oct 2019, 19:34:40 UTC

My Radeon HD 7970 seems to run at 98% GPU usage with just one task. Is multiple units no longer required due to better coding at the Milkyway end? Or are some cards different to others? I deliberately got a card that does double precision very fast compared to the price.

As for the number of workunits - I remember when there was a limit of 80 at once (per machine or card I forget which). Now I seem to get 600 at a time, but it waits until they're all done before I can get another load. Although if I pause the GPU when I'm playing a game, then I seem to get a topup. How is the download limit calculated?
ID: 69187 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
_heinz

Send message
Joined: 23 Feb 09
Posts: 28
Credit: 10,775,220
RAC: 0
Message 69319 - Posted: 2 Dec 2019, 15:38:50 UTC

I'm running 5 wu's at once on my three old NVIDIA GTX Titan SC, bought on March 2013
<app_config>
<app>
<name>milkyway</name>
<max_concurrent>50</max_concurrent>
<gpu_versions>
<cpu_usage>0.1</cpu_usage>
<gpu_usage>0.2</gpu_usage>
</gpu_versions>
</app>
</app_config>
---------------------------------------------------------
2x Xeon E5405 a 2400MHz
3x GTX Titan SC
CPU:99%
GPU1:93%
GPU2:99%
GPU3:95%
GPU1 Temp:71 grd Celsius
GPU2 Temp:71 grd Celsius
GPU3 Temp:59 grd Celsius
CPU1 Temp 36 grd Celsius
CPU2 Temp 48 grd Celsius
the machine is air coold
ID: 69319 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69320 - Posted: 2 Dec 2019, 19:55:54 UTC - in response to Message 69319.  
Last modified: 2 Dec 2019, 20:03:39 UTC

Are those Titan XP cards?

Sorry, didn't notice you said SC.

I figured XP from the speed you were getting. Can't find anywhere to compare the two.

If it's similar to the XP, you should be running a single precision project like Einstein on it. It's great for that, but rubbish at double precision for Milkyway.

My list of cards and how good they are:
https://www.dropbox.com/s/fthl41hovchfye2/gpu.xlsx?dl=0
ID: 69320 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 69321 - Posted: 2 Dec 2019, 21:00:12 UTC - in response to Message 68865.  

Thanks for the optimizing tips for the Radeon VII.
Now I have another problem ..... is it possibe to get more than 300 workunits at a time? :P
Every time I have crunched 300 WU's and asks for more it takes more than 5 minutes to get another batch of 300.


MAYBE...the thing is the MilkyWay Server won't send any tasks until after youhave a 5 to 10 minute break in communications from them, hence the running out of wu's for everyone. Now if you could set it so MilkyWay does NOT report the completed wu's as completed until after that time out time then yes you could get wu's on a regular recurring basis.There is a person who provided a custom version of Boinc in another thread and a timeout script who says he's getting no breaks from crunching, I don't know if there is a way to write a script preventing MilkyWay from reporting completed tasks while still letting other projects report their completed wu's, or if it even needs too as often as it does. I know alot of us a long time added <report_results_immediately>1</report_results_immediately> to the cc_config file so we could get the rac more stabilized but here at MilkyWay it seems to report them pretty quickly even without it.
ID: 69321 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69322 - Posted: 2 Dec 2019, 21:08:10 UTC - in response to Message 69321.  

The problem would also be solved with more work units. The reason our clients are continually pestering the MW server is they want say the default of 1 day of buffer. Every other project is happy to provide 1 or 2 WEEKS of work, why isn't Milkyway?
ID: 69322 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 69323 - Posted: 3 Dec 2019, 0:39:15 UTC - in response to Message 69322.  

The problem would also be solved with more work units. The reason our clients are continually pestering the MW server is they want say the default of 1 day of buffer. Every other project is happy to provide 1 or 2 WEEKS of work, why isn't Milkyway?


Not EVERY other Project but I don't know the answer. Some Projects base the next set if of workunits on the last set of workunits so don't provide too many, others just go thru a list until it's done so can provide lots of workunits. I believe MilkyWay gets their raw data from somewhere else but don't know how much or how often.
ID: 69323 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69324 - Posted: 3 Dec 2019, 20:41:09 UTC - in response to Message 69323.  
Last modified: 3 Dec 2019, 20:47:35 UTC


Not EVERY other Project but I don't know the answer. Some Projects base the next set if of workunits on the last set of workunits so don't provide too many, others just go thru a list until it's done so can provide lots of workunits. I believe MilkyWay gets their raw data from somewhere else but don't know how much or how often.


What do you mean by "base the next set of workunits on the last set of workunits"?

Milkyway seems to be fixed at 300 per GPU, 900 per client. I've always dutifully returned all mine completed properly in a few hours, despite the time limit being a week. I should get more! I don't have this problem with Einstein which gives me as many as I ask for.

If Milkyway didn't have a tiny limit, my client wouldn't be trying to get more every 1.5 minutes, so I wouldn't be using up server resources, and I wouldn't be triggering limitations on sending more units out.

P.S. are those 17 computers all yours or are they at your work? 17 computers is a lot of electricity! I used to have 500 running at my work, unfortunately no GPUs. and unfortunately I don't work there any more.
ID: 69324 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 24 Jan 11
Posts: 714
Credit: 554,788,711
RAC: 35,072
Message 69325 - Posted: 4 Dec 2019, 1:17:20 UTC - in response to Message 69324.  

What do you mean by "base the next set of workunits on the last set of workunits"?

Some projects base the next set of tasks that the previous set results produced new parameter sets or hints about how to format the next work. Gpugrid is a project that works this way for example.
ID: 69325 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 69328 - Posted: 4 Dec 2019, 11:20:08 UTC - in response to Message 69324.  
Last modified: 4 Dec 2019, 11:21:22 UTC


Not EVERY other Project but I don't know the answer. Some Projects base the next set if of workunits on the last set of workunits so don't provide too many, others just go thru a list until it's done so can provide lots of workunits. I believe MilkyWay gets their raw data from somewhere else but don't know how much or how often.


What do you mean by "base the next set of workunits on the last set of workunits"?

Milkyway seems to be fixed at 300 per GPU, 900 per client. I've always dutifully returned all mine completed properly in a few hours, despite the time limit being a week. I should get more! I don't have this problem with Einstein which gives me as many as I ask for.

If Milkyway didn't have a tiny limit, my client wouldn't be trying to get more every 1.5 minutes, so I wouldn't be using up server resources, and I wouldn't be triggering limitations on sending more units out.

P.S. are those 17 computers all yours or are they at your work? 17 computers is a lot of electricity! I used to have 500 running at my work, unfortunately no GPUs. and unfortunately I don't work there any more.


MilkyWay, under the last Admin, had a problem with people storing 10 days of wu's and then them not getting crunched and having to all be recycled so they reduced the total number each device could get, this past year they raised it to where it is now. They said they discussed it in house and were happy with it but knew some people would not be, and they understood that but the decision stands.

And yes all 17 of those pc's are at my house and yes the electric bill is high especially when it's gets into the 100's here at the beach on the East Coast of the US.
ID: 69328 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69329 - Posted: 4 Dec 2019, 20:35:39 UTC - in response to Message 69328.  

That's odd because Boinc shouldn't be downloading more than can be crunched on time. I don't see it happening with other projects. I doubt there were that many messing about. If it was me, I'd have just clamped down their accounts only, reduced them to lower quantities at once.
ID: 69329 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Cliff Harding
Avatar

Send message
Joined: 2 Jul 09
Posts: 27
Credit: 253,069,838
RAC: 0
Message 69501 - Posted: 1 Feb 2020, 23:02:23 UTC

I'm attempting to run 2 WUs each on twin NVidia 1070s only, no CPU tasks, on an Intel 7700K under the current Win 10 Pro with the following app_config.xml
<app_config>
<app>
<name>milkyway</name>
<max_concurrent>4</max_concurrent>
<gpu_versions>
<gpu_usage>.5</gpu_usage>
<cpu_usage>1</cpu_usage>
</gpu_versions>
</app>
</app_config>

I'm keep getting the following line in the event log
02/01/2020 17:49:59 | Milkyway@Home | Your settings do not allow fetching tasks for CPU. To fix this, you can change Project Preferences on the project's web site.

Currently, it's defaulting to (running 0.96 CPUs + 1 NVIDIA device 0)) and device 1.
What to I need to rectify this?


I don't buy computers, I build them!
ID: 69501 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69502 - Posted: 1 Feb 2020, 23:35:38 UTC - in response to Message 69501.  
Last modified: 1 Feb 2020, 23:37:19 UTC

I'm attempting to run 2 WUs each on twin NVidia 1070s only, no CPU tasks, on an Intel 7700K under the current Win 10 Pro with the following app_config.xml
<app_config>
<app>
<name>milkyway</name>
<max_concurrent>4</max_concurrent>
<gpu_versions>
<gpu_usage>.5</gpu_usage>
<cpu_usage>1</cpu_usage>
</gpu_versions>
</app>
</app_config>

I'm keep getting the following line in the event log
02/01/2020 17:49:59 | Milkyway@Home | Your settings do not allow fetching tasks for CPU. To fix this, you can change Project Preferences on the project's web site.

Currently, it's defaulting to (running 0.96 CPUs + 1 NVIDIA device 0)) and device 1.
What to I need to rectify this?


That should work, I have a very similar setup - two GPUs running GPU tasks only, 2 each (I forbid CPU tasks on the Milkyway website).

My app config is almost the same as yours (are you out of CPU cores? Are you running other projects on CPU? If so reduce the cpu_usage in app_config), I use this:

<app_config>
<app>
<name>milkyway</name>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>.45</cpu_usage>
</gpu_versions>
</app>
</app_config>

Your event log you can ignore, it's just pestering you to take CPU tasks, which you don't want anyway.
The "0.96 CPUs + 1 NVIDIA device 0" could be out of date, it doesn't always change what it displays straight away until it gets new tasks.

If it's only running 2 tasks on your 2 cards, instead of 4 tasks in total, you need to tell it to read the config file.
If you're using Boinctasks, click the extra menu, then read config files.
If you're using plain Boinc, click the options menu, then read config files.
It should take effect immediately and run 4 at once, if not, restart.
ID: 69502 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Cliff Harding
Avatar

Send message
Joined: 2 Jul 09
Posts: 27
Credit: 253,069,838
RAC: 0
Message 69503 - Posted: 2 Feb 2020, 0:20:47 UTC - in response to Message 69502.  

I'm running basically the same setup in SETI, with 4 GPU tasks and 1 CPU task. The manager is set at 70% CPU. I'm also running current Lunatics, and the manager is 7.6.13 (x64), but I don't think that either should be relevant to my situation.

When SETI has no GPU tasks available, I had hoped to run Milkyway the same way.


I don't buy computers, I build them!
ID: 69503 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69504 - Posted: 2 Feb 2020, 0:28:04 UTC - in response to Message 69503.  
Last modified: 2 Feb 2020, 0:29:57 UTC

I'm running basically the same setup in SETI, with 4 GPU tasks and 1 CPU task. The manager is set at 70% CPU. I'm also running current Lunatics, and the manager is 7.6.13 (x64), but I don't think that either should be relevant to my situation.

When SETI has no GPU tasks available, I had hoped to run Milkyway the same way.


Your
"(running 0.96 CPUs + 1 NVIDIA device 0)) and device 1."
does not equate with your app config of
"<gpu_usage>.5</gpu_usage>
<cpu_usage>1</cpu_usage>"
It has not taken into account what you told it.
Restart the computer, then you're sure it's read the config you gave it.
Then if it still says 0.96 and 1 instead of 1 and 0.5, then abort all tasks and let it get new ones.
If it still doesn't work, you might have made a slight error in the config file. I've done that before, the slightest mistake in brackets etc and it ignores the whole lot.

I have done the same as you with Milkyway and Einstein as a backup (as Milkyway has 10 minute gaps between batches of tasks it will hand out).
ID: 69504 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Cliff Harding
Avatar

Send message
Joined: 2 Jul 09
Posts: 27
Credit: 253,069,838
RAC: 0
Message 69505 - Posted: 2 Feb 2020, 1:13:58 UTC - in response to Message 69504.  

It seems that the app_config.xml is being completely ignored. There is no line in the event log that states that the manager recognized it.


I don't buy computers, I build them!
ID: 69505 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Cliff Harding
Avatar

Send message
Joined: 2 Jul 09
Posts: 27
Credit: 253,069,838
RAC: 0
Message 69508 - Posted: 2 Feb 2020, 14:52:08 UTC - in response to Message 69504.  

I'm running basically the same setup in SETI, with 4 GPU tasks and 1 CPU task. The manager is set at 70% CPU. I'm also running current Lunatics, and the manager is 7.6.13 (x64), but I don't think that either should be relevant to my situation.

When SETI has no GPU tasks available, I had hoped to run Milkyway the same way.


Your
"(running 0.96 CPUs + 1 NVIDIA device 0)) and device 1."
does not equate with your app config of
"<gpu_usage>.5</gpu_usage>
<cpu_usage>1</cpu_usage>"
It has not taken into account what you told it.
Restart the computer, then you're sure it's read the config you gave it.
Then if it still says 0.96 and 1 instead of 1 and 0.5, then abort all tasks and let it get new ones.
If it still doesn't work, you might have made a slight error in the config file. I've done that before, the slightest mistake in brackets etc and it ignores the whole lot.

I have done the same as you with Milkyway and Einstein as a backup (as Milkyway has 10 minute gaps between batches of tasks it will hand out).


Problem resolved. I keep certain files in a separate folder for use at a later time in case something catastrophic happens. It seems that when I copied this file into my project folder, I didn't pay attention to the full file name. Adjusted the name and it is working as expected -- A man's mind grows dim in his later years.


I don't buy computers, I build them!
ID: 69508 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jimbocous
Avatar

Send message
Joined: 7 Mar 20
Posts: 22
Credit: 105,738,225
RAC: 6,942
Message 69585 - Posted: 7 Mar 2020, 4:12:09 UTC
Last modified: 7 Mar 2020, 4:13:09 UTC

Just curious to see if anyone has really studied whether there's an appreciable gain in running >1 WU/GPU?
New here at MW@H, but in previous tests at other projects I never saw any significant gain.
Regards, Jim ...
ID: 69585 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3339
Credit: 524,010,781
RAC: 0
Message 69586 - Posted: 7 Mar 2020, 12:13:53 UTC - in response to Message 64197.  

Hi Mikey,
yeah, makes sense, but I just wish someone in the programming dept at BOINC would makes those sorts of changes immediate.. Save a lot of faffing about to sort out a problem, particularly if you have a large[ish] cache of WU.

Regards


I believe this is a MW 'tweak' they made to their Server software, it isn't true at all the other projects.


But it is true at several Projects, so I think it's a Server side setting that the Admins are okay with, yes some Projects make the changes immediately but not all do.
ID: 69586 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 5 Jul 11
Posts: 990
Credit: 376,143,149
RAC: 0
Message 69587 - Posted: 7 Mar 2020, 18:51:18 UTC - in response to Message 69585.  

Just curious to see if anyone has really studied whether there's an appreciable gain in running >1 WU/GPU?
New here at MW@H, but in previous tests at other projects I never saw any significant gain.
Regards, Jim ...


It depends on your CPU speed related to your GPU speed. I have a machine with a fast CPU and a slow GPU, it gets no benefit from two WUs per GPU. I have a machine with a slow CPU and a fast GPU, it certainly benefits from two WUs per GPU. Presumably because with only one WU per GPU, when it gets to a part of the WU that requires come calculations over on the CPU, the GPU is sat idle waiting for it. With a good CPU, that time is minimal. With a slower CPU, having a second WU the GPU can do while it's waiting means it's in use more often.

All I did was try it with 1 WU per GPU, note how long it took to complete a WU, then try two WUs, and note how long those WUs took (obviously taking exactly twice as long would mean no change, as it's doing two at once). Then I even tried 3 and 4 WUs. In my setups, I usually find 2 WUs is an improvement, and 3 4 etc makes no change. But I have old computers with good GPUs - as I spend all the cash on GPUs.
ID: 69587 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Hurr1cane78

Send message
Joined: 7 May 14
Posts: 57
Credit: 206,540,646
RAC: 34
Message 69786 - Posted: 10 May 2020, 8:40:43 UTC

hi all made vid on youtube for multiple instances instruction's and at full load on a Radeon VII
RADEON VII GIGABYTE// 3 Instances_ Milkyway@home WUs BOINC_ 3_instances
https://www.youtube.com/watch?v=4xKy9wGKmz4
all the best and welcome to earth
ID: 69786 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 11 · Next

Message boards : Number crunching : Run Multiple WU's on Your GPU

©2024 Astroinformatics Group