Welcome to MilkyWay@home

GPU WU runtimes too short: Wast of compute power

Message boards : Number crunching : GPU WU runtimes too short: Wast of compute power
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Michael H.W. Weber

Send message
Joined: 22 Jan 08
Posts: 29
Credit: 242,726,778
RAC: 0
Message 63692 - Posted: 10 Jun 2015, 11:39:03 UTC

Is it possible to extend the individual runtimes of GPU WUs, please?

My AMD 290X completes its WUs within 16 to 47 sec per WU (depending on the WU type). Then there is a limited number of WUs that can be downloaded, too. The time required to power down and to restart the next WU takes a good fraction of the short running WUs, so making these WUs as short as they are currently, wastes a significant proportion of computation time.
On top of that there is a high frequency of internet connections required. The system is virtually permanently up- or downloading work.

All these issues could be resolved if the WUs could be "bundeled" to take, say, around 30 min to a few hours of compute time per WU.

Would that be an option?

Michael.
President of Rechenkraft.net e.V. - This planet's first and largest distributed computing organization.

ID: 63692 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jake Weiss
Volunteer moderator
Project developer
Project tester
Project scientist

Send message
Joined: 25 Feb 13
Posts: 580
Credit: 94,200,158
RAC: 0
Message 63693 - Posted: 10 Jun 2015, 12:21:06 UTC

Hey Michael,

We have been looking into "bundling" work units to make them a bit bigger. This has its own problems that we are currently trying to solve. Hopefully by the end of the Summer session we will have a working option for this.

It should be possible to request more work units and crunch multiple work units on your GPU. For more info on what other users have found you can look here: http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=3677#62811.


Jake W
ID: 63693 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 22 Jan 08
Posts: 29
Credit: 242,726,778
RAC: 0
Message 63702 - Posted: 11 Jun 2015, 12:16:38 UTC

Well, thanks for the feedback. I think it is an important issue and I hope it can be solved soon.

Michael.
President of Rechenkraft.net e.V. - This planet's first and largest distributed computing organization.

ID: 63702 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skildude
Avatar

Send message
Joined: 13 Jan 08
Posts: 52
Credit: 59,823,507
RAC: 0
Message 63945 - Posted: 20 Sep 2015, 17:48:17 UTC

You could do like Collatz did and provide different sized WU's and allow the users to choose what size they want.
Blah blah blah you know the rest
ID: 63945 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 22 Jan 08
Posts: 29
Credit: 242,726,778
RAC: 0
Message 64329 - Posted: 11 Feb 2016, 9:28:40 UTC - in response to Message 63693.  

Hey Michael,

We have been looking into "bundling" work units to make them a bit bigger. This has its own problems that we are currently trying to solve. Hopefully by the end of the Summer session we will have a working option for this.

Jake W

Well, summer is over and I patiently waited for another 8 months.
How is the implementation of longer GPU WUs progressing?

Michael.
President of Rechenkraft.net e.V. - This planet's first and largest distributed computing organization.

ID: 64329 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MindCrime

Send message
Joined: 5 Mar 14
Posts: 24
Credit: 500,964,006
RAC: 0
Message 64385 - Posted: 15 Mar 2016, 19:44:24 UTC - in response to Message 64329.  
Last modified: 15 Mar 2016, 19:48:08 UTC

Hey Michael,

We have been looking into "bundling" work units to make them a bit bigger. This has its own problems that we are currently trying to solve. Hopefully by the end of the Summer session we will have a working option for this.

Jake W

Well, summer is over and I patiently waited for another 8 months.
How is the implementation of longer GPU WUs progressing?

Michael.



Run more concurrent, this will increase the run time while reducing "down-time" inbetween WUs.

If you have a high double precision card such as a Tahiti you can run 6+ concurrent and they still last around 1 min or 2 each.

A rough of the top of my head no error checking app_config would be

app_config.xml

<app_config>
<project_max_concurrent>2</project_max_concurrent>
<app>
<name>milkyway</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>.5</gpu_usage>
<cpu_usage>.1</cpu_usage>
</gpu_versions>
</app>
<app>
<name>MODIFIEDFIT</name>
<max_concurrent>2</max_concurrent>
<gpu_versions>
<gpu_usage>.5</gpu_usage>
<cpu_usage>.1</cpu_usage>
</gpu_versions>
</app>
</app_config>

You'll have to check the app names, boinc will tell you which apps it currently knows of in the event-log if you have an app_config with the wrong names. Check it and replace with the correct ones.

The above will run a maximum of 2 milkyway WUs, and 2 of any one type at once. Adjust as needed. If you want to run more you'll have to adjust the gpu_usage to .333, .250, .125 or whatever fraction of 1 you want to get 3, 4, 8 concurrent. Nvidia cards I wouldn't go past 2 or 3 concurrent. Unless you have a good double precision card you won't see very good gpu credit on milkyway.
ID: 64385 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 22 Jan 08
Posts: 29
Credit: 242,726,778
RAC: 0
Message 65144 - Posted: 15 Sep 2016, 8:00:51 UTC

Multiple tasks don't run in parallel on 290X cards for unknown reasons. So bundling is the proper choice to solve the problem.

Moreover, even if you run 6 WUs in parallel, they are also completed in a few minutes, so the problem detailed above persists. And by the way, for 280X cards, you may run even up to 12 tasks in parallel (tested).

Michael.
President of Rechenkraft.net e.V. - This planet's first and largest distributed computing organization.

ID: 65144 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tex1954

Send message
Joined: 22 Apr 11
Posts: 64
Credit: 899,267,437
RAC: 7,473
Message 65145 - Posted: 15 Sep 2016, 8:20:25 UTC - in response to Message 65144.  
Last modified: 15 Sep 2016, 8:35:11 UTC

The server seems to have slowed down. I can't cache enough WU's to keep busy!!!

I run 2 WU per card and have two 7970's and one R9 280X doing Milkyway...

It always pauses 1 minute after each update and buy the time 1 minute expires, the WU's are already all done OR the setup requests more work and gets none/little several times in a row!!!

Win-F13

96220 Milkyway@Home 9/15/2016 3:25:03 AM Reporting 16 completed tasks
96221 Milkyway@Home 9/15/2016 3:25:03 AM Requesting new tasks for AMD/ATI GPU
96222 Milkyway@Home 9/15/2016 3:25:04 AM Scheduler request completed: got 0 new tasks
96223 Milkyway@Home 9/15/2016 3:25:09 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_3_1471352126_34821001_1 finished

96322 Milkyway@Home 9/15/2016 3:29:30 AM Requesting new tasks for AMD/ATI GPU
96323 Milkyway@Home 9/15/2016 3:29:32 AM Scheduler request completed: got 7 new tasks
96324 Milkyway@Home 9/15/2016 3:29:34 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322413_0
96325 Milkyway@Home 9/15/2016 3:29:34 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322405_0
96326 Milkyway@Home 9/15/2016 3:29:34 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322406_0
96327 Milkyway@Home 9/15/2016 3:29:34 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322410_0
96328 Milkyway@Home 9/15/2016 3:29:52 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322413_0 finished
96329 Milkyway@Home 9/15/2016 3:29:52 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322405_0 finished
96330 Milkyway@Home 9/15/2016 3:29:52 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322406_0 finished
96331 Milkyway@Home 9/15/2016 3:29:52 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322410_0 finished
96332 Milkyway@Home 9/15/2016 3:29:52 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_2_1471352126_36322419_0
96333 Milkyway@Home 9/15/2016 3:29:52 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322404_0
96334 Milkyway@Home 9/15/2016 3:29:52 AM Starting task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322407_0
96335 Milkyway@Home 9/15/2016 3:30:04 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322407_0 finished
96336 Milkyway@Home 9/15/2016 3:30:10 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_2_1471352126_36322419_0 finished
96337 Milkyway@Home 9/15/2016 3:30:10 AM Computation for task de_modfit_fast_19_3s_136_ModfitConstraints1_1_1471352126_36322404_0 finished
96338 Milkyway@Home 9/15/2016 3:30:38 AM Sending scheduler request: To report completed tasks.
96339 Milkyway@Home 9/15/2016 3:30:38 AM Reporting 7 completed tasks
96340 Milkyway@Home 9/15/2016 3:30:38 AM Requesting new tasks for AMD/ATI GPU
96341 Milkyway@Home 9/15/2016 3:30:40 AM Scheduler request completed: got 2 new tasks
96342 Milkyway@Home 9/15/2016 3:30:42 AM Starting task de_modfit_fast_15_3s_136_fixedangles3_3_1471352126_36322870_0
96343 Milkyway@Home 9/15/2016 3:30:42 AM Starting task de_modfit_fast_15_3s_136_fixedangles3_3_1471352126_36322868_0

Win-F21

104581 Milkyway@Home 9/15/2016 3:34:05 AM Reporting 10 completed tasks
104582 Milkyway@Home 9/15/2016 3:34:05 AM Requesting new tasks for AMD/ATI GPU
104583 Milkyway@Home 9/15/2016 3:34:06 AM Scheduler request completed: got 3 new tasks


HELP!

8-)
ID: 65145 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : GPU WU runtimes too short: Wast of compute power

©2024 Astroinformatics Group