Welcome to MilkyWay@home

Getting more than 18 work units at a time

Message boards : Number crunching : Getting more than 18 work units at a time
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile PartyDroid

Send message
Joined: 19 Oct 10
Posts: 4
Credit: 10,702,535
RAC: 0
Message 43404 - Posted: 1 Nov 2010, 23:49:21 UTC

Just wondering if its possible to get more than 18 units from the server. My internet connection's a bit sketch at the moment and i can complete all 18 units in 10-15 minutes when i run my graphics at 950Mhz which can give me some time where i'm not computing
ID: 43404 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John Clark

Send message
Joined: 4 Oct 08
Posts: 1734
Credit: 64,228,409
RAC: 0
Message 43405 - Posted: 1 Nov 2010, 23:56:00 UTC

Milkyway allows 6 WUs per core. So, if you have a quad then the limit is 24.
Go away, I was asleep


ID: 43405 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
55degrees

Send message
Joined: 8 Sep 09
Posts: 62
Credit: 61,330,584
RAC: 0
Message 43474 - Posted: 4 Nov 2010, 19:51:04 UTC - in response to Message 43404.  

and that applies to crunchers with 'no cpu' setting. I get 48 for an i7 and the gpus go through them <36min.
ID: 43474 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mdhittle*
Avatar

Send message
Joined: 25 Jun 10
Posts: 284
Credit: 260,490,091
RAC: 0
Message 43475 - Posted: 4 Nov 2010, 20:04:01 UTC

We know that the cache is limited to 6 WUs per core. But, it is high time that this limit be changed. The server is VERY UNRELIABLE with daily outages.

Two HD 5970s can do 4 WUs in less than 90 seconds. This makes the current cache size RIDICULUS for GPU crunchers. For a person crunching on CPUs, 6 WUs per core is a 2 or 3 day cache size.

The people running GPUs should have the same opportunity to maintain a 2 to 3 day cache, also.

I know, I have read all of the excuses for way it can’t be done. But, it could be done if the administrators would devote some time to accomplishing it. Yes, the database will need to be expanded, so expand it. This should have a higher priority than the screensaver that is just eye candy.

The large majority of the WUs that get crunched are crunched by GPUs. When the server fails, it brings this to a very quick stop due to the ridiculously low cache size that has been set.

-Mike
ID: 43475 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John Clark

Send message
Joined: 4 Oct 08
Posts: 1734
Credit: 64,228,409
RAC: 0
Message 43478 - Posted: 4 Nov 2010, 20:49:19 UTC

Mike

Believe the reasons why they need a 6 core limit, as these are not excuses and based on things that have happened over the last many months.

If this is still a problem, then get a back up GPU project (Collatz/DNETC/etc).

End of argument.
Go away, I was asleep


ID: 43478 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mdhittle*
Avatar

Send message
Joined: 25 Jun 10
Posts: 284
Credit: 260,490,091
RAC: 0
Message 43480 - Posted: 4 Nov 2010, 21:12:38 UTC - in response to Message 43478.  

Mike

Believe the reasons why they need a 6 core limit, as these are not excuses and based on things that have happened over the last many months.

If this is still a problem, then get a back up GPU project (Collatz/DNETC/etc).

End of argument.


Good deal on ending the argument with you, since you are in no position to change the cache size, or confirm that it will never be changed. I wasn't addressing my comments to you. I am really not sure why you would think that you have the final say in this.

A person should not need a backup project. That is what the cache is for. The cache is supposed to provide enough work for the computer so it doesn't need to download work every 90 seconds. It should also provied a buffer for the times that the server has failed.

-Mike

ID: 43480 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 43481 - Posted: 4 Nov 2010, 21:21:09 UTC - in response to Message 43480.  


A person should not need a backup project. That is what the cache is for. The cache is supposed to provide enough work for the computer so it doesn't need to download work every 90 seconds. It should also provied a buffer for the times that the server has failed.

-Mike


Well this is still classified as a beta project. And frequent outages have almost always occured, especially when anything is touched.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 43481 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mdhittle*
Avatar

Send message
Joined: 25 Jun 10
Posts: 284
Credit: 260,490,091
RAC: 0
Message 43482 - Posted: 4 Nov 2010, 22:10:51 UTC - in response to Message 43481.  

Well this is still classified as a beta project. And frequent outages have almost always occured, especially when anything is touched.


Agreed, it is still a BETA project.

That is why this is a good time to work out the problems with the cache size. CPU crunchers probably do not notice the frequent outages as much as the GPU crunchers due to the cache size.

Due to the fact that an ATI GPU can do in 90 seconds the same amount of work a CPU can do in 10 hours, this project wouldn't be as far along as it is right now without the GPUs.

Due to the frequent failures of the server, the processing power of the GPUs is not being fully realized by the project.

It isn't the end of the project or the end of the world, if they adjusted the size of the cache that a GPU can have.

-Mike
ID: 43482 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 15
Message 43485 - Posted: 4 Nov 2010, 23:51:21 UTC - in response to Message 43480.  

Mike, one of the rationales for the BOINC client early on was for multiple project support. So, 'a person should not need a backup project' -- well no, that doesn't state the case fairly. It is clear that *you* believe that *you* should not need a backup project. Fair enough, then for *you* MW is likely not a good *single* project to run, just as SETI is a lousy single project to run.

If you feel that you should only run a single project (as you've stated) then perhaps you should look into projects that are 1) more reliable than MilkyWay and/or 2) support larger caches.

Projects that fit that mode for GPU folks include Dnetc and Collatz (which John mentioned).

Ideally, no project should 'require' that you run multiple projects. Ideally, the unemployment rate should be under 5%. I suspect you (and everyone else here) has as much control over either of these.





A person should not need a backup project. That is what the cache is for. The cache is supposed to provide enough work for the computer so it doesn't need to download work every 90 seconds. It should also provied a buffer for the times that the server has failed.

-Mike



ID: 43485 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mdhittle*
Avatar

Send message
Joined: 25 Jun 10
Posts: 284
Credit: 260,490,091
RAC: 0
Message 43486 - Posted: 5 Nov 2010, 0:28:05 UTC - in response to Message 43485.  
Last modified: 5 Nov 2010, 0:28:59 UTC

Mike, one of the rationales for the BOINC client early on was for multiple project support. So, 'a person should not need a backup project' -- well no, that doesn't state the case fairly. It is clear that *you* believe that *you* should not need a backup project. Fair enough, then for *you* MW is likely not a good *single* project to run, just as SETI is a lousy single project to run.

If you feel that you should only run a single project (as you've stated) then perhaps you should look into projects that are 1) more reliable than MilkyWay and/or 2) support larger caches.

Projects that fit that mode for GPU folks include Dnetc and Collatz (which John mentioned).

Ideally, no project should 'require' that you run multiple projects. Ideally, the unemployment rate should be under 5%. I suspect you (and everyone else here) has as much control over either of these.




I do run other projects. Running other projects isn't the point of this thread. The subject and point of the thread is the cache size at Milky Way.

With the higher speeds of the newer cards, the cache size of 6 WUs per core is antiquated. You do not notice the problem as acutely as the people running the modern high speed cards because you are running cards that are incapable of doing a WU in 90 seconds.

Your average is about 2 hours with one GPU to burn through 24 WUs, you have 4 cores. A machine with 2 ATI HD5970s and 4 cores can burn through 24 WUs in 9 minutes. The cache size of 6 WUs per core does not work for the modern high speed GPUs that being run on the Milky Way project.

It is time to adjust the cache size to accommodate the new GPUs that are being used. It isn’t that much to ask for. The bulk of the crunching that happens for this project is done on the new high speed GPUs at less than 90 seconds per WU. CPUs and less powerful GPUs might be fully utilized by the current small cache size, but the new high modern high speed cards are not.

The people who are willing to DONATE their time, money, and resources to THIS project should be supported equally by the project. If you are crunching on a CPU, NVIDIA GPU, or an ATI GPU that is slower than a 5800 series, the current cache size works for you. If you are running an ATI 5800 or 6000 GPU, the cache size is a joke.

-Mike
ID: 43486 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 15
Message 43487 - Posted: 5 Nov 2010, 0:48:36 UTC - in response to Message 43486.  

For MilkyWay, the issue of inadequate cache sizes has been out there since the early days. Once an optimized client (and then an optimized ATI client) was available, cache size issues surfaced big time. Since this project has (since the time that it offered GPU support) always required double precision GPU capable cards, the 'newer cards' issue was there from the start *it is NOT a new phenomenom for this project*. That was over two years ago. In the interim, the actual work unit size HAS been increased somewhat. It is still an issue. Far a long while folks have asked for either a larger cache size (in terms of number of units) or longer running GPU work units with the same cache size (to achieve the same effect).

As I understand it, a large part of this is a resource issue - for the project. And while some folks might feel the project exists for the users benefit, it doesn't quite work out that way. The project resources don't magically scale up as users upgrade their hardware and additional power users join in. Frankly, I would like to see a new application design which would result in longer running GPU work units with the same 'credit per hour run time'. I would think that would avoid the server I/O stress that simply increasing the cache size spec would present. But again, I think that would entail a fair amount of work on the project side that they lack the resources to do. In the past, some forays into larger cache sizes (10 and at one point 20 work units up from 6) simply resulted in more server down time. Given the downtime we are seeing on server, it is my *opinion* that increasing cache sizes might well make things worse rather than better for the project.

I do find the *expectation* that the project *owes us* a tad troublesome, if you think the project's cache limits are a joke, you certainly have other project alternatives.

Personally, the area I tend to have high expectations of a project (any project is that the project communicates clearly and in a timely manner issues they have. Many projects don't meet that standard for me, I still participate in them, but I do increase my noise level a bit.

This project is middle of the road in that regard, so sometimes I put my noisemaker mode in gear.



I do run other projects. Running other projects isn't the point of this thread. The subject and point of the thread is the cache size at Milky Way.

With the higher speeds of the newer cards, the cache size of 6 WUs per core is antiquated. You do not notice the problem as acutely as the people running the modern high speed cards because you are running cards that are incapable of doing a WU in 90 seconds.

Your average is about 2 hours with one GPU to burn through 24 WUs, you have 4 cores. A machine with 2 ATI HD5970s and 4 cores can burn through 24 WUs in 9 minutes. The cache size of 6 WUs per core does not work for the modern high speed GPUs that being run on the Milky Way project.

It is time to adjust the cache size to accommodate the new GPUs that are being used. It isn’t that much to ask for. The bulk of the crunching that happens for this project is done on the new high speed GPUs at less than 90 seconds per WU. CPUs and less powerful GPUs might be fully utilized by the current small cache size, but the new high modern high speed cards are not.

The people who are willing to DONATE their time, money, and resources to THIS project should be supported equally by the project. If you are crunching on a CPU, NVIDIA GPU, or an ATI GPU that is slower than a 5800 series, the current cache size works for you. If you are running an ATI 5800 or 6000 GPU, the cache size is a joke.

-Mike


ID: 43487 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 43488 - Posted: 5 Nov 2010, 1:28:30 UTC

I believe that MW has amassed quite a pile of data with the addition of gpu processing that they aren't exactly worried about some down time or maxing outgpu usage in down time. Also the data done now is many 100's of times faster and much more data being done as well per wu than initially.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 43488 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
bill

Send message
Joined: 15 Jul 09
Posts: 12
Credit: 45,145,989
RAC: 0
Message 43490 - Posted: 5 Nov 2010, 2:35:43 UTC - in response to Message 43486.  

(snippage)

The people who are willing to DONATE their time, money, and resources to THIS project more (snippage)
-Mike


Are owed nothing by this, or any other, project. You should look up what "donate" means. Or do you have a contract that states otherwise?
ID: 43490 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile The Gas Giant
Avatar

Send message
Joined: 24 Dec 07
Posts: 1947
Credit: 240,884,648
RAC: 0
Message 43491 - Posted: 5 Nov 2010, 2:56:43 UTC

Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer.

ID: 43491 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mdhittle*
Avatar

Send message
Joined: 25 Jun 10
Posts: 284
Credit: 260,490,091
RAC: 0
Message 43492 - Posted: 5 Nov 2010, 3:34:35 UTC - in response to Message 43491.  
Last modified: 5 Nov 2010, 3:38:53 UTC

Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer.



Your input is appreciated.

I have read every post in this forum on the subject of the cache size. It would require a significant change to the database. Also, new work units are generated from the results of the work units that have been completed and returned.

The database work is really the only hurdle. The work unit generation would just be speeded up because the more powerful cards are returning work units in under ten minutes. They would just have to bring it up slowly over a period of time to accommodate new "seed" work units.

I am a much newer complainer than you, but I have done my research. Unfortunately, this conversation in the past has suffered a real high “noise” to signal ratio, as this thread has.

Ultimately, I am hoping to have an intelligent conversation with the project administrators and the high speed GPU users in this forum instead of the usual "noise makers".

Instead of devoting time and resources to creating a screen saver (eye candy) and generating new CPU binaries, the resources should be devoted to fixing the server and/or increasing the size of the database. Get the system working correctly, and then develop eye candy and new binaries. The new CPU binaries have created significant down time, and they really account for a small portion of the work done on a daily basis compared to the high speed GPUs.

-Mike
ID: 43492 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 15
Message 43493 - Posted: 5 Nov 2010, 4:51:26 UTC - in response to Message 43492.  

Sounds like you have done a fair amount of research into this, are you offering to help the project administrators with the database redesign?

You might not realize this, but at least some of the attitude you toss out in your message style joins you in an august group of noisemakers. Must be august, it includes me <smile>.



The database work is really the only hurdle. The work unit generation would just be speeded up because the more powerful cards are returning work units in under ten minutes. They would just have to bring it up slowly over a period of time to accommodate new "seed" work units.

I am a much newer complainer than you, but I have done my research. Unfortunately, this conversation in the past has suffered a real high “noise” to signal ratio, as this thread has.

Ultimately, I am hoping to have an intelligent conversation with the project administrators and the high speed GPU users in this forum instead of the usual "noise makers".



ID: 43493 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BarryAZ

Send message
Joined: 1 Sep 08
Posts: 520
Credit: 302,524,931
RAC: 15
Message 43494 - Posted: 5 Nov 2010, 4:57:26 UTC - in response to Message 43491.  

I'll readily agree that the cache size here is one of the reasons my farm supports Dnetc and Collatz for more work than here. That, and of course the support those projects are able to offer for the stack of less powerful GPU's I tend to run.

While there is seeming agreement regarding the limited size of the cache, it seems there is a debate with Michael regarding the simplicity (or complexity) of changing things to support a larger cache. Regarding this, having watched that discussion over the years (yup years), I tend to accept, rather than reject out of hand, the explanations offered from the project folks. Then again, I'm certainly no expert regarding database design in general, let alone the specifics of the project specific design involved.


Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer.



ID: 43494 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Logforme

Send message
Joined: 13 Aug 10
Posts: 10
Credit: 115,945,904
RAC: 0
Message 43497 - Posted: 5 Nov 2010, 8:42:05 UTC

To me this seem to boil down to:
1. Pick the projects that maximize your credits
or
2. Pick the projects that do work you find worthwhile

I'm solidly in camp #2. The "worthiness" of the MW project is far more important to me than any rank I have on some list and me losing credits when the server goes down.
ID: 43497 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John Clark

Send message
Joined: 4 Oct 08
Posts: 1734
Credit: 64,228,409
RAC: 0
Message 43501 - Posted: 5 Nov 2010, 9:33:42 UTC - in response to Message 43494.  

I support the comments made by Barry, and other, about the MW cache size, and would like to see it larger.

My faster card(s) tend to be concentrated, ATM, on DNETZC and I leave my slower ATI GPU (an AGP one) to Milkyway. So, even this GPU gets affected by the restricted cache size, and the newer faster cards (568xx series) get a worse and worse deal as they are introduced (the 69xx will really feel it). I use Collatz as the back up project, and accept the position because that is how it is!

I have every sympathy with "mdhittle's" (Mike is it?) points and position. Indeed I agree with most of them.

But, this cache size issue has been going on for 2 years now, since the introduction of the GPU capability, which showed, and was causative, of the current cache limits. We have lived with them for this time, and, unfortunately, it is not likely to change anytime soon.




I'll readily agree that the cache size here is one of the reasons my farm supports Dnetc and Collatz for more work than here. That, and of course the support those projects are able to offer for the stack of less powerful GPU's I tend to run.

While there is seeming agreement regarding the limited size of the cache, it seems there is a debate with Michael regarding the simplicity (or complexity) of changing things to support a larger cache. Regarding this, having watched that discussion over the years (yup years), I tend to accept, rather than reject out of hand, the explanations offered from the project folks. Then again, I'm certainly no expert regarding database design in general, let alone the specifics of the project specific design involved.

The main reason for change, I see, is if the high precision mapping of the Milkyway needed a new approach and new calculations which may result in a different approach to the science.

I will apologise to Mke for my rather abrupt last post, when others have, and are, taking a more relaxed posting reply.


Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer.




Go away, I was asleep


ID: 43501 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile verstapp
Avatar

Send message
Joined: 26 Jan 09
Posts: 589
Credit: 497,834,261
RAC: 0
Message 43503 - Posted: 5 Nov 2010, 11:56:41 UTC

Easily fixed, Mike - just get a slower GPU. :)
Google Nvidia.
Cheers,

PeterV

.
ID: 43503 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Getting more than 18 work units at a time

©2024 Astroinformatics Group