Message boards :
Number crunching :
Getting more than 18 work units at a time
Message board moderation
Author | Message |
---|---|
Send message Joined: 19 Oct 10 Posts: 4 Credit: 10,702,535 RAC: 0 |
Just wondering if its possible to get more than 18 units from the server. My internet connection's a bit sketch at the moment and i can complete all 18 units in 10-15 minutes when i run my graphics at 950Mhz which can give me some time where i'm not computing |
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
Milkyway allows 6 WUs per core. So, if you have a quad then the limit is 24. Go away, I was asleep |
Send message Joined: 8 Sep 09 Posts: 62 Credit: 61,330,584 RAC: 0 |
and that applies to crunchers with 'no cpu' setting. I get 48 for an i7 and the gpus go through them <36min. |
Send message Joined: 25 Jun 10 Posts: 284 Credit: 260,490,091 RAC: 0 |
We know that the cache is limited to 6 WUs per core. But, it is high time that this limit be changed. The server is VERY UNRELIABLE with daily outages. Two HD 5970s can do 4 WUs in less than 90 seconds. This makes the current cache size RIDICULUS for GPU crunchers. For a person crunching on CPUs, 6 WUs per core is a 2 or 3 day cache size. The people running GPUs should have the same opportunity to maintain a 2 to 3 day cache, also. I know, I have read all of the excuses for way it can’t be done. But, it could be done if the administrators would devote some time to accomplishing it. Yes, the database will need to be expanded, so expand it. This should have a higher priority than the screensaver that is just eye candy. The large majority of the WUs that get crunched are crunched by GPUs. When the server fails, it brings this to a very quick stop due to the ridiculously low cache size that has been set. -Mike |
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
Mike Believe the reasons why they need a 6 core limit, as these are not excuses and based on things that have happened over the last many months. If this is still a problem, then get a back up GPU project (Collatz/DNETC/etc). End of argument. Go away, I was asleep |
Send message Joined: 25 Jun 10 Posts: 284 Credit: 260,490,091 RAC: 0 |
Mike Good deal on ending the argument with you, since you are in no position to change the cache size, or confirm that it will never be changed. I wasn't addressing my comments to you. I am really not sure why you would think that you have the final say in this. A person should not need a backup project. That is what the cache is for. The cache is supposed to provide enough work for the computer so it doesn't need to download work every 90 seconds. It should also provied a buffer for the times that the server has failed. -Mike |
Send message Joined: 12 Nov 07 Posts: 2425 Credit: 524,164 RAC: 0 |
Well this is still classified as a beta project. And frequent outages have almost always occured, especially when anything is touched. Doesn't expecting the unexpected make the unexpected the expected? If it makes sense, DON'T do it. |
Send message Joined: 25 Jun 10 Posts: 284 Credit: 260,490,091 RAC: 0 |
Well this is still classified as a beta project. And frequent outages have almost always occured, especially when anything is touched. Agreed, it is still a BETA project. That is why this is a good time to work out the problems with the cache size. CPU crunchers probably do not notice the frequent outages as much as the GPU crunchers due to the cache size. Due to the fact that an ATI GPU can do in 90 seconds the same amount of work a CPU can do in 10 hours, this project wouldn't be as far along as it is right now without the GPUs. Due to the frequent failures of the server, the processing power of the GPUs is not being fully realized by the project. It isn't the end of the project or the end of the world, if they adjusted the size of the cache that a GPU can have. -Mike |
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,524,931 RAC: 1 |
Mike, one of the rationales for the BOINC client early on was for multiple project support. So, 'a person should not need a backup project' -- well no, that doesn't state the case fairly. It is clear that *you* believe that *you* should not need a backup project. Fair enough, then for *you* MW is likely not a good *single* project to run, just as SETI is a lousy single project to run. If you feel that you should only run a single project (as you've stated) then perhaps you should look into projects that are 1) more reliable than MilkyWay and/or 2) support larger caches. Projects that fit that mode for GPU folks include Dnetc and Collatz (which John mentioned). Ideally, no project should 'require' that you run multiple projects. Ideally, the unemployment rate should be under 5%. I suspect you (and everyone else here) has as much control over either of these.
|
Send message Joined: 25 Jun 10 Posts: 284 Credit: 260,490,091 RAC: 0 |
Mike, one of the rationales for the BOINC client early on was for multiple project support. So, 'a person should not need a backup project' -- well no, that doesn't state the case fairly. It is clear that *you* believe that *you* should not need a backup project. Fair enough, then for *you* MW is likely not a good *single* project to run, just as SETI is a lousy single project to run. I do run other projects. Running other projects isn't the point of this thread. The subject and point of the thread is the cache size at Milky Way. With the higher speeds of the newer cards, the cache size of 6 WUs per core is antiquated. You do not notice the problem as acutely as the people running the modern high speed cards because you are running cards that are incapable of doing a WU in 90 seconds. Your average is about 2 hours with one GPU to burn through 24 WUs, you have 4 cores. A machine with 2 ATI HD5970s and 4 cores can burn through 24 WUs in 9 minutes. The cache size of 6 WUs per core does not work for the modern high speed GPUs that being run on the Milky Way project. It is time to adjust the cache size to accommodate the new GPUs that are being used. It isn’t that much to ask for. The bulk of the crunching that happens for this project is done on the new high speed GPUs at less than 90 seconds per WU. CPUs and less powerful GPUs might be fully utilized by the current small cache size, but the new high modern high speed cards are not. The people who are willing to DONATE their time, money, and resources to THIS project should be supported equally by the project. If you are crunching on a CPU, NVIDIA GPU, or an ATI GPU that is slower than a 5800 series, the current cache size works for you. If you are running an ATI 5800 or 6000 GPU, the cache size is a joke. -Mike |
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,524,931 RAC: 1 |
For MilkyWay, the issue of inadequate cache sizes has been out there since the early days. Once an optimized client (and then an optimized ATI client) was available, cache size issues surfaced big time. Since this project has (since the time that it offered GPU support) always required double precision GPU capable cards, the 'newer cards' issue was there from the start *it is NOT a new phenomenom for this project*. That was over two years ago. In the interim, the actual work unit size HAS been increased somewhat. It is still an issue. Far a long while folks have asked for either a larger cache size (in terms of number of units) or longer running GPU work units with the same cache size (to achieve the same effect). As I understand it, a large part of this is a resource issue - for the project. And while some folks might feel the project exists for the users benefit, it doesn't quite work out that way. The project resources don't magically scale up as users upgrade their hardware and additional power users join in. Frankly, I would like to see a new application design which would result in longer running GPU work units with the same 'credit per hour run time'. I would think that would avoid the server I/O stress that simply increasing the cache size spec would present. But again, I think that would entail a fair amount of work on the project side that they lack the resources to do. In the past, some forays into larger cache sizes (10 and at one point 20 work units up from 6) simply resulted in more server down time. Given the downtime we are seeing on server, it is my *opinion* that increasing cache sizes might well make things worse rather than better for the project. I do find the *expectation* that the project *owes us* a tad troublesome, if you think the project's cache limits are a joke, you certainly have other project alternatives. Personally, the area I tend to have high expectations of a project (any project is that the project communicates clearly and in a timely manner issues they have. Many projects don't meet that standard for me, I still participate in them, but I do increase my noise level a bit. This project is middle of the road in that regard, so sometimes I put my noisemaker mode in gear.
|
Send message Joined: 12 Nov 07 Posts: 2425 Credit: 524,164 RAC: 0 |
I believe that MW has amassed quite a pile of data with the addition of gpu processing that they aren't exactly worried about some down time or maxing outgpu usage in down time. Also the data done now is many 100's of times faster and much more data being done as well per wu than initially. Doesn't expecting the unexpected make the unexpected the expected? If it makes sense, DON'T do it. |
Send message Joined: 15 Jul 09 Posts: 12 Credit: 45,145,989 RAC: 0 |
(snippage) Are owed nothing by this, or any other, project. You should look up what "donate" means. Or do you have a contract that states otherwise? |
Send message Joined: 24 Dec 07 Posts: 1947 Credit: 240,884,648 RAC: 0 |
Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer. |
Send message Joined: 25 Jun 10 Posts: 284 Credit: 260,490,091 RAC: 0 |
Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer. Your input is appreciated. I have read every post in this forum on the subject of the cache size. It would require a significant change to the database. Also, new work units are generated from the results of the work units that have been completed and returned. The database work is really the only hurdle. The work unit generation would just be speeded up because the more powerful cards are returning work units in under ten minutes. They would just have to bring it up slowly over a period of time to accommodate new "seed" work units. I am a much newer complainer than you, but I have done my research. Unfortunately, this conversation in the past has suffered a real high “noise” to signal ratio, as this thread has. Ultimately, I am hoping to have an intelligent conversation with the project administrators and the high speed GPU users in this forum instead of the usual "noise makers". Instead of devoting time and resources to creating a screen saver (eye candy) and generating new CPU binaries, the resources should be devoted to fixing the server and/or increasing the size of the database. Get the system working correctly, and then develop eye candy and new binaries. The new CPU binaries have created significant down time, and they really account for a small portion of the work done on a daily basis compared to the high speed GPUs. -Mike |
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,524,931 RAC: 1 |
Sounds like you have done a fair amount of research into this, are you offering to help the project administrators with the database redesign? You might not realize this, but at least some of the attitude you toss out in your message style joins you in an august group of noisemakers. Must be august, it includes me <smile>.
|
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,524,931 RAC: 1 |
I'll readily agree that the cache size here is one of the reasons my farm supports Dnetc and Collatz for more work than here. That, and of course the support those projects are able to offer for the stack of less powerful GPU's I tend to run. While there is seeming agreement regarding the limited size of the cache, it seems there is a debate with Michael regarding the simplicity (or complexity) of changing things to support a larger cache. Regarding this, having watched that discussion over the years (yup years), I tend to accept, rather than reject out of hand, the explanations offered from the project folks. Then again, I'm certainly no expert regarding database design in general, let alone the specifics of the project specific design involved. Yes the cache size is a joke for the newer cards, but the Project Admins have explained why this is needed many times - a little research is required by the newer complainers - with me being an older complainer. |
Send message Joined: 13 Aug 10 Posts: 10 Credit: 115,945,904 RAC: 0 |
To me this seem to boil down to: 1. Pick the projects that maximize your credits or 2. Pick the projects that do work you find worthwhile I'm solidly in camp #2. The "worthiness" of the MW project is far more important to me than any rank I have on some list and me losing credits when the server goes down. |
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
I support the comments made by Barry, and other, about the MW cache size, and would like to see it larger. My faster card(s) tend to be concentrated, ATM, on DNETZC and I leave my slower ATI GPU (an AGP one) to Milkyway. So, even this GPU gets affected by the restricted cache size, and the newer faster cards (568xx series) get a worse and worse deal as they are introduced (the 69xx will really feel it). I use Collatz as the back up project, and accept the position because that is how it is! I have every sympathy with "mdhittle's" (Mike is it?) points and position. Indeed I agree with most of them. But, this cache size issue has been going on for 2 years now, since the introduction of the GPU capability, which showed, and was causative, of the current cache limits. We have lived with them for this time, and, unfortunately, it is not likely to change anytime soon. I'll readily agree that the cache size here is one of the reasons my farm supports Dnetc and Collatz for more work than here. That, and of course the support those projects are able to offer for the stack of less powerful GPU's I tend to run. Go away, I was asleep |
Send message Joined: 26 Jan 09 Posts: 589 Credit: 497,834,261 RAC: 0 |
|
©2024 Astroinformatics Group