Welcome to MilkyWay@home

Problem with tiny cache in MW

Message boards : Number crunching : Problem with tiny cache in MW
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · Next

AuthorMessage
Profile Dan T. Morris
Avatar

Send message
Joined: 17 Mar 08
Posts: 165
Credit: 410,228,216
RAC: 0
Message 33886 - Posted: 28 Nov 2009, 10:24:21 UTC - in response to Message 33885.  

Brian, it seems one solution that has been overlooked is cutting the credits in 1/2 again. In that case all the GPU's would be looking at other projects that pay more and those with CPU's that were only here for the science would be left.

I'd think that would be a long term solution to an overtaxed infrastructure. <smile>



I really only see two solutions.

1. Add one more extra computer to handle the load.

2. Get rid of the slow cpu support that way Science gets done much faster.

Then all that we will hear is my gpu is not fast enough for the project.

Hence no more complaining about slow and no cpu work and the credits will be the same for everyone on the project. :)

ID: 33886 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STE\/E

Send message
Joined: 29 Aug 07
Posts: 486
Credit: 576,516,164
RAC: 36,864
Message 33887 - Posted: 28 Nov 2009, 10:45:40 UTC - in response to Message 33886.  

Brian, it seems one solution that has been overlooked is cutting the credits in 1/2 again. In that case all the GPU's would be looking at other projects that pay more and those with CPU's that were only here for the science would be left.

I'd think that would be a long term solution to an overtaxed infrastructure. <smile>



I really only see two solutions.

1. Add one more extra computer to handle the load.

2. Get rid of the slow cpu support that way Science gets done much faster.

Then all that we will hear is my gpu is not fast enough for the project.

Hence no more complaining about slow and no cpu work and the credits will be the same for everyone on the project. :)


No, I don't agree with your #2 Dan, it could have a Negative Ripple through out the BOINC Projects if you Alienate the CPU User's here by cutting them off at this Project. Many of the CPU User's might just say the heck with BOINC if I'm going to be denied Work at certain Projects because it interferes with the High & Mighty GPU User's. They have just as much right to the Wu's as the GPU User's have and they were the ones that got the Project started in the first place so to bar them now would be like throwing them under the Bus to satisfy GPU User's.

What would be next if you Bar the CPU User's, the slower GPU Users ??? then the Semi slow users and then finally it would be well if you don't have a 2011 ATI 6990 Super Duper 16 Core 20GB Memory Video Card we're not going to give you any work.
ID: 33887 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 33889 - Posted: 28 Nov 2009, 11:25:10 UTC - in response to Message 33887.  

Brian, it seems one solution that has been overlooked is cutting the credits in 1/2 again. In that case all the GPU's would be looking at other projects that pay more and those with CPU's that were only here for the science would be left.

I'd think that would be a long term solution to an overtaxed infrastructure. <smile>



I really only see two solutions.

1. Add one more extra computer to handle the load.

2. Get rid of the slow cpu support that way Science gets done much faster.

Then all that we will hear is my gpu is not fast enough for the project.

Hence no more complaining about slow and no cpu work and the credits will be the same for everyone on the project. :)


No, I don't agree with your #2 Dan, it could have a Negative Ripple through out the BOINC Projects if you Alienate the CPU User's here by cutting them off at this Project. Many of the CPU User's might just say the heck with BOINC if I'm going to be denied Work at certain Projects because it interferes with the High & Mighty GPU User's. They have just as much right to the Wu's as the GPU User's have and they were the ones that got the Project started in the first place so to bar them now would be like throwing them under the Bus to satisfy GPU User's.

What would be next if you Bar the CPU User's, the slower GPU Users ??? then the Semi slow users and then finally it would be well if you don't have a 2011 ATI 6990 Super Duper 16 Core 20GB Memory Video Card we're not going to give you any work.

It makes me wonder what distributed processing (as in BOINC) is all about. Whether it's about turning up and helping Milkyway with spare processing ability, or whether it's about demanding that Milkyway changes it's research methods, length of WUs, credits, who can crunch where, etc just because some have a stack of PCs with or without GPU's attached.


ID: 33889 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Dan T. Morris
Avatar

Send message
Joined: 17 Mar 08
Posts: 165
Credit: 410,228,216
RAC: 0
Message 33893 - Posted: 28 Nov 2009, 12:59:05 UTC
Last modified: 28 Nov 2009, 13:01:36 UTC

My Number 2. item was meant as sarcasm. poor attempt I might add.

I just get very weary of those who think that we should not progress with faster application for getting work done at an accelerated step. And then when we do some complain that we get too much credit. Its a no win scenario...Go figure.:)

But one thing that should be understood that if you have a slow computer don't expect the project to go slower.
ID: 33893 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 33895 - Posted: 28 Nov 2009, 15:15:03 UTC

I think projects should have a cpu option or ability and the future is most likely with gpus so they should work towards that too. If no project used cpus anymore than a whole lot of computers would sit idle that could do something. It may be a small part compared to gpus, but it is more than without them. Why dump cpu's already, now in 10 years who knows, but cpus are still viable especialy with higher end ones such as the i7.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 33895 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 33897 - Posted: 28 Nov 2009, 16:39:31 UTC - in response to Message 33895.  
Last modified: 28 Nov 2009, 16:40:42 UTC

I think projects should have a cpu option

I continue to find this staggering. How many people just happen to have a double precision GPU that happens to be ATI or NVIDIA of a small particular model amongst myriads of other GPUs? You may as well throw distributed processing out completely and just have a small select club of people who buy expensive GPUs just to show off how big their computing power is.

Projects should ALWAYS have a CPU option, or I'm the first one outta here.

ID: 33897 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile The Gas Giant
Avatar

Send message
Joined: 24 Dec 07
Posts: 1947
Credit: 240,884,648
RAC: 0
Message 33899 - Posted: 28 Nov 2009, 19:04:24 UTC - in response to Message 33897.  

I think projects should have a cpu option

I continue to find this staggering. How many people just happen to have a double precision GPU that happens to be ATI or NVIDIA of a small particular model amongst myriads of other GPUs? You may as well throw distributed processing out completely and just have a small select club of people who buy expensive GPUs just to show off how big their computing power is.

Projects should ALWAYS have a CPU option, or I'm the first one outta here.

LOL...there's something we've seen before!

There is one project that is high end GPU only and that is GPUGRID. They started as a PS3 only project then released a CUDA app and now their CUDA app will only run on the later GPUs. So my CUDA capable 9600 got left behind. Ho hum, it appears as though they are getting along quite well with out it or my CPUs.

If a project wants to do anything they like with their app, alienating a subset of crunchers along the way, I don't think the loss would be felt all that much.

We need to get over ourselves and let the projects do what they want, hopefully with them listening a little to what their crunchers are suggesting.

I'm suggesting that MW review it's 6 wu/cpu policy in light of the fact that they get a large majority of their work completed these days by GPUs.

To help, maybe Travis could spend an hour or so reviewing the lengths of outages in the last 6 to 12 months for frequency and length of the outages and let us know the results. In my case any outrage longer than 10 minutes would mean my quad core with 2 GPUs was out of work and an outage longer than 20 minutes would mean my old P4 with the 3850 was out of work. So extending this out to GPU crunchers in general and making some assumptions, it implies that any outrage longer than 20 minutes would mean that all GPU crunchers would have run out of cached work. I wonder how often an outrage of 20 minutes or longer has occurred in the last 6 to 12 months?

Would increasing the amount of cached wu's mean that the overall crunching availability would have increased from the mid 90%'s to the high 90%'s and would it be worth the effort?

Live long and BOINC.
ID: 33899 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 33900 - Posted: 28 Nov 2009, 19:30:29 UTC - in response to Message 33899.  
Last modified: 28 Nov 2009, 19:31:20 UTC

Projects should ALWAYS have a CPU option, or I'm the first one outta here.


There is one project that is high end GPU only and that is GPUGRID.

Exactly my point. I don't crunch GPUGRID any more since they made it high end GPU, as opposed to a lesser requirement prevously which my onboard laptop NVIDIA GPU could handle (ie, not bought in and not bolted in just to run BOINC). And LOL or not, make Milkyway@home GPU only and I'm outta here. This drive to satisfy high end crunchers and suggestions that CPU crunching is getting in the way is not for me.

ID: 33900 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 33903 - Posted: 28 Nov 2009, 21:51:44 UTC - in response to Message 33899.  
Last modified: 28 Nov 2009, 21:52:31 UTC


I'm suggesting that MW review it's 6 wu/cpu policy in light of the fact that they get a large majority of their work completed these days by GPUs.

To help, maybe Travis could spend an hour or so reviewing the lengths of outages in the last 6 to 12 months for frequency and length of the outages and let us know the results. In my case any outrage longer than 10 minutes would mean my quad core with 2 GPUs was out of work and an outage longer than 20 minutes would mean my old P4 with the 3850 was out of work. So extending this out to GPU crunchers in general and making some assumptions, it implies that any outrage longer than 20 minutes would mean that all GPU crunchers would have run out of cached work. I wonder how often an outrage of 20 minutes or longer has occurred in the last 6 to 12 months?

Would increasing the amount of cached wu's mean that the overall crunching availability would have increased from the mid 90%'s to the high 90%'s and would it be worth the effort?



:facepalm:

Travis has already told you that giving you more WUs in your cache ends up bringing the project to a halt due to the server not being able to handle it.

In other words, giving you all more of these small tasks will end up with you having MORE outages, so it would mean you had less work in total. The "crunching availability" would decrease, not increase...
ID: 33903 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bryan

Send message
Joined: 26 Jun 09
Posts: 47
Credit: 276,827,695
RAC: 0
Message 33904 - Posted: 28 Nov 2009, 21:52:35 UTC

And of course there is AQUA that requires high end CPUs. They deny the non multicore CPUs and GPU users. Their project is also time sensitive.
Bryan

ID: 33904 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile The Gas Giant
Avatar

Send message
Joined: 24 Dec 07
Posts: 1947
Credit: 240,884,648
RAC: 0
Message 33907 - Posted: 28 Nov 2009, 23:57:51 UTC - in response to Message 33900.  

Projects should ALWAYS have a CPU option, or I'm the first one outta here.


There is one project that is high end GPU only and that is GPUGRID.

Exactly my point. I don't crunch GPUGRID any more since they made it high end GPU, as opposed to a lesser requirement prevously which my onboard laptop NVIDIA GPU could handle (ie, not bought in and not bolted in just to run BOINC). And LOL or not, make Milkyway@home GPU only and I'm outta here. This drive to satisfy high end crunchers and suggestions that CPU crunching is getting in the way is not for me.

As I said "If a project wants to do anything they like with their app, alienating a subset of crunchers along the way, I don't think the loss would be felt all that much." Which for you and me is a case in point. I don't think the loss of 2 low end GPUs hurt them in the slightest...or even 20 or 30.
ID: 33907 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile The Gas Giant
Avatar

Send message
Joined: 24 Dec 07
Posts: 1947
Credit: 240,884,648
RAC: 0
Message 33908 - Posted: 28 Nov 2009, 23:58:53 UTC - in response to Message 33903.  


I'm suggesting that MW review it's 6 wu/cpu policy in light of the fact that they get a large majority of their work completed these days by GPUs.

To help, maybe Travis could spend an hour or so reviewing the lengths of outages in the last 6 to 12 months for frequency and length of the outages and let us know the results. In my case any outrage longer than 10 minutes would mean my quad core with 2 GPUs was out of work and an outage longer than 20 minutes would mean my old P4 with the 3850 was out of work. So extending this out to GPU crunchers in general and making some assumptions, it implies that any outrage longer than 20 minutes would mean that all GPU crunchers would have run out of cached work. I wonder how often an outrage of 20 minutes or longer has occurred in the last 6 to 12 months?

Would increasing the amount of cached wu's mean that the overall crunching availability would have increased from the mid 90%'s to the high 90%'s and would it be worth the effort?



:facepalm:

Travis has already told you that giving you more WUs in your cache ends up bringing the project to a halt due to the server not being able to handle it.

In other words, giving you all more of these small tasks will end up with you having MORE outages, so it would mean you had less work in total. The "crunching availability" would decrease, not increase...

Can you please read my posts on this subject before replying....
ID: 33908 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Angus

Send message
Joined: 8 Nov 07
Posts: 20
Credit: 257,763
RAC: 0
Message 33909 - Posted: 28 Nov 2009, 23:59:03 UTC - in response to Message 33903.  



Travis has already told you that giving you more WUs in your cache ends up bringing the project to a halt due to the server not being able to handle it.

In other words, giving you all more of these small tasks will end up with you having MORE outages, so it would mean you had less work in total. The "crunching availability" would decrease, not increase...


Maybe if BOINC didn't use a hobbyist database and the projects didn't use rinky-dink servers the thing would work better.
ID: 33909 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GalaxyIce
Avatar

Send message
Joined: 6 Apr 08
Posts: 2018
Credit: 100,142,856
RAC: 0
Message 33910 - Posted: 29 Nov 2009, 0:37:38 UTC - in response to Message 33907.  
Last modified: 29 Nov 2009, 0:41:54 UTC

Projects should ALWAYS have a CPU option, or I'm the first one outta here.


There is one project that is high end GPU only and that is GPUGRID.

Exactly my point. I don't crunch GPUGRID any more since they made it high end GPU, as opposed to a lesser requirement prevously which my onboard laptop NVIDIA GPU could handle (ie, not bought in and not bolted in just to run BOINC). And LOL or not, make Milkyway@home GPU only and I'm outta here. This drive to satisfy high end crunchers and suggestions that CPU crunching is getting in the way is not for me.

As I said "If a project wants to do anything they like with their app, alienating a subset of crunchers along the way, I don't think the loss would be felt all that much." Which for you and me is a case in point. I don't think the loss of 2 low end GPUs hurt them in the slightest...or even 20 or 30.

Yes. Of course you are right. We can come and go by choice, and they can do as they wish with their projects. We can of course help the projects with our advice and kind supporting words. Or we could push for changes that suit just us and embelish our egos. Or we can do nothing and wait for the sky to fall on our heads.

At the end of the day what really matters is what we achieved. Did we help science? Did we understand any of it? How much? Did it amount to anything? Were we ever a top cruncher? You know, things to brag to your grandchildren about in your old age... ;)

ID: 33910 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 33921 - Posted: 29 Nov 2009, 5:16:52 UTC - in response to Message 33908.  
Last modified: 29 Nov 2009, 6:05:07 UTC


I'm suggesting that MW review it's 6 wu/cpu policy in light of the fact that they get a large majority of their work completed these days by GPUs.

To help, maybe Travis could spend an hour or so reviewing the lengths of outages in the last 6 to 12 months for frequency and length of the outages and let us know the results. In my case any outrage longer than 10 minutes would mean my quad core with 2 GPUs was out of work and an outage longer than 20 minutes would mean my old P4 with the 3850 was out of work. So extending this out to GPU crunchers in general and making some assumptions, it implies that any outrage longer than 20 minutes would mean that all GPU crunchers would have run out of cached work. I wonder how often an outrage of 20 minutes or longer has occurred in the last 6 to 12 months?

Would increasing the amount of cached wu's mean that the overall crunching availability would have increased from the mid 90%'s to the high 90%'s and would it be worth the effort?



:facepalm:

Travis has already told you that giving you more WUs in your cache ends up bringing the project to a halt due to the server not being able to handle it.

In other words, giving you all more of these small tasks will end up with you having MORE outages, so it would mean you had less work in total. The "crunching availability" would decrease, not increase...

Can you please read my posts on this subject before replying....


Can you please read Travis' posts on this subject before replying?

Could you PLEASE try to figure out that he is trying to make sure that you have a steady amount of work, even if it is less than what you would like to see?

Could you PLEASE try to figure out that I'm actually not just advocating something advantageous to myself, but advantageous to YOU as well because having work continuously is, well, better than having a lot, then none for a while, then some, then none, then a little, then none, then a lot, then none...?

Again, he has told you all that there are two separate reasons why he doesn't increase the cache. You are choosing to look at only one of them, the need for faster turnaround times, and are saying that it doesn't make sense because GPUs are faster.

If GPU users were given 20x the current amount, that would work out to a 3-hour cache, which I guarantee will still not be "satisfactory" to some people. To make people happier, it would likely need to be 40x the current amount, or 6-hour caches. Even then people would likely still be grumpy, but it would be "a start"...

The problem is that you are completely ignoring the second reason why he isn't increasing caches, which is that what happens is the result table gets to the point where it doesn't function properly and causes the entire project to become slow, run out of work, etc, etc, etc...

The project needs to concern itself with a good, steady, manageable level of work being done. Increasing the cache to placate certain people works against that goal.
ID: 33921 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PeteS

Send message
Joined: 19 Mar 09
Posts: 27
Credit: 117,670,452
RAC: 0
Message 33991 - Posted: 30 Nov 2009, 10:53:10 UTC
Last modified: 30 Nov 2009, 11:04:23 UTC

I understand that fast turnaround times are important (not even going to the fact that slow Quad CPU's take ages to return the wu's).

BUT defining the amount of WUs to send to GPUs by the amount of CPU cores is just flawed by nature. I have f.ex. two GPU crunching boxes with AMD A64 one core CPUs with powerful GPUs, these have some five minute of MW work cache, that is just ridiculous. Compare to GPUGrid with two WU's / GPU and these WU's run some 10 hours. They also depend on results returned to generate new, yes?



-------
edit, read only with time:) Just read all the fighting below. I think this project is already only good for a specific group of people. To crunch here means you need to babysit computers, work hard to get a working OS/Driver/ATI app configuration just to get the app running decently. If you want to attract masses this should work as reliably as WCG f.ex. where you can keep Boinc Mgr in simple mode. The app's never crash, no manual intervention needed.

I find other needs more important when selecting my driver and OS, I want the latest stuff. Not legacy old versions just to be able to crunch one project. Most of us pay high electricity bills and get new components so that we can do this kind of charity work called Distributed Computing. But we should be provided with good support, stable applications and so on for our free help... /end
ID: 33991 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bymark
Avatar

Send message
Joined: 6 Mar 09
Posts: 51
Credit: 492,109,133
RAC: 0
Message 34015 - Posted: 30 Nov 2009, 19:18:02 UTC - in response to Message 33991.  
Last modified: 30 Nov 2009, 19:34:35 UTC

I am happy here if work units going to be 4 times bigger, as told on the front page.

edit:
Got my first "big" one: 30 Nov 2009 19:31:51 UTC Completed and validated 192.50 1.03 213.76
ID: 34015 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bryan

Send message
Joined: 26 Jun 09
Posts: 47
Credit: 276,827,695
RAC: 0
Message 34021 - Posted: 30 Nov 2009, 20:21:47 UTC - in response to Message 34015.  

wow the big wu are a blessing! Thank you Travis!!!
Bryan

ID: 34021 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
localizer

Send message
Joined: 28 Jan 08
Posts: 40
Credit: 379,931,801
RAC: 0
Message 34022 - Posted: 30 Nov 2009, 20:40:32 UTC

4x longer and credit accordingly. Like it.

ID: 34022 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Temujin

Send message
Joined: 12 Oct 07
Posts: 77
Credit: 404,471,187
RAC: 0
Message 34023 - Posted: 30 Nov 2009, 21:02:57 UTC - in response to Message 34015.  

I am happy here if work units going to be 4 times bigger, as told on the front page.

Working well here, thanks Travis
ID: 34023 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · Next

Message boards : Number crunching : Problem with tiny cache in MW

©2024 Astroinformatics Group