Message boards :
Number crunching :
Why is it so hard to get work?
Message board moderation
Previous · 1 . . . 7 · 8 · 9 · 10 · 11 · 12 · Next
Author | Message |
---|---|
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,528,196 RAC: 276 |
The 'week or two' might well be longer unless the optimized ATI GPU application is in that timeframe. A lot of the load here is from the ATI crunchers -- after all, this is the ONLY project with an ATI_GPU application in the BOINC world. One really can't blame them for the load here, it is much a matter of there being no other home for those cycles. When the millenial day of ATI_GPU support existing in multiple BOINC projects occurs, then life might be sweet indeed.
|
Send message Joined: 12 Nov 07 Posts: 2425 Credit: 524,164 RAC: 0 |
To be honest, before people started using scripts to hammer the server, we were getting around 9-11 workunits a second. Now we're seeing around 6-7 workunits a second. Now shouldn't this get some/all to let off their scripts so they could actually get more work? What more proof is needed that the scripts are hindering the project? Doesn't expecting the unexpected make the unexpected the expected? If it makes sense, DON'T do it. |
Send message Joined: 3 Jan 09 Posts: 139 Credit: 50,066,562 RAC: 0 |
Well i have noticed that since the GPU code came out that i have'nt had hardly any work at all. But thats all right seti has taking up the slack and I'm all most at the half million mark. I came here when seti was having big problems, ill stay here now that Milkyway is having problems. I want to get to that half million mark here to. As the doctor said to the patient with kidney stones, this to shall pass. And besides I really want to see the finished 3D model. |
Send message Joined: 6 Apr 08 Posts: 2018 Credit: 100,142,856 RAC: 0 |
To be honest, before people started using scripts to hammer the server, we were getting around 9-11 workunits a second. Now we're seeing around 6-7 workunits a second. Proof? I wonder what 'proof' you are talking about? That the only people using scripts are those crunching GPU WUs? Or that most, by far, of crunchers here are CPU crunchers, and that most, by far of script users are CPU crunchers? Or that the user base has increased rapidly since word got out that pretty damn good credits can be had by using optimized apps at MilkyWay? And that most of these new crunchers are CPU crunchers and not GPU crunchers? And that the talk and detailed publication of script useage has made it more and more likely that more and more are taking to script usage? Or a combination of some or all of the above? Let's hear the 'proof' please, and perhaps think before making unsubstantiated allegations?" |
Send message Joined: 21 Aug 08 Posts: 625 Credit: 558,425 RAC: 0 |
The scripts simply cannot be helping, unless you are only concerned with your limited viewpoint of your own machines, and even then it might not be helping long-term. A classic example of how people don't understand how things work long-term with BOINC is the assertion that "Project 'X' is hogging my CPU" when the CPU scheduler has started trying to make sure that work gets done by deadline. The resource shares get honored over the long-term, but might not be honored to exacting precision short-term. To give another example, if there's a fire in a building and only one exit, more people can get out if people don't panic and stampede the exit. It is apparently just as hard to get you folks to understand that you might be throttling yourselves as it is to get people to remain calm in a fire... |
Send message Joined: 7 Jul 08 Posts: 47 Credit: 13,629,944 RAC: 0 |
Ice, the most recent postings from banditwolf and Brian were platform independent. It's about overloading the server. Sure, as you point out, the heavy server traffic is caused by many factors, but as they and Travis and others have pointed out, the scripts are just making things worse, and adding to the decrease in available work throughput. --Bill |
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
It is probably true that script users are slowing down the transfer of Milkyway WUs to both GPU and CPU users. But other factors enter in to the slow down including limited shared memory on the server for the WU transfer. I have this quad core running 2 projects - Malaria Control and Milkyway - concurrently on the CPU with a 50:50 share. This means that 2 cores are dedicated to each project. I have run the PC long enough to have settled down the project RACs, and for Milkyway this box gives an RAC of 8,413. When I ran all 4 cores of this CPU on Milkyway the RAC was heading up north of 16,000. This can get a goodly percentage of the slower ATI GPU outputs. I find that my CPU boxes are not having that much difficulty getting work, and maintaining their limited caches. Perhaps I am very luck getting work, but things should calm down in a few weeks time with MW_GPU. So continuous whining is not going to help your concerns or corner. Go away, I was asleep |
Send message Joined: 29 Sep 08 Posts: 1618 Credit: 46,511,893 RAC: 0 |
|
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
How nice to see everyone so happy! ;-D Good afternoon Moon. Stealth should be joining us in the other place today. Go away, I was asleep |
Send message Joined: 29 Sep 08 Posts: 1618 Credit: 46,511,893 RAC: 0 |
|
Send message Joined: 1 Sep 08 Posts: 520 Credit: 302,528,196 RAC: 276 |
I'd suggest that scripts, which might be making things unhappy, especially for those not using scripts, were in fact a reaction to what was seen as an existing difficulty in getting work. That difficulty in getting work *PRECEDED* the frequent use of scripts. The timing as I see it (others have mentioned this before in addition to me), is something like this. Last summer increasing numbers of people became aware of the high credit values offered in MilkyWay and so migrated CPU cycles this way. As the code was open for optimization, a few hardy souls figured out how to optimize the applications while producing legitimate results. At the the time these optimizations were privately used and not broadly available, but even with only a few running optimized code, the total amount of work being completed jumped big time. Adjustments were made as to credits per work units and maximum number of completed work units in order to respond to the vastly more efficient optimized code. Then, in the early fall (late September), the optimized code went public (and somewhat viral). This resulted in a huge increase in the number of work units requested and increasing the load on the servers. Also, the rapid rise of participants really took off. There were some server issues dealing with the increased load, and there was a further adjustment to credits (perhaps under influence from DA). Cache sizes were reduced as well. Later in the fall, folks optimizing the application code were able to complete an optimized application with support for double precision ATI video cards (38xx and 48xx). As this was (AND IS) the only place for ATI video cards to participate with those resources in a BOINC project, there was an additional draw to new users. By late October, early November, the vast increase in CPU and GPU resources available to run MilkyWay along with the vast increase in efficiency, basically outstripped the ability of the server have adequate work available for all comers. There were some project issues resulting from this overload. When the servers were tweaked to respond to the vastly higher demand, the work units being requested by the 4x increase in users, 10x increase in CPU efficiency and the totally new work unit capability of ATI GPU's resulted in a sparse work unit availability scenario. Which still exists today. Some users (both CPU and GPU) began then in response to the difficulty in obtaining work, to use scripts which request work from the server at scripted intervals. The effect of these scripts has been first to in effect redirect available work to those running scripts -- resulting in some folks who are running scripts achieving (with the optimized GPU application in particular) some VERY HIGH RAC's (in the >100K range). Folks with CPU's are also running those scripts, though with the significant processing difference, few if any CPU only folks can achieve RAC's in the >50K range. The contention and strife going on here at the moment has some GPU advocates (many of whom are running scripts) flailing at the CPU folks for wasting energy and whining about the morally superior way of the GPU. The CPU folks (and some few GPU folks who aren't running scripts) are complaining about the GPU folks and scriptors in general in their immoral redistribution of work. When (if) the MilkyWay_GPU project gets off the ground, it may do much to mitigate the contention -- though until that project includes an optimized ATI GPU application I doubt that will happen. Travis has indicated that when the Milkyway_GPU project is running (and I'm assuming when it is running along with an optimized ATI GPU application), the current MilkyWay project will revert to a CPU only project (via control methods as yet unannounced). That will certainly reduce the load on this (CPU) side of the research effort. Will that load reduction be adequate to render scripts of no value in getting work? Fair question -- we shall quite possibly know the answer to this in the summer. In the meantime, I think folks are wasting a LOT of organic cycles on yammering on one another. Ice, the most recent postings from banditwolf and Brian were platform independent. It's about overloading the server. Sure, as you point out, the heavy server traffic is caused by many factors, but as they and Travis and others have pointed out, the scripts are just making things worse, and adding to the decrease in available work throughput. |
Send message Joined: 13 Feb 08 Posts: 1124 Credit: 46,740 RAC: 0 |
"If you don't like my scripting screwing up the project, then you should shut up and leave..." Well that depends on your aims. Caring Sharing eco-friendly contributors who wish to ensure a constant output of scientific throughput will be welcomed of course, its just them script-kiddies who are after personal gain that'll be ditched. |
Send message Joined: 27 Aug 07 Posts: 915 Credit: 1,503,319 RAC: 0 |
As the doctor said to the patient with kidney stones, this to shall pass. Painfully. me@rescam.org |
Send message Joined: 21 Aug 08 Posts: 625 Credit: 558,425 RAC: 0 |
I'd suggest that scripts, which might be making things unhappy, especially for those not using scripts, were in fact a reaction to what was seen as an existing difficulty in getting work. That difficulty in getting work *PRECEDED* the frequent use of scripts. All technically correct, but the issue is that using scripts in the environment that we're in, which is a work shortage for everyone, is akin to saying "I don't have enough, so I'm going to try to take some of yours". That's all well and good and stuff, but I think if someone were to do this, say, in a grocery store or in Wal-Mart, people might be rightfully offended. If work was plentiful and/or shortages were few and far between, it would be a different matter. However, the shortage is constant and it takes a certain bit of gall that I don't have to attempt to butt in line like some are doing. However, getting past all of the greed aspects, there's another underlying issue...which is that in all likelihood, the scripts themselves are actually throttling the work distribution. The irony of that, if it is indeed the case, and I'm still curious about what Travis said yesterday, but if the scripts are throttling the outbound distribution, then people clamoring for more are simply shooting themselves in the feet. Of course getting people to realize this is next to impossible, probably a combination of not wanting to believe it and that they're not going to stop doing it because they know that whoever they are competing with is likely not going to stop...
I truly hope that the project gets running sometime within the next few weeks... |
Send message Joined: 20 Sep 08 Posts: 1391 Credit: 203,563,566 RAC: 0 |
..its just them script-kiddies who are after personal gain that'll be ditched. Good comment Phil, I agree. |
Send message Joined: 24 Dec 07 Posts: 1947 Credit: 240,884,648 RAC: 0 |
As I pointed out in this message and to Travis... To stop the scriptors hitting the project so hard, you could increase the minimum time between host contacts at the server end. LHC@home increased theirs to just over 15 minutes....maybe you could try 2 minutes and see what happens. I believe it is a simple server side setting. |
Send message Joined: 9 Nov 07 Posts: 151 Credit: 8,391,608 RAC: 0 |
Q. When does a work unit become 'yours'? A. When it's on your system and your crunching it! Q. When are they your groceries? A. When you've paid for them! |
Send message Joined: 21 Aug 08 Posts: 625 Credit: 558,425 RAC: 0 |
As I pointed out in this message and to Travis... That's what Bill was suggesting, and got shot down for it... Personally, I don't think 2 minutes is long enough. Needs to be at least 10. I'm sure though that 10 will cause a huge amount of complaining about how that means that someone might have 3-5 minutes of not being able to get anything, although I'm not sure how that's different from the current situation, but then again, I view this from the perspective of solving a problem, not from the eyes of a competitor in a competition. |
Send message Joined: 21 Aug 08 Posts: 625 Credit: 558,425 RAC: 0 |
Do you ever wonder why it is that movies about global destruction from a comet, asteroid, or whatever seem to initially focus on the fact that none of us make any kind of overt attempt at being cooperative with each other until disaster looms large, and even then it might take a lot of convincing? It's all about who can get the most of whatever, no matter what the detrement to others. While I don't advocate Socialism or Communism, I think it would be nice if we were not as petty and greedy all the time... Think about this: If I were to follow your logic to the fullest extent, then I would be within my rights to reach into your shopping cart and take the item that is in your cart that I want, because you haven't paid for it yet. Or how about people in a flooded area that are in need of water, and I just take all of yours and tell you "tough luck"? Behavior like this only ends up in escalation of bad feelings / hostility if it is over something very important (like food, water, etc...) However, as I said in my previous post, which you left out, the irony is that the greed might be causing a further slowdown. So much for the "more science than ever" claim too... |
Send message Joined: 12 Dec 08 Posts: 56 Credit: 269,889,439 RAC: 0 |
As I pointed out in this message and to Travis... And something I posted got shot down as well... If you limit the # of WUs per core, there should be plenty for all. 5k per CPU is a bit much, even if you run only CPUs... Click to help Seti City. |
©2024 Astroinformatics Group