Welcome to MilkyWay@home

Questionable Cruncher Discussion Thread


Advanced search

Message boards : Number crunching : Questionable Cruncher Discussion Thread
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
HireMe.geek.nz

Send message
Joined: 12 Jun 09
Posts: 12
Credit: 384,707
RAC: 128
100 thousand credit badge10 year member badge
Message 46488 - Posted: 7 Mar 2011, 17:13:06 UTC - in response to Message 45752.  
Last modified: 7 Mar 2011, 17:15:42 UTC

Your current settings Allow up to ~10000 work units per day!

This is taken from the linked page from the post above this post :
"Maximum daily WU quota per CPU : 9948/day"

!!!!!!!!!!!!!!!!!!!!!
Tweak it down to 100 -- for everyone.
!!!!!!!!!!!!!!!!!!!!!

That will probably be a better fix, and the dodgy computers will drop out...

No fuss, no muss.

This will not be the first project to do this.
ID: 46488 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileBeyond

Send message
Joined: 15 Jul 08
Posts: 383
Credit: 501,817,790
RAC: 0
500 million credit badge10 year member badge
Message 46490 - Posted: 7 Mar 2011, 18:31:27 UTC - in response to Message 46488.  
Last modified: 7 Mar 2011, 18:34:45 UTC

Your current settings Allow up to ~10000 work units per day!

This is taken from the linked page from the post above this post :
"Maximum daily WU quota per CPU : 9948/day"

!!!!!!!!!!!!!!!!!!!!!
Tweak it down to 100 -- for everyone.
!!!!!!!!!!!!!!!!!!!!!

That will probably be a better fix, and the dodgy computers will drop out...

No fuss, no muss.

This will not be the first project to do this.

Lots of fuss and muss. Many with fast GPUs will drop out. A machine with 4 5870 cards can do in the neighborhood of 4500 WUs/day. What really needs to be done is to untie the WU quota from the number of CPU cores completely. CPU cores have nothing to do with output in this project.
ID: 46490 · Rating: 0 · rate: Rate + / Rate - Report as offensive
John Clark

Send message
Joined: 4 Oct 08
Posts: 1734
Credit: 64,228,409
RAC: 0
50 million credit badge10 year member badge
Message 46495 - Posted: 7 Mar 2011, 21:21:17 UTC - in response to Message 46490.  
Last modified: 7 Mar 2011, 21:22:21 UTC

What really needs to be done is to untie the WU quota from the number of CPU cores completely. CPU cores have nothing to do with output in this project.



That is highly unlikely to happen for all the reasons discussed here many times over the last 2 years.
Go away, I was asleep


ID: 46495 · Rating: 0 · rate: Rate + / Rate - Report as offensive
sparkler99

Send message
Joined: 6 Mar 11
Posts: 6
Credit: 59,274
RAC: 0
10 thousand credit badge8 year member badge
Message 46939 - Posted: 2 Apr 2011, 22:59:31 UTC - in response to Message 46490.  

Lots of fuss and muss. Many with fast GPUs will drop out. A machine with 4 5870 cards can do in the neighborhood of 4500 WUs/day. What really needs to be done is to untie the WU quota from the number of CPU cores completely. CPU cores have nothing to do with output in this project.


not everyone has access to gpu's i use virtual machines to crunch so only have cpu's to use and even if i used my host with access to a HD5850 due to faulty amd drivers or card i don't know i would probably end up reporting failures all the time due to driver not responding error's so i think the WUs/day should depend on reliability as im sure if its changed people will report having 10000gpu's instead of 10000cpu's




ID: 46939 · Rating: 0 · rate: Rate + / Rate - Report as offensive
KWSN-GMC-Peeper of the Castle Anthrax
Avatar

Send message
Joined: 16 Dec 10
Posts: 21
Credit: 13,709,128
RAC: 0
10 million credit badge9 year member badge
Message 49424 - Posted: 19 Jun 2011, 14:22:26 UTC
Last modified: 19 Jun 2011, 14:31:09 UTC

I don't understand the point here. If users return invalid results the WU gets issued again if necessary. In fact, as all the serious BOINC projects list as having more crunchers than work, these results aren't even on the grid. If one result is bad, the other 8 are good. (they only needed 3, but they gotta keep us busy)
Why are you all getting your panties into a twist over this and wasting the time of the people running the project?
;
ID: 49424 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Zydor
Avatar

Send message
Joined: 24 Feb 09
Posts: 620
Credit: 100,587,625
RAC: 0
100 million credit badge10 year member badgeextraordinary contributions badge
Message 49427 - Posted: 19 Jun 2011, 15:17:33 UTC - in response to Message 49424.  
Last modified: 19 Jun 2011, 15:27:20 UTC

Because the people running the project want to know of hosts who are running out of control trashing WUs, and would like to be notified when one is spotted. Its a massive waste of server assets and disc space when tens of thousands of WUs - arguably hundreds of thousands - a day get trashed for no other reason than extremely poor local PC housekeeping.

One of the major reasons for restriction on the number of WUs being sent to each cruncher is disc space issues, a major factor in that is tracking and storing a WU and its different stages in being crunched. To do so for hundreds of thousands that end up being trashed - whether by constant error or by-passing restrictions on holdings and timing out, both are the same effect - is not the best idea on the block, and anything that can be done to eliminate that - despite the pain of twisted pants - can only be a good thing.
ID: 49427 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileBlurf
Volunteer moderator
Project administrator

Send message
Joined: 13 Mar 08
Posts: 804
Credit: 26,380,161
RAC: 0
20 million credit badge10 year member badgeextraordinary contributions badge
Message 49428 - Posted: 19 Jun 2011, 15:18:19 UTC

GMC-thanks for the thoughts but the project maintains strict rules about how many WU's people can have. If people are running an app that allows someone to hoard WU's, it is not only unfair to other crunchers but could be giving false reports that skew project results.

ID: 49428 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileWerkstatt

Send message
Joined: 19 Feb 08
Posts: 350
Credit: 132,841,343
RAC: 17,917
100 million credit badge10 year member badge
Message 49442 - Posted: 20 Jun 2011, 5:49:32 UTC - in response to Message 49424.  

If users return invalid results the WU gets issued again if necessary

Can You remember the days when MW was down three or four times a week? Stuff has done their job to optimize the project and reduce data traffic, now the crunchers need to do their part.

(they only needed 3, but they gotta keep us busy)

Can You remember the days when some people compiled a app to crunch in single precision (faster) and bad wu's validated against bad wu's and good ones did not validate? It's up to the project to decide how many results are necessary.
ID: 49442 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 357
Credit: 16,320,358
RAC: 14
10 million credit badge9 year member badge
Message 49446 - Posted: 20 Jun 2011, 11:09:26 UTC

Looking at the application pages of all that hosts I think that the starting value for "max tasks per day" is waaay to high. Since here at MW most returned results get validated immediately, even for fast GPUs a starting value of 1000/GPU should be more than enough, specially considering the fact, that every valid result increases it by one and occasional errors don't reset it to the starting value like on many other projects, but only decrease it by 1. For CPUs 100 per day should be enough.

That way those hosts should be down to 1WU/day a lot faster.
.
ID: 49446 · Rating: 0 · rate: Rate + / Rate - Report as offensive
KWSN-GMC-Peeper of the Castle Anthrax
Avatar

Send message
Joined: 16 Dec 10
Posts: 21
Credit: 13,709,128
RAC: 0
10 million credit badge9 year member badge
Message 49449 - Posted: 20 Jun 2011, 13:35:14 UTC - in response to Message 49442.  

If users return invalid results the WU gets issued again if necessary

Can You remember the days when MW was down three or four times a week? Stuff has done their job to optimize the project and reduce data traffic, now the crunchers need to do their part.

(they only needed 3, but they gotta keep us busy)

Can You remember the days when some people compiled a app to crunch in single precision (faster) and bad wu's validated against bad wu's and good ones did not validate? It's up to the project to decide how many results are necessary.




Thank you Mr. Obvious. as to the '3'...it was 'pick a number' the 'tell me 3 times' model is the most common. The truth is, they probably get 5 identical verifiable results back from every WU.
As to data traffic..we're streaming HD movies these days..are you seriously gonna tell me data traffic of this nature is still an issue? LOL Server load? come ON..computing power is so dirt cheap these days it's pretty much made BOINC obsolete altogether.

The simple fact of the matter is, with a volunteer project of this nature you are NEVER going to control the users input the way we're discussing here. It is Simply Not Going To Happen without enough effort on the part of the project people to negate the entire benefit of the volunteer project.

The responses to my comments above are all non issues due to the fact that most of the 9 or 12 people doing the same WU return good results. The bad stuff should be going into the trash without human effort. If the users can really jam up the works, I submit the server side of the project isn't set up properly.

;
ID: 49449 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileSimplex0
Avatar

Send message
Joined: 11 Nov 07
Posts: 232
Credit: 178,221,048
RAC: 0
100 million credit badge10 year member badge
Message 49456 - Posted: 20 Jun 2011, 14:55:00 UTC - in response to Message 49449.  

I'm not follow you.

If I produce bad results I want to know it so that I, if possible, have a chance to correct it or maybe chose an other project that works better with my computer.


To have a the computer that constantly produce errors or no results at all is just a stupid waste of resources, both his own and the projects.
ID: 49456 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 357
Credit: 16,320,358
RAC: 14
10 million credit badge9 year member badge
Message 49460 - Posted: 20 Jun 2011, 15:49:47 UTC - in response to Message 49449.  

As to data traffic..we're streaming HD movies these days..are you seriously gonna tell me data traffic of this nature is still an issue? LOL Server load? come ON..computing power is so dirt cheap these days it's pretty much made BOINC obsolete altogether.

Yes, network load is a problem. If you need something to read about problems related to that, go to Seti forums. And computing power is not cheap, if you want to know what it costs, that is a good place to start.


The responses to my comments above are all non issues due to the fact that most of the 9 or 12 people doing the same WU return good results.

Have you ever seen a WU done by that many people? Most of them validates after the first result, some needs 2, if some errors occur than eventually more. Since MW sends out the next task for a particular WU after the first result was reported or timed out, it's highly unlikely, that you will ever see a WU successfully crunched by 9 people (that's btw maximum, 12 not possible).
.
ID: 49460 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Bill Walker

Send message
Joined: 19 Aug 09
Posts: 23
Credit: 631,303
RAC: 0
500 thousand credit badge10 year member badge
Message 49462 - Posted: 20 Jun 2011, 19:02:07 UTC - in response to Message 49449.  
Last modified: 20 Jun 2011, 19:05:28 UTC

As to data traffic..we're streaming HD movies these days..are you seriously gonna tell me data traffic of this nature is still an issue? LOL Server load? come ON


If this project had Netflix' or Youtube's hardware budget I expect things would be smoother. Don't hold your breath though. ;)

..computing power is so dirt cheap these days it's pretty much made BOINC obsolete altogether.


I really don't follow that one. BOINC exists because little projects like MW can't afford computing power. They solicit it from us.
ID: 49462 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Link
Avatar

Send message
Joined: 19 Jul 10
Posts: 357
Credit: 16,320,358
RAC: 14
10 million credit badge9 year member badge
Message 49747 - Posted: 29 Jun 2011, 12:45:24 UTC
Last modified: 29 Jun 2011, 12:47:35 UTC

Just something, that the project admins should eventually check: on some hosts the quota system seems not to work, i.e. the value for "max tasks per day" is frozen at 10000.

Examples:
Host 287332
Host 256358
Host 216388

Eventually it can be something user account related, if you look at the FreeBSD machines of this user, they are all at 10000 and have never returned a valid result, at least not with the app listed on the host application page.

I think having a workling quota system is the best way to stopp such hosts from trashing WUs, as can be seen on this two: 82319 & 233613. Here is even one, that ATM can't get even a single WU: 184480.

_
.
ID: 49747 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileSimplex0
Avatar

Send message
Joined: 11 Nov 07
Posts: 232
Credit: 178,221,048
RAC: 0
100 million credit badge10 year member badge
Message 49798 - Posted: 1 Jul 2011, 19:25:31 UTC

I reported this one http://milkyway.cs.rpi.edu/milkyway/show_host_detail.php?hostid=27701 23 days ago.

He still have 3090 wu's
He still download more than 100 wu\day

And he still have 'Number of tasks completed' = 0
Yes I know the staff is 'working on it' and that it's not your fault Blurf but is it any point at all to report an 'Owner Anonymous'?
ID: 49798 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileBlurf
Volunteer moderator
Project administrator

Send message
Joined: 13 Mar 08
Posts: 804
Credit: 26,380,161
RAC: 0
20 million credit badge10 year member badgeextraordinary contributions badge
Message 49800 - Posted: 1 Jul 2011, 22:15:53 UTC - in response to Message 49798.  

I reported this one http://milkyway.cs.rpi.edu/milkyway/show_host_detail.php?hostid=27701 23 days ago.

He still have 3090 wu's
He still download more than 100 wu\day

And he still have 'Number of tasks completed' = 0
Yes I know the staff is 'working on it' and that it's not your fault Blurf but is it any point at all to report an 'Owner Anonymous'?


Thank you for not assigning fault.

Fact is this is a new process we're trying to use to deal with people using old expired apps or hoarding WU's. As with any new process, it needs to evolve and will do so over time. I am working with the other admins so that I can have more direct control over this process (since it was my brainchild) and more access to address these issues. We will see what happens....but I say yes, please include them but understand they are more difficult to address.

ID: 49800 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profilebanditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
500 thousand credit badge10 year member badge
Message 49803 - Posted: 2 Jul 2011, 0:22:03 UTC

I am sure the Annon. computers atleast could be blocked/removed or equivalent.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 49803 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileSimplex0
Avatar

Send message
Joined: 11 Nov 07
Posts: 232
Credit: 178,221,048
RAC: 0
100 million credit badge10 year member badge
Message 49804 - Posted: 2 Jul 2011, 2:10:44 UTC

I believe the main reason is that the staff simply does not have the recourses and it can be both security and legal issues regarding giving the administration more access to handle the system and maybe this issue is no big problem after all.
ID: 49804 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profilebanditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
500 thousand credit badge10 year member badge
Message 49805 - Posted: 2 Jul 2011, 2:45:57 UTC - in response to Message 49804.  

I believe the main reason is that the staff simply does not have the recourses and it can be both security and legal issues regarding giving the administration more access to handle the system and maybe this issue is no big problem after all.

Someone has access, Travis atleast. That was all I ment.
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 49805 · Rating: 0 · rate: Rate + / Rate - Report as offensive
ProfileBlurf
Volunteer moderator
Project administrator

Send message
Joined: 13 Mar 08
Posts: 804
Credit: 26,380,161
RAC: 0
20 million credit badge10 year member badgeextraordinary contributions badge
Message 49957 - Posted: 6 Jul 2011, 20:44:06 UTC
Last modified: 6 Jul 2011, 20:47:05 UTC

Due to some personal stuff going on, my spare time is really limited this week. I'm taking a few days off from working on these. Please keep listing them and I'll get back to them early next week or maybe Sunday.

One slight procedural adjustment-don't worry about the anonymous ones. They're too difficult for me to personally address.

ID: 49957 · Rating: 0 · rate: Rate + / Rate - Report as offensive
1 · 2 · Next

Message boards : Number crunching : Questionable Cruncher Discussion Thread

©2020 Astroinformatics Group