Welcome to MilkyWay@home

Marked as Invalid? (Part 2)


Advanced search

Message boards : Number crunching : Marked as Invalid? (Part 2)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5

AuthorMessage
ProfileTravis
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 30 Aug 07
Posts: 2046
Credit: 26,480
RAC: 0
10 thousand credit badge10 year member badge
Message 38572 - Posted: 11 Apr 2010, 8:14:13 UTC - in response to Message 38570.  


So how does this work?

on this WU I got marked as invalid

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=91766141

The first person reporting was stock .23 application on a 5800.

I reported second using the opti .23 application on a 5800.

The third result was someone running the .20b application on a 5800 and they and the first person got marked as valid and I got marked as invalid.

It seems the validator is doing something wanky:

http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=1658&nowrap=true#38383

http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=1658&nowrap=true#38390



If you read the news, theres an issue with some of the 0.23 applications not correctly updating their brookX.dll file, which means they can return the same result as a 0.20 application, and thus quorum against valid results.

The validator is doing exactly what it should -- matching fitnesses and making quorums. The problem is that if enough computers out their are returning bad results they can quorum against good results.

For the most part we're down to < 4% invalid results which means most of the people running the bad 58x0 application have upgraded


Does not that mean that if the results that was validated with each other all was using the same bad application they could deliver a result that was equal to each other but had errors that was to large and they would still be marked as 'Valid'? Or in other words you will end up with all results marked 'Valid' but they are all incorrect.

How do you know that the results you are collecting have errors within the expected limits?


That's actually a really good question :)

There's a couple answers to these. Mainly, our evolutionary algorithms are self healing -- if a incorrect result does make it into the population, so long as it fitness isn't extremely good (which would be noticeable to us), it will be replaced later by another correct individual with a better fitness.

Also, we have some other fail safes. When we think a search has ended, we have an entire population of similar individuals to look at. If one is off (parameters are significantly different for similar fitnesses), we can run it on one of our machines with the stock app to check and see if it really was a valid result.

Likewise, for astronomical publications the results are double-checked vs the our stock applications here. Sometimes we even run a local search method (like gradient descent, or the simplex method -- like your name :P ) from the point found by the evolutionary algorithms in order to further squeeze as much accuracy as we can out of it.

But anyways, you're right that the quorum system isn't a perfect method of guaranteeing the validity of a result. However, the self healing nature of the EAs we're using along with double checking of the final results makes it more than good enough. Most of the validation is to prevent scamming and because people are so sensitive about the credit system :P Like I've said before, on our end the only validation we really need is for individuals that get inserted into the populations.
ID: 38572 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileSimplex0
Avatar

Send message
Joined: 11 Nov 07
Posts: 232
Credit: 178,229,009
RAC: 0
100 million credit badge10 year member badge
Message 38573 - Posted: 11 Apr 2010, 8:53:13 UTC

Thank you Travis for taking the time to give me an insight
in how the procedure works.

I wish that there where more guy's like you and CP so that
the ATI\AMD cards could be used to it's full potential in more
scientific project's than just one.
ID: 38573 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5

Message boards : Number crunching : Marked as Invalid? (Part 2)

©2020 Astroinformatics Group