Are these only test units just to verify the validator? I've had 2 issues with them - Completed, validation inconclusive and when the consensus does come in, I get - Completed, can't validate due to the settings of - max # of error/total/success tasks 1, 6, 1 errors Too many success results

Are these only test units just to verify the validator? I've had 2 issues with them - Completed, validation inconclusive and when the consensus does come in, I get - Completed, can't validate due to the settings of - max # of error/total/success tasks 1, 6, 1 errors Too many success results

Ahh, thats the issue. I was wondering why WUs weren't coming back for additional validation. This should be fixed with new WUs.

Validation will now work as follows: Every result that could improve one of our searches will be validated (with a min quorum of 3[...]

So what about the wus that say "Checked, but no consensus yet" also come up as pending. Will they be granted credit eventually?
____________
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.

Most of mine (edit: many, not most) are ending up with "Completed, validation inconclusive". 4 results, with a max of 4, and the no one gets any credits. Something doesn't seem right with this scheme.

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=82607813

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=87913982
____________

Okay, I've run across a few with:

minimum quorum 3
initial replication 5
max # of error/total/success tasks 1, 6, 6

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=90198132
http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=90151888

Max # of error 1....is that really what you want? Also, why is my machine marked as invalid? It is using the stock app, with a 5870.
____________

Looks to be more hardware related than application related to me. Many of the results marked invalid are using the stock application that is automatically downloaded.

Cypress class 58xx cards are validating against other Cypress if they are in the majority reported first and 48xx cards are being marked as invalid. If 48xx cards are the majority reported first then they validate against other 48xx and 58xx are marked invalid. Uncertain about NVIDIA, but I think they validate against 48xx but not 58xx.

Not sure if this is exactly correct in all details but it's something like that.

Ouch. If correct, that's going to make this messy. Does HR (homogeneous redundancy) exist for GPUs?

Edit: From what I have just learned, the answer is maybe.
____________

Seems to me that someone should do a standalone test and compare results between 48xx and 58xx again, just to make sure everything works properly.

As stated in corresponding thread, 5xxx ability to work is very questionable right now. Bugs in ATI's OpenCL SDK implementation. They promised to fix those in new SDK release, will see...

Looks to be more hardware related than application related to me. Many of the results marked invalid are using the stock application that is automatically downloaded.

Cypress class 58xx cards are validating against other Cypress if they are in the majority reported first and 48xx cards are being marked as invalid. If 48xx cards are the majority reported first then they validate against other 48xx and 58xx are marked invalid. Uncertain about NVIDIA, but I think they validate against 48xx but not 58xx.

Not sure if this is exactly correct in all details but it's something like that.

HD5870 running ati13ati app. factory oc (875, 1250).

Getting a 1 invalid/1 pending/1 valid split at the moment (roughly).

Also noticing the 48xx/cpu relationship as well.

IS validator looking at time taken?

Shouldn't it read:

Completed, waiting for validation

instead of

Completed, validation inconclusive

if one result is returned and no other results are reported?

[EDIT]: Fixed a typo...

Just counting pages at 20 tasks each, I am currently at 9 valid and 2 invalid. 82% valid tasks, is not going to work in the long run, obviously. But I'll hang around for the shakedown.
____________

If not, what exactly is used?

The only thing used is the fitness value reported by the application. If the fitness returned is within 10e-11 of 2 other fitnesses for the quorum, it's valid.

Here is another one:

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=89444466

errors: Too many success results

One 0.19 result on a CPU and two 0.20b results on a HD 47xx/48xx and on a HD 58xx lead to this weird result.

Here is another one:

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=89444466

errors: Too many success results

Here is another one:

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=89444466

errors: Too many success results

Bug already fixed.

Check the third post in this thread.

82% valid tasks, is not going to work in the long run, obviously. But I'll hang around for the shakedown.

The 1 max error is because our application really shouldn't error out. Chances are if there's an error it was our fault (ie, a badly generated or specified workunit), and we don't want to send out more bad WUs.

I don't mind upping it to 3 if people would prefer that, however.

Here's another example of the 'Too many success results' bug. Note that one of the victims actually invested over a day of CPU time for no reward. I guess he wont be particularly impressed.

I wonder why people still persist with slow CPUs on a project like this?

And here's one that is rather more important that I've just noticed. Looks like Travis has set 3,6,6 for errors/total/success and this quorum has failed with the error message "Too many total results". However there are only 6 tasks listed in the quorum, one of which is a 'client detached'. Perhaps that triggered an attempt to send out a 7th copy which junked the whole quorum. Because of the conflict between 48xx and 58xx, there mustn't have been 3 agreeing results at the time the attempt was made to send out the 7th copy. Until things are sorted regarding validation, perhaps it should be 3,9,6 rather than 3,6,6 to prevent this problem.

EDIT: If you think about it, it makes sense to have the 'total' equal to the sum of 'errors' and 'success' so that all bases are covered.
____________
Cheers,
Gary.

So it is possible that most of these results would be accurate to 10e-11 if compared only against an unoptimised CPU application but the results from ATI 48xx, NVIDIA and optimised CPU applications are on one side of the required fitness value and the results from ATI 58xx and 5970 are on the other side. Therefore the difference between these two sets of hardware is less accurate than 10e-11, even though individual results compared against an unoptimised CPU application may still have the required accuracy.

And this is the reason that some projects that validate results with a quorum need to use homogeneous redundancy to ensure accurate results on different types of hardware?

Is the "Canonical" result used in anyway in determining the validity of results? I haven't checked too many, but have noted that the first result in sometimes determines validity or invalidity.

Strangely enough, the main variance that I can see is if the first in is either 48xx or 57xx/58xx and made the Canonical result, then all wus returned with that series card is validated whereas higher (or lower) cards are invalidated. All other data showing in the text file we get to see is usually the same. It is expecially annoying when the seen calcs return 10e-13 on the GPU FPU, the required figure on the GPU, as with the canonical whereas we are ruled invalid. Stars are usually at 10-e9

Possibly invalid opinion, but sometimes a coincidence ......

Is the "Canonical" result used in anyway in determining the validity of results? I haven't checked too many, but have noted that the first result in sometimes determines validity or invalidity.

....
It is expecially annoying when the seen calcs return 10e-13 on the GPU FPU, the required figure on the GPU, as with the canonical whereas we are ruled invalid. Stars are usually at 10-e9.

Oh well...NNT until the problems with the validator have been overcome.

I'm still gathering up a lot of "can't validate" messages. What does "check skipped" mean anyway?

Oh well...NNT until the problems with the validator have been overcome.

I think all these examples are definitely helping, at least. Other than that, it may be best to wait for the new application versions if you hate losing crunching time (if you don't, I'm sure Travis would appreciate your continued help testing).

I'm still gathering up a lot of "can't validate" messages. What does "check skipped" mean anyway?

http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=90246907

This happens in stock application? Interesting

http://milkyway.cs.rpi.edu/milkyway/result.php?resultid=96603214

It says wu error and lists a series of app_info.xml . I'm running stock applications.

Ok. I edited the previous post as you asked me to do. It says too many total results. Status: Completed, can't validate

What they have in common is ignoring unknown input argument in app_info.xml. Normal behaviour?

Ok. I edited the previous post ...

as you asked me to do.

It says too many total results. Status: Completed, can't validate

What they have in common is ignoring unknown input argument in app_info.xml. Normal behaviour?

Dumb Question time:

1) as multiple validations are now required, are duplicate/triplicate wus being issued, or is just one being issued, returned and then re-issued until the result quorum is met?

2) As some of the errors/failure to validate appear to be 'clashes' between 47xx/48xx and 58xx series cards, is there anyway to send wus to the same version of cards?

3) What is the precise reason for the failure of the above cards to actually agree. Could it be something as simple as using older drivers/CAL runtime 1.4.3/4xx

4)I have noticed in the 20-odd validations I have based these questions on that nVidia 260s and 285s are erroring out or completing the wu but not being granted credit. In the <stderr_txt>, the wu knows where the GPU is located but can't use it, and yet in some cases there are times given in the job summary (where all returned results are recorded)

Dumb statement: 58xx cards validated regardless of GPU or Memory frequencies or App used or CAL Runtime: 1.4.5xx have been predominant.

The above may not be useful, but am looking at it from the what has been validated, not the what hasn't or is in pending; the majority of pending giving rise to Q 1). As for pending, it appears to be mainly the clash between cards (nVidia/ATI and ATI series) and CPU produced results and the Validator trying to get two or three matching results.

And yet, I've had results Validated on just my own wu, without any correlation from another source. So maybe there is a "magic number".

As I said above, dumb questions......

Well, here's the first "magic number" failure I have in my records:

Job No 90254161 - completed - can't Validate 3,6,6

47xx/47xx 1.4.415 -3.169546804554361 standard app
58xx 1.4.533 -3.169546898027065 standard app
58xx 1.4.553 -3.169546898027031 standard app
47xx/48xx 1.4.515 -3.169546804554371 Anon app
58xx 1.4.515 -3.169546898027031 Standard app
47xx/48xx 1.4.553 -3.169546804554361 standard app

(hoping it makes sense after posting)

BAsically of the six issues, there were two agreements in both 47xx/48xx and also 58xx. In both cases, the drivers made no difference as both "valid" and 'invalid' results within same series missed out for some other reason whilst using the same CAL runtime as one of the 'valid' wus.

Or to put it another way..... it may not be directly related to drivers, but the architecture within the actual cards.
____________
You should see the world from my eyes.

82% valid tasks, is not going to work in the long run, obviously. But I'll hang around for the shakedown.

Well I'm out of here for the time being as 82% is not satisfactory for me! I realize that MilkyWay is still classed (as far as I know) as an Alpha project, but IMHO it is mature enough that they shouldn't be running tests in a production environment - at least some of these bugs (if not the majority of them) SHOULD have been caught in testing before releasing this new version validator into the wild. See y'all later.

Right now it looks like the problem isn't the validator but the (optimized?) GPU applications.

Since I don't know if purges are running real quick, and I don't know what the major amount of noise is in this thread since I last read it, I'm going ahead and posting this. It will likely be formatted badly, and may already be covered by the numerous postings, but I just wanted to state that it's quite unfair to me to have an app that is known to be working fine and spend 4.5 hours on a task for zip, zap, zero... Oh, and in case the 4.5 hours didn't tell you which system is mine, it's the non-GPU system, the first one in the quorum...

name de_s11_3s_free_6_1544383_1270347650
application MilkyWay@Home
created 4 Apr 2010 2:20:50 UTC
minimum quorum 3
initial replication 4
max # of error/total/success tasks 3, 6, 1
errors Too many success results
This is displayed on the workunit pageTask ID
click for details Computer Sent Time reported
or deadline
explain Status Run time
(sec) CPU time
(sec) Claimed credit Granted credit Application
95656877 26452 4 Apr 2010 2:22:25 UTC 5 Apr 2010 5:49:34 UTC Completed, can't validate 0.00 16,347.27 70.47 0.00 Anonymous platform
96480761 26133 5 Apr 2010 5:50:54 UTC 5 Apr 2010 7:17:50 UTC Completed, can't validate 216.20 212.47 1.11 0.00 MilkyWay@Home v0.21 (ati13ati)
96480762 141414 5 Apr 2010 5:50:38 UTC 5 Apr 2010 6:27:01 UTC Completed, can't validate 89.52 87.25 0.61 0.00 MilkyWay@Home v0.21 (ati13ati)

I'm not using an optimized app, I'm using what is given to me by the project. I have NEVER before had invalid results, after the "upgrade" I was getting many.

On another note, does anyone know if the 48xx or the 58xx ATI GPU is the one validating correctly vs the stock application?

Seams to be a lot of wasted computer power.
Why do you keep on sending out new wu's?

What I'm trying to figure out is if say a stock application result and 2 48xx GPU results come back, do they validate to a quorum? (that would mean the issue is with the 58xx GPU application).

Otherwise, if a stock application result and 2 58xx GPU results make a quorum, that would mean the 48xx GPU application is the problem.

As far as I know, all the applications we provide (the stock applications) provide results with 10e-13 of each other, regardless of them being on a CPU or GPU.

Double precision is accurate to the 12th decimal. What about these "completed, validation inconclusive" ones? Are they waiting for the other quorums to be validated?

As far as I know, all the applications we provide (the stock applications) provide results with 10e-13 of each other, regardless of them being on a CPU or GPU.

As far as I know, all the applications we provide (the stock applications) provide results with 10e-13 of each other, regardless of them being on a CPU or GPU.

Awhile back you said you only needed to the 8th place and would like to get to the 10th. That is what I thought all of these applications were based off of.

As far as I remember, we've always wanted 10+ degrees of precision. If we were content with 8 we could be using single precision applications.

Two comparisons which include unoptimised CPU application in the quorum.

wuid=90266299
-3.16876629868666400000 48xx Anonymous v0.20b Invalid
-3.16876629868695500000 CPU Core2 Duo E6750 v0.19 Invalid
-3.16876639500725200000 58xx Anonymous v0.22 Valid
-3.16876639500725200000 58xx Anonymous v0.22 Valid
-3.16876639500713000000 58xx Anonymous v0.20b Valid

wuid=90249810
-3.17153809050878900000 48xx v0.21 (ati13ati) Valid
-3.17153809050878900000 48xx v0.21 (ati13ati) Valid
-3.17153818333960200000 58xx Anonymous v0.22 Invalid
-3.17153809050889600000 CPU Core2 Duo E6750 v0.19 Valid

48xx results agree with stock CPU application results to 12 decimal places but 58xx results only agree with stock CPU application results to 6 decimal places.

I do not know if results from an unoptimised stock CPU application are considered to have the highest degree of accuracy, so I will not draw any conclusions, except to repeat my original belief that the differences seen are hardware based and not application based. It appears that Cypress based ATI cards give different results to other hardware classes.

I´m wondering about the "new" observation that different architectures lead to different results even with the same application. I´m crunching CPDN for about five years and hardly have seen two identical or nearly identical results even with the same application (because the applications are closed). Maybe the goal of accuracy is to challenging.

In my ten years of DC I remember a project (but I don´t know which) that sent WUs only to hosts with the same CPU knowing that AMD and Intel produces different results. Of course remember the pentium bug on the first pentiums ;-)

I've posted some results including the optimized CPU and CUDA application in the other thread. Only the HD5800 series GPUs deviate substantially. All others are well within the bounds set by the project.

There is something fishy going on for sure. Bad thing is I have no idea what. I guess the best would be if the HD5800 users take a short break or support another project until we figure out what is going on here.

I´m wondering about the "new" observation that different architectures lead to different results even with the same application. I´m crunching CPDN for about five years and hardly have seen two identical or nearly identical results even with the same application (because the applications are closed). Maybe the goal of accuracy is to challenging.

In my ten years of DC I remember a project (but I don´t know which) that sent WUs only to hosts with the same CPU knowing that AMD and Intel produces different results. Of course remember the pentium bug on the first pentiums ;-)

Ok, well I just set all my 5850s to NNW. I have been using the optimized .20b application.... Let us know when it is safe to return :)

HD3800/4700/4800 GPUs return the exact same fitness value as CPUs with my versions. So at least here it is possible. I've told Anthony and Travis already some time ago, that I think the CUDA version could probably return the same values too.

Is it possible for you to compile a test case for those of us who have both 48xx and 58xx cards so we can run them and see what is really doing the good work.

At Lunatics we have a bench suite that allows us to test new builds of the Astropulse apps.
____________

This WU:
3x 58xx -> valid
1x 48xx/47xx -> invalid

Am running down MW wus until problem solved.

*putting on devil's advocate hat*

Could the problem not be that in the 58x0 series, an instruction has been included in cards that actually makes them more accurate?

*removing hat*

EDIT: make the DELETING wus
____________
You should see the world from my eyes.

<removed>

A few questions.



1. The 58xx cards have been around for 6 months does this means the all the results during that time can have an error that are twice as big as they was supposed to be?



2. I am using 4870 cards to crunch an have something like 150 results that are marked as 'Invalid', caused by 58xx cards I assume. Will I get credit granted for that later?



3. If this has been going on for 6 months and you have spotted the problem just recently you obviously have serious problem with your validation method. Do you have a strategy now to prevent it from happen again?


Despite this problems I still think you guy's in Milkyway@home are the nr.1 in BOINC

But at any rate, I think with the more strict validation we have in place now, this kind of thing shouldn't happen again.

Two anonymous platforms sporting versions 0.20b and 0.22 out-quorumed an HD5870 running version 0.23.

Anybody noticing cases of 5800 series cards still running the wrong app should send a PM to the owner (if possible) since that will give them an email as well.

Anybody noticing cases of 5800 series cards still running the wrong app should send a PM to the owner (if possible) since that will give them an email as well.

I personally think it would be more appropriate if RPI were sending out these e-mails but...

Shouldn't 5xxx-series GPU results from applications prior to 0.23 be automatically discarded?

They should... but that's takes a couple extra database queries per workunit, and the server is crashing enough as it is. I had the check in there for awhile and the server couldn't keep up with it.

I have unfortunately had to swap 7 machines running various 3850, 4850, and 4870 cards on to another project as each one was producing 90% computation errors or work not validated. I'll check back in a few days and see if this is still continuing.
____________
Don't drink water, that's the stuff that rusts pipes

hi Travis

I've got question, my 3 hosts (2x5870, 5870, 4850) were disconected form MW. And after you published new app 0.23 reconnected.
Everything works fine.
But from time to time i receve some WU for app 0.19 , even when i manualy delete an app from hdd
Is it suppose to happen ?
____________

Join us at www.boincatpoland.org

yes i run Win7 and XP
but not using CPU at all (never used for MW)
hmm maybe it's "fault" of MW preferences

Use CPU
(enforced by 6.10+ clients)
____________

Join us at www.boincatpoland.org

Did your machines upgrade to the correct application (0.23) and are they running the right brook32/64.dll?

If they're giving that many errors it's probably because they're using the wrong application.

My second WU to be marked invalid: Workunit 91877832
The cuda and the HD5800 validated and left my poor Mac's CPU out in the cold.

In case it's purged:

de_s222_3s_13_360570_1270920195
application MilkyWay@Home
created 10 Apr 2010 17:23:15 UTC
canonical result 99637572
granted credit 213.76
minimum quorum 2
initial replication 3
max # of error/total/success tasks 3, 9, 6
This is displayed on the workunit page
Task ID
click for details Computer Sent Time reported
or deadline
explain Status Run time
(sec) CPU time
(sec) Claimed credit Granted credit Application
99637572 121302 10 Apr 2010 17:27:11 UTC 11 Apr 2010 21:10:34 UTC Completed and validated 813.94 24.32 0.18 213.76 MilkyWay@Home v0.24 (cuda23)
99698813 42872 11 Apr 2010 23:32:02 UTC 14 Apr 2010 11:42:03 UTC Completed, marked as invalid 27,759.43 25,764.73 195.20 0.00 MilkyWay@Home v0.26
101340563 155415 14 Apr 2010 11:43:12 UTC 14 Apr 2010 11:59:27 UTC Completed and validated 78.36 6.51 0.05 213.76 MilkyWay@Home v0.23 (ati13ati)

wann funkrioniert die scheisse hier wieder es kann nicht sein das 50% für den arsch sind

ob übertaktet oder nicht das Ergebnis ist nicht fürs Gesicht

Bitte, Bitte,

wir machen hier Wissenschaft und da kommt es vor, das manchmal Dinge nicht funktionieren. Ich möchte nicht daran zurückdenken, wie viele Experimente in meiner Diss (organische Chemie) schief gelaufen sind, und der schwarze Schleim im Kolben sah wirklich aus wie Sch...

Also bitte die Haltung bewahren :-)

Please,

we are all making science here and so it can happen that sometimes things go wrong. I don´t want to remember, how many experiments in my PhD-thesis (organic chemistry) went wrong, and the black slime in the flask looked like Sch...

So please save your countenance :-)

wenn meine karte bei allen anderen Projekten funktioniert, nur hier nicht liegt das nicht an meiner karte

Kann man so allgemein auch nicht sagen. Vielleicht vertragen andere Projekte einfach grössere Abweichungen bei den Ergebnissen. Eine geringere Fehlertoleranz ist aber kein "Fehler" des Projektes. :-P
____________
Lovely greetings, Cori

ist ja nicht so als das meine karte hier noch nie funktioniert hat. sie hat ja schon mal zu 100% funktioniert

Ein paar Leute haben geschrieben, dass sie mit der neuen ATI app ihre Übertaktung etwas reduzieren mussten.

Different programs are stressing the GPU in a different way. If you are running at the limit of the card, even a new prog version can push it over the line.
Suggestion: Set your card back to default speed for the moment. Wait until the new programm (MW3) is out and stable, than see where your new oc limit is.

Maybe it could be a good idea to send WUs for validation to different types of systems if possible (for example no more than 1 to 58X0, etc.) or no more than one to systems that have many validation problems recently (58X0 whould currently qualify for that, but in future that could of course change).

aber wenn die fehlertoleranz so gering ist das man die mouse nicht mehr bewegen kann ist es vieleicht etwas übertrieben. man hat auch mal was zu tun am pc

aber wenn die fehlertoleranz so gering ist das man die mouse nicht mehr bewegen kann ist es vieleicht etwas übertrieben. man hat auch mal was zu tun am pc

So, am I the only one running nVidia that's still having troubles?

Host reference :
http://milkyway.cs.rpi.edu/milkyway/show_host_detail.php?hostid=161353

I've been keeping up w/the news about the validation problem. I'm using the stock applications from MW@Home, and I'm running nVidia (dual GTX 295).

I don't overclock the CPU or GPU because I don't want to void the warranty.

I uploaded 24 WUs a little bit ago, only to have 9 of them go Validation inconclusive.

Any help would be appreciated.

- Safari Hood

I uploaded 24 WUs a little bit ago, only to have 9 of them go Validation inconclusive.

I posted the following message earlier, but I may have stuck it in the wrong thread:

I don't know if these are the workunits that I am having problems with but I am using the stock app with a stock Mac mini core duo (plus a few on a core 2 duo) and more than half of the units I return are marked invalid. Many of the units start with a quorum of one and then after they are returned the quorum jumps to two. I usually end up paired with a GPU and then it will validate against a third machine (also running a GPU) leaving mine as odd man out. Shouldn't the stock app on a stock CPU return (mostly) valid results?

I like this project and I realize that I am not helping very much compared with the GPU folks but I would really like to be able to return valid units. Any help or insight would be appreciated. I have tried to keep up with the postings on the forum boards here, so if I missed something vital somewhere please point me in the right direction. Thank you.

____________

Plus SETI Classic = 21,082 WUs

I uploaded 24 WUs a little bit ago, only to have 9 of them go Validation inconclusive.

For some obscure reason WUs are listed as "Completed, validation inconclusive" instead of "Completed, waiting for validation" even if there is no other deviating result that would make the validation inconclusive. Currently I see 8 of 10 of your WUs in the "validation inconclusive" state although there are no deviating results.

One WU is labeled as "Completed, waiting for validation" with one task "Completed, waiting for validation" and the other task "Completed, validation inconclusive".

The validator is still buggy in this regard.

I uploaded 24 WUs a little bit ago, only to have 9 of them go Validation inconclusive.

For some obscure reason WUs are listed as "Completed, validation inconclusive" instead of "Completed, waiting for validation" even if there is no other deviating result that would make the validation inconclusive. Currently I see 8 of 10 of your WUs in the "validation inconclusive" state although there are no deviating results.

One WU is labeled as "Completed, waiting for validation" with one task "Completed, waiting for validation" and the other task "Completed, validation inconclusive".

The validator is still buggy in this regard.

"Completed, waiting for validation" = waiting on another task
"Completed, validation inconclusive" = wu wasn't quite on target, and another was sent out and waiting for it to come back.

I uploaded 24 WUs a little bit ago, only to have 9 of them go Validation inconclusive.

OOPS it would seem that the server is out of work..

Ouch.. :)


____________