Unless I am being silly I am confused as to why the Nbody unsent are not dropping very quickly. I have completed over 200 today that were validated. There are 4300 users on currently so assuming they are similar to me that could be 860,000 completed. The unsents have dropped by around 20,000 in the last 24 hours , not what I would expect. My validation inconclusive backlog has not changed at all, in fact it has grown very slightly. At the present rate I estimate it will take 550 days to clear the backlog.. Hope I am wrong.

Unless I am being silly I am confused as to why the Nbody unsent are not dropping very quickly. I have completed over 200 today that were validated. There are 4300 users on currently so assuming they are similar to me that could be 860,000 completed. The unsents have dropped by around 20,000 in the last 24 hours , not what I would expect. My validation inconclusive backlog has not changed at all, in fact it has grown very slightly. At the present rate I estimate it will take 550 days to clear the backlog..

I've been tracking this as well and can confirm that observation. For what it's worth, over 90% of the tasks I've seen recently are second tries which get validated more or less instantly, so the work units should be getting thinned out a bit!

I can think of three possibilities for the count staying high, not mutually exclusive:

1. It might still be counting "Didn't need" tasks as unsent (in which case they won't disappear until the relevant work units are validated and assimilated;
   
2. I've noticed a fairly large number of the tasks I've got in the Inconclusive category are retries for tasks recently aborted - the way N-Body seems to work at present, if someone aborts a retry the transitioner will produce another one!
   
3. The work unit generator might have decided to add some new stuff again (despite claims that that shouldn't happen if there are lot of tasks already waiting to go out!)
   

You'll notice there's no "the unsent tasks query has a bug in it" option there -- I've looked at the Server status PHP code and the query is really simplistic (and is almost certainly counting "Didn't need" items in the total!)

If N-Body used adaptive replication (as Separation does) this wouldn't be a problem -- "reliable" hosts would validate without a wingman most of the time! However, if N-Body is supposed to be using adaptive replication it isn't working (configuration issues?)

I had thought of asking Tom about that in the "N-Body flush" thread, but I reckon it would get lost amongst the noise there :-) Perhaps he'll see this and let us know whether wingman-free validation should be working for N-Body or not -- it might be the only way of clearing the backlog without having to delete entire work units!

Cheers - Al.

[Edited to note my validate-to-inconclusive ratio and to rephrase the bit about the D/B query and "Didn't need" tasks]

Thanks alanb 1951, very reassuring.

I can think of three possibilities for the count staying high, not mutually exclusive:

1. It might still be counting "Didn't need" tasks as unsent (in which case they won't disappear until the relevant work units are validated and assimilated;
   
2. I've noticed a fairly large number of the tasks I've got in the Inconclusive category are retries for tasks recently aborted - the way N-Body seems to work at present, if someone aborts a retry the transitioner will produce another one!
   
3. The work unit generator might have decided to add some new stuff again (despite claims that that shouldn't happen if there are lot of tasks already waiting to go out!)
   

You'll notice there's no "the unsent tasks query has a bug in it" option there -- I've looked at the Server status PHP code and the query is really simplistic (and is almost certainly counting "Didn't need" items in the total!)

Correction -- option 1 shouldn't apply; I had another look at the PHP, and provided the "Didn't need" tasks are flagged properly they shouldn't be counted. So I'm at [more of] a loss to explain the increasing count if the generator isn't adding new work units! (And I sincerely hope it isn't...)

Cheers - Al.

Observation: it's a bit difficult trying to investigate/"debug" something {in order to explain it] from the opposite side of the Atlantic, especially without detailed access to data. :-)

I can think of three possibilities for the count staying high, not mutually exclusive:

1. It might still be counting "Didn't need" tasks as unsent (in which case they won't disappear until the relevant work units are validated and assimilated;
   
2. I've noticed a fairly large number of the tasks I've got in the Inconclusive category are retries for tasks recently aborted - the way N-Body seems to work at present, if someone aborts a retry the transitioner will produce another one!
   
3. The work unit generator might have decided to add some new stuff again (despite claims that that shouldn't happen if there are lot of tasks already waiting to go out!)
   

You'll notice there's no "the unsent tasks query has a bug in it" option there -- I've looked at the Server status PHP code and the query is really simplistic (and is almost certainly counting "Didn't need" items in the total!)

Correction -- option 1 shouldn't apply; I had another look at the PHP, and provided the "Didn't need" tasks are flagged properly they shouldn't be counted. So I'm at [more of] a loss to explain the increasing count if the generator isn't adding new work units! (And I sincerely hope it isn't...)

Cheers - Al.

Observation: it's a bit difficult trying to investigate/"debug" something {in order to explain it] from the opposite side of the Atlantic, especially without detailed access to data. :-)

And Tom the Admin is out of town as well so the Server could be doing things 'on it's own' again, 'on it's own' being what's is supposed to do but not exactly what Tom thinks it's supposed to be doing. MW certainly needs to get Tom some more help in the form of some IT grad student or someone like that that can help decipher the complexities of Boinc.

Did a spot check from yesterday against today and NBody unsents have risen by almost 120,000. Still no movement up or down on my backlog of inconclusives .

I can tell you that my Validation Inconclusive has gone down by over 400 in the last 2 days and I still have another 14,306 that are still listed as Validation Inconclusive so it's going to take awhile to get my Validation Inconclusive down to 0.I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

---

I can tell you that my Validation Inconclusive has gone down by over 400 in the last 2 days and I still have another 14,306 that are still listed as Validation Inconclusive so it's going to take awhile to get my Validation Inconclusive down to 0.I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

+1

One of the reasons validation seems slow moving (and the queue) is because of tasks like this: https://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=391645869. There are probably thousands of them maybe even a few million (from the 14 million that were created after disk reconstruction). Eventually things will get cleared up but it will take time.

I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

The damage has already been done. I stopped crunching late March after seeing my inconclusives flatline without improvement. I'm sure I'm not alone. Moved to universe@home and now check only irregularly here just to see what WU's remain.

One of the reasons validation seems slow moving (and the queue) is because of tasks like this: https://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=391645869. There are probably thousands of them maybe even a few million (from the 14 million that were created after disk reconstruction). Eventually things will get cleared up but it will take time.

Yes alot of people have no clue about how to reduce the cache size when they reduce the amount of time their pc can run full speed, so end up with sooooo many units that can't finish in time and yet still let them just expire instead of aborting them.

I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

The damage has already been done. I stopped crunching late March after seeing my inconclusives flatline without improvement. I'm sure I'm not alone. Moved to universe@home and now check only irregularly here just to see what WU's remain.

I'm sure your Team is liking your contribution to the Boinc Pentathlon assuming they signed up for it.

MW certainly needs to get Tom some more help in the form of some IT grad student or someone like that that can help decipher the complexities of Boinc.

Amen. And Amen.

We will provide an update on the result of our efforts to resolve the issue Sunday evening.

I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

Sunday evening passed by, no news. But the didn't say which Sunday ...

We will provide an update on the result of our efforts to resolve the issue Sunday evening.

I predict if World Community Grid comes back on May 9th as it's scheduled that some of the people will go back to World Community Grid which will make the number of NBody work units to be validated take even longer

Sunday evening passed by, no news. But the didn't say which Sunday ...

Guessing that means they ran into more problems

Think the validator has stopped, waiting for validation has shotup to over 35,000

Think the validator has stopped, waiting for validation has shotup to over 35,000

The validator is still working -- I've just checked my N-Body tasks flagged valid, and there are plenty returned since your post... However, there is an explanation for why the "Waiting for validation" is rising...

A lot of the N-Body work units that were stalled had no successful result returned when they were put on hold; As N=Body doesn't seem to be using adaptive replication, when someone returns the first successful unit it'll end up "Inconclusive" until a wingman returns a result, and as the request for a second result ends up at the back of the [over-large] queue those work units are going to push up the "Waiting for validation" count for quite a while :-(

For what it's worth, at the moment about 10% of the tasks my systems process each day are flagged inconclusive because of this. I have no idea how that compares with what other users are seeing, but if it's typical then (given the rate at which the unsent tasks count is being reduced) the "Waiting for validation" count could keep increasing by 20,000 or more per day until we get round to much more recent work units.

Cheers - Al.

Thank you.

Think the validator has stopped, waiting for validation has shotup to over 35,000

The validator is still working -- I've just checked my N-Body tasks flagged valid, and there are plenty returned since your post... However, there is an explanation for why the "Waiting for validation" is rising...

A lot of the N-Body work units that were stalled had no successful result returned when they were put on hold; As N=Body doesn't seem to be using adaptive replication, when someone returns the first successful unit it'll end up "Inconclusive" until a wingman returns a result, and as the request for a second result ends up at the back of the [over-large] queue those work units are going to push up the "Waiting for validation" count for quite a while :-(

For what it's worth, at the moment about 10% of the tasks my systems process each day are flagged inconclusive because of this. I have no idea how that compares with what other users are seeing, but if it's typical then (given the rate at which the unsent tasks count is being reduced) the "Waiting for validation" count could keep increasing by 20,000 or more per day until we get round to much more recent work units.

Cheers - Al.

Oops = sleep-deprived misread of source code, and too late to edit the original... Whilst the above was right regarding how long it might take for inconclusive status to be resolved, it wasn't right about the validation count!!!

I note that the waiting for validation count has dropped back again -- I wonder if it was actually a rush of Separation tasks all being returned at once...

Sorry about that - Al.

Think the validator has stopped, waiting for validation has shotup to over 35,000

The validator is still working -- I've just checked my N-Body tasks flagged valid, and there are plenty returned since your post... However, there is an explanation for why the "Waiting for validation" is rising...

A lot of the N-Body work units that were stalled had no successful result returned when they were put on hold; As N=Body doesn't seem to be using adaptive replication, when someone returns the first successful unit it'll end up "Inconclusive" until a wingman returns a result, and as the request for a second result ends up at the back of the [over-large] queue those work units are going to push up the "Waiting for validation" count for quite a while :-(

For what it's worth, at the moment about 10% of the tasks my systems process each day are flagged inconclusive because of this. I have no idea how that compares with what other users are seeing, but if it's typical then (given the rate at which the unsent tasks count is being reduced) the "Waiting for validation" count could keep increasing by 20,000 or more per day until we get round to much more recent work units.

Cheers - Al.

I have taken my machines off of MilkyWay for right now but the last tasks I finished up yesterday were all _2 tasks, no _0 or _1 tasks at all, I wonder if that means we are getting to a batch of validator tasks or if Tom found a way to send out the validator tasks sooner than the end of the database?