New Separation Runs 1/6/2021

Hello Everyone,

I have put up the following separation runs:

de_modfit_80_bundle4_4s_south4s_bgset_4
de_modfit_81_bundle4_4s_south4s_bgset_4
de_modfit_82_bundle4_4s_south4s_bgset_4
de_modfit_83_bundle4_4s_south4s_bgset_4
de_modfit_84_bundle4_4s_south4s_bgset_4
de_modfit_85_bundle4_4s_south4s_bgset_4
de_modfit_86_bundle4_4s_south4s_bgset_4

And I have taken down the following separation runs:

de_modfit_80_bundle4_4s_south4s_bgset_3
de_modfit_81_bundle4_4s_south4s_bgset_3
de_modfit_82_bundle4_4s_south4s_bgset_3
de_modfit_83_bundle4_4s_south4s_bgset_3
de_modfit_84_bundle4_4s_south4s_bgset_3
de_modfit_86_bundle4_4s_south4s_bgset_3

The new runs are copies of the previous bgset_3 runs and bgset_2 runs, in order to get yet another set of data to compare against. In the last set of runs, Stripe 84 didn't seem to cause problems like it did in the bgset_2 runs, but Stripe 85 did. As before, I will keep a close eye on these forums for a few weeks in order to see whether you are having any trouble with some of the Stripes.

You may continue to see workunits from bgset_2 runs for the next few days as workunits finish up and complete validation. This is expected and normal. You may start seeing the new runs as soon as your machine downloads new workunits from the server.

Thank you all for your continued support with this project. I hope you are staying safe in the pandemic world that we're stuck in for a little while longer.

Best,
Tom

Two wrong de_modfit_85. Very few, but still there.

Thanks for the update, I'm going to keep that stripe running for now, but I'll keep an eye on it.

I'm catching a few validate errors on de_modfit_85_bundle4_4s_south4s_bgset_4. Not sure if it need be reported but wanted to say something as it's the only validate errors I'm getting.

Thanks for the report. I'm going to try to keep these runs up until it starts becoming a problem, since it doesn't look like they've converged yet. If you start getting a large percentage of validate errors (>10% or so) please let me know and I can stop the bad stripes.

Hello, I'm now approaching that 10% of invalid tasks on de_modfit_85_bundle4_4s_south4s_bgset_4. I am at circa 8000 valid / 800 invalid.

I am up to 7% now from 5% yesterday.

---

Over 10% invalid and counting.

Right at 10% invalid now and climbing with the gpus running nothing but MW now since both Einstein and GPUGrid can't produce work.

---

I have taken down the de_modfit_85_bundle4_4s_south4s_bgset_4 run. It was not converged, but that's okay, and it would have only become more and more of a problem. Please let me know if Stripe 84 (de_modfit_84_bundle4_4s_south4s_bgset_4) workunits start causing any problems.

It is normal to see de_modfit_85_bundle4_4s_south4s_bgset_4 workunits trickle in over the next couple days as validations finish up. In a few days you should not be getting any more Stripe 85 workunits.

Best,
Tom