Probably a while more, I am getting tasks generated on the 16th of March :D

Doesn't that mean the server has caught up, but we haven't?

Server has caught up with work that was sent back. But tasks generated on the 16th of march haven't been sent out yet.

Server has done the heavy lifting of validating work that we've completed, now we can compute knowing stuff is running normally

Server has caught up with work that was sent back. But tasks generated on the 16th of march haven't been sent out yet.

Server has done the heavy lifting of validating work that we've completed, now we can compute knowing stuff is running normally

Not sure that means anything. Just that work was generated a while ago. The telescope data it's from is probably 6 months ago.

project: http://milkyway.cs.rpi.edu/milkyway/
gpu_limit: 30
report_delay: 750

The report_delay gets around the MW server misconfiguration where you are unable to request work if reporting work in the same scheduler connection.
The delay is greater than the MW 600 second timeout period after depleting a cache that normal clients incur.
The scheduler connection is the default 91 seconds. So it asks for new work every 91 seconds to top off my cache.
I always have my 30 task count maintained.

I can't find "report_delay" in the Boinc configuration files. Where do I put it? This is why I currently let it get the maximum 300 per GPU because there will be a 10 minute gap at the end.

That option is only for those that uses a custom made BOINC, also it's only for Linux.

project: http://milkyway.cs.rpi.edu/milkyway/
gpu_limit: 30
report_delay: 750

The report_delay gets around the MW server misconfiguration where you are unable to request work if reporting work in the same scheduler connection.
The delay is greater than the MW 600 second timeout period after depleting a cache that normal clients incur.
The scheduler connection is the default 91 seconds. So it asks for new work every 91 seconds to top off my cache.
I always have my 30 task count maintained.

I can't find "report_delay" in the Boinc configuration files. Where do I put it? This is why I currently let it get the maximum 300 per GPU because there will be a 10 minute gap at the end.

That option is only for those that uses a custom made BOINC, also it's only for Linux.

Can I have a big description of this? I'm going to ask in Github for them to put it into normal Boinc, as it would be very useful.

That option is only for those that uses a custom made BOINC, also it's only for Linux.

Can I have a big description of this? I'm going to ask in Github for them to put it into normal Boinc, as it would be very useful.

Well I don't know that much, but maybe Keith Myers can explain things, try send him a PM.

project: http://milkyway.cs.rpi.edu/milkyway/
gpu_limit: 30
report_delay: 750

The report_delay gets around the MW server misconfiguration where you are unable to request work if reporting work in the same scheduler connection.
The delay is greater than the MW 600 second timeout period after depleting a cache that normal clients incur.
The scheduler connection is the default 91 seconds. So it asks for new work every 91 seconds to top off my cache.
I always have my 30 task count maintained.

I can't find "report_delay" in the Boinc configuration files. Where do I put it? This is why I currently let it get the maximum 300 per GPU because there will be a 10 minute gap at the end.

It's not in the standard BOINC client. I use our optimized GPUUG team client which has the setting especially for Milkyway.
I asked our dev to put it in just for me since I am the only team member that does MW. [Edit] Or initially was. A few newer team members also do MW now.
Our client does a lot more to overcome the failures of the standard BOINC client. Setting specific task count sizes is the one everyone uses on all projects.
The next most used is the request_min_cooldown setting to choose our own scheduler interval connection intervals.
It's pretty ridiculous to ping a cpu only project server every 11 seconds for a scheduler connection as the project default when the tasks take 1-12 hours to complete. With most of my projects I add 2-3 minutes to the stock project interval.
And back in the old Seti project days when it was difficult to get enough work to keep our GPUUG team "special sauce" application busy we can also spoof however many cpus or gpus we wanted to build up large cache sizes.

There is another MW optimized client by a different developer over at Github that achieves the same thing.
https://github.com/JStateson/MilkywayNewWork

---

I get a date for the WU and another date for the task. I should have been more specific. I'm looking at task date.

For Example... I get this info when looking at the workunit : created 15 Mar 2022, 13:16:34 UTC
but when I look at the specific task that I have I get : Created 24 Mar 2022, 20:57:13 UTC

I'm guessing that a workunit is created, but isn't used to generate a task for the queue until the queue size falls below some threshold . So I'm looking at the task creation (date,time) as I think it is more interesting to see where in the queue we are.

A couple of days ago we were as much as 10+ days behind in the queue, and now we are 7-8 days behind. I'm not sure what is "normal" , but it gives me another data point to watch as the system heals.

I get a date for the WU and another date for the task. I should have been more specific. I'm looking at task date.

For Example... I get this info when looking at the workunit : created 15 Mar 2022, 13:16:34 UTC
but when I look at the specific task that I have I get : Created 24 Mar 2022, 20:57:13 UTC

I'm guessing that a workunit is created, but isn't used to generate a task for the queue until the queue size falls below some threshold . So I'm looking at the task creation (date,time) as I think it is more interesting to see where in the queue we are.

A couple of days ago we were as much as 10+ days behind in the queue, and now we are 7-8 days behind. I'm not sure what is "normal" , but it gives me another data point to watch as the system heals.

I got this task about... 4 hours ago https://milkyway.cs.rpi.edu/milkyway/result.php?resultid=174582283
Created 16 Mar 2022, 3:10:33 UTC
And the associated workunit
created 15 Mar 2022, 3:29:37 UTC

The difference between workunit creation and task creation is due to the transitioner being backed up.

Workunits are generated by the work generator typically and is independent of queue size. Tasks are generated by the transitioner and is independent of queue size, it'll keep generating as long as there are workunits without tasks associated.

You can see a flaw...

Thanks Kiska for the info.

I'm not doing any nbody tasks so it looks like the nbody tasks have a short window between generation and being sent out even though the queue is huge (17 million +).

I just got some new separation tasks and it looks like I'm working on tasks created 24 Mar 2022, 21:04:17 UTC . I'm betting there are a ton of resends for March 30, 31 later in the queue as the system finally worked through the backlog of validations.

Is there really no limit on queue size? I guess not if the nbody queue was over 18 million.

Another day without credits……

Over 17 million WUs "Ready to Send" yet I still can't get any new tasks.
What gives ?

02/04/2022 11:17:54 | Milkyway@Home | work fetch resumed by user
02/04/2022 11:17:57 | Milkyway@Home | Sending scheduler request: To fetch work.
02/04/2022 11:17:57 | Milkyway@Home | Requesting new tasks for NVIDIA GPU
02/04/2022 11:18:00 | Milkyway@Home | Scheduler request completed: got 0 new tasks
02/04/2022 11:18:00 | Milkyway@Home | Project requested delay of 91 seconds

Over 17 million WUs "Ready to Send" yet I still can't get any new tasks.
What gives ?

02/04/2022 11:17:54 | Milkyway@Home | work fetch resumed by user
02/04/2022 11:17:57 | Milkyway@Home | Sending scheduler request: To fetch work.
02/04/2022 11:17:57 | Milkyway@Home | Requesting new tasks for NVIDIA GPU
02/04/2022 11:18:00 | Milkyway@Home | Scheduler request completed: got 0 new tasks
02/04/2022 11:18:00 | Milkyway@Home | Project requested delay of 91 seconds

Let Tom have the weekend off to not focus on the project. Its running fairly well at this time and I just got 100 tasks for my CPU so it may be the case that the buffer hasn't filled completely before everyone has drained it

I have had a machine ready for an hour , even tried manually, nothing. Doing other work now.

Let Tom have the weekend off to not focus on the project. Its running fairly well at this time and I just got 100 tasks for my CPU so it may be the case that the buffer hasn't filled completely before everyone has drained it

Exactly. Thank you Tom for all your hard work in making the situation stable, have a great weekend and get some rest :)

Been 2 to 3 hours since I've gotten a WU, just to confirm what others are seeing. I've got the machine doing other projects, so no stress, hopefully the server will start giving WUs again on its own at some point.

I was working on some resends from March 26 (separation queue) last I had some WUs to work on.

Another day without credits……

But think of the big christmas present you'll soon get!

Another day without credits……

But think of the big christmas present you'll soon get!

SOON?
I'd hate to have to wait till December ....

Another day without credits……

But think of the big christmas present you'll soon get!

SOON?
I'd hate to have to wait till December ....

Does it really matter?

Another day without credits……

But think of the big christmas present you'll soon get!

SOON?
I'd hate to have to wait till December ....

Does it really matter?

+1

I get the feeling that the server thinks there are 17M jobs ready to send out, so it doesn't make more jobs. However, I cancelled all of those jobs in order to try to clear the validation backlog. I'm not sure where the jobs are stuck, but I will turn things off and reset their transition times, and see if that clears them.

The jobs should get removed from the DB once they are cancelled, but they just might not have been transitioned yet because they haven't gone out to volunteers (because they were cancelled)