Welcome to MilkyWay@home

Server Downtime March 28, 2022 (12 hours starting 00:00 UTC)

Message boards : News : Server Downtime March 28, 2022 (12 hours starting 00:00 UTC)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 15 · Next

AuthorMessage
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 72282 - Posted: 28 Mar 2022, 18:21:51 UTC
Last modified: 28 Mar 2022, 18:22:20 UTC

Not sure if all of this was getting sent out as notices or not. That has happened in the past. Apologies if it was, I've turned that off now.
ID: 72282 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bull Schuck

Send message
Joined: 20 Feb 22
Posts: 12
Credit: 16,852,000
RAC: 34
Message 72283 - Posted: 28 Mar 2022, 18:36:55 UTC - in response to Message 72280.  

I'm not getting the "feeder down" message any more but I'm not getting any WU either. Just "Scheduler request completed: got 0 new tasks"
I suppose it will pick up in a bit.

I was able to finally upload some completed WU's though. So that's a plus.

Thanks again!
ID: 72283 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 72284 - Posted: 28 Mar 2022, 18:37:53 UTC

Glad to hear it, that means that something is going right, at least. I'll keep monitoring things just in case.
ID: 72284 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robert Coplin

Send message
Joined: 23 Sep 13
Posts: 19
Credit: 36,223,867
RAC: 0
Message 72285 - Posted: 28 Mar 2022, 18:40:09 UTC - in response to Message 72283.  

I actually did get some work units on all 4 of my computers
ID: 72285 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72286 - Posted: 28 Mar 2022, 19:50:36 UTC - in response to Message 72280.  

@ Tom:
Okay, I've restarted things. Is the feeder still down for you?

Sorry for the delay.
Things are working now.
Many thanks - good work!
ID: 72286 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bill F
Avatar

Send message
Joined: 4 Jul 09
Posts: 98
Credit: 17,393,606
RAC: 1,420
Message 72288 - Posted: 28 Mar 2022, 20:19:06 UTC
Last modified: 28 Mar 2022, 20:19:30 UTC

Another area of confusion for who was looking where. Here is the current Time Stamp from the bottom of the Server Status page... showing yesterdays date and status.

Another process to restart

Upstream server release: 1.0.4
Database schema version: 27028
Task data as of 27 Mar 2022, 18:54:47 UTC

Thanks
Bill F
Dallas TX
In October of 1969 I took an oath to support and defend the Constitution of the United States against all enemies, foreign and domestic;
There was no expiration date.


ID: 72288 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
unixchick
Avatar

Send message
Joined: 21 Feb 22
Posts: 66
Credit: 817,008
RAC: 0
Message 72290 - Posted: 28 Mar 2022, 23:24:37 UTC

anyone getting any resends? I'm only getting _0 WUs. I'm not complaining, I'm so happy to be getting WUs again.

Thank you Tom!
ID: 72290 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
unixchick
Avatar

Send message
Joined: 21 Feb 22
Posts: 66
Credit: 817,008
RAC: 0
Message 72291 - Posted: 29 Mar 2022, 3:48:03 UTC

And I spoke too soon. I'm not getting any WUs now. doesn't seem to be an error, just that there are no WUs to send.
ID: 72291 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72292 - Posted: 29 Mar 2022, 5:06:10 UTC - in response to Message 72288.  

@ Bill:

Another process to restart

Upstream server release: 1.0.4
Database schema version: 27028
Task data as of 27 Mar 2022, 18:54:47 UTC

Thanks
Bill F
Dallas TX


Seems to be fixed!
It is now 29 Mar 2022, 2:23:27 UTC

AND Backlog is 0.00
ID: 72292 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72293 - Posted: 29 Mar 2022, 7:06:19 UTC - in response to Message 72284.  

@ Tom:

Glad to hear it, that means that something is going right, at least. I'll keep monitoring things just in case.

... back to the workbench ...

Good work Tom, but
I'm not able to get new work.
Well, not quite, two of eight rigs got work, but the six others have not - no matter what I do.
There is enough work to be sent.

The task page takes very long to react, but when it does, it shows that tasks numbers are changing/moving.

It would be nice to have the "Project status" updated every hour. Was that not the way it used to be?
ID: 72293 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
macgeyer

Send message
Joined: 2 Mar 18
Posts: 9
Credit: 457,043,383
RAC: 0
Message 72294 - Posted: 29 Mar 2022, 7:24:30 UTC

@ Tom,

Same here :

29/03/2022 09:21:30 | Milkyway@Home | Sending scheduler request: To fetch work.
29/03/2022 09:21:30 | Milkyway@Home | Requesting new tasks for AMD/ATI GPU
29/03/2022 09:21:32 | Milkyway@Home | Scheduler request completed: got 0 new tasks
29/03/2022 09:21:32 | Milkyway@Home | Project requested delay of 91 seconds
ID: 72294 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Max_Pirx

Send message
Joined: 13 Dec 17
Posts: 46
Credit: 2,421,362,376
RAC: 0
Message 72295 - Posted: 29 Mar 2022, 8:22:51 UTC

Yes, unfortunately, something is still not quite right with the server. I can confirm that none of my hosts are getting work (as per the two previous posts). Yesterday, after a lot of manual requests I got a handful of WUs (far less than the max amount per host) and that was it. No work since then.
ID: 72295 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robert Coplin

Send message
Joined: 23 Sep 13
Posts: 19
Credit: 36,223,867
RAC: 0
Message 72296 - Posted: 29 Mar 2022, 9:49:55 UTC - in response to Message 72295.  

All 4 of my computes are getting new work units on a regular basis so it seems to be fine with me.I wish the work units were validated a little faster then they are now but at least we are making progress on getting the work units to be validated
ID: 72296 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Septimus

Send message
Joined: 8 Nov 11
Posts: 205
Credit: 2,900,464
RAC: 0
Message 72297 - Posted: 29 Mar 2022, 10:06:01 UTC - in response to Message 72296.  

Doesnt want to send me any work. Says no tasks available for Separation when there is nearly 5 Million. Machine has been waiting for nearly an hour.
ID: 72297 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72298 - Posted: 29 Mar 2022, 11:20:55 UTC
Last modified: 29 Mar 2022, 11:50:28 UTC

Still not getting work:

3/29/2022 1:12:07 PM | Milkyway@Home | Sending scheduler request: To fetch work.
3/29/2022 1:12:07 PM | Milkyway@Home | Requesting new tasks for NVIDIA GPU
3/29/2022 1:12:09 PM | Milkyway@Home | Scheduler request completed: got 0 new tasks
3/29/2022 1:12:09 PM | Milkyway@Home | Project requested delay of 91 seconds
3/29/2022 1:16:57 PM | Milkyway@Home | work fetch suspended by user
3/29/2022 1:17:06 PM | Milkyway@Home | work fetch resumed by user
3/29/2022 1:18:01 PM | Milkyway@Home | update requested by user
3/29/2022 1:18:05 PM | Milkyway@Home | Sending scheduler request: Requested by user.
3/29/2022 1:18:05 PM | Milkyway@Home | Requesting new tasks for NVIDIA GPU
3/29/2022 1:18:07 PM | Milkyway@Home | Scheduler request completed: got 0 new tasks
3/29/2022 1:18:07 PM | Milkyway@Home | Project requested delay of 91 seconds

Very strange ...

EDIT:
Now I am getting
3/29/2022 1:44:59 PM | Milkyway@Home | work fetch resumed by user
3/29/2022 1:45:00 PM | Milkyway@Home | Sending scheduler request: To fetch work.
3/29/2022 1:45:00 PM | Milkyway@Home | Requesting new tasks for NVIDIA GPU
3/29/2022 1:45:01 PM | Milkyway@Home | Scheduler request completed: got 0 new tasks
3/29/2022 1:45:01 PM | Milkyway@Home | No work available
3/29/2022 1:45:01 PM | Milkyway@Home | Project requested delay of 91 seconds
NO WORK AVAILABLE ??
ID: 72298 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Septimus

Send message
Joined: 8 Nov 11
Posts: 205
Credit: 2,900,464
RAC: 0
Message 72299 - Posted: 29 Mar 2022, 11:54:34 UTC - in response to Message 72297.  
Last modified: 29 Mar 2022, 12:23:54 UTC

Doesnt want to send me any work. Says no tasks available for Separation when there is nearly 5 Million. Machine has been waiting for nearly an hour.



Still nothing through.

I am finding the same as San Fernando Valley did all those things still nothing.
ID: 72299 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72300 - Posted: 29 Mar 2022, 12:14:33 UTC

It says 600 in progress:
State: All (82650) · In progress (600) · Validation pending (28154) · Validation inconclusive (8569) · Valid (45223) · Invalid (0) · Error (104)
Application: All (82650) · Milkyway@home N-Body Simulation (0) · Milkyway@home Separation (82650)
I wish I could find out where they are.
None of my rigs show any tasks.

Stopped BOINC on all rigs (aka PC).
Restarted my rigs.
Started BOINC on all rigs.
NOTHING!!

After requesting work the Event Log either says that there is no work available or just no work gotten.

Hope Tom will get to his office early.

Something is wrong.
ID: 72300 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Max_Pirx

Send message
Joined: 13 Dec 17
Posts: 46
Credit: 2,421,362,376
RAC: 0
Message 72303 - Posted: 29 Mar 2022, 13:29:37 UTC

Same situation here, my stats say 622 tasks in progress, although none of my machines has received a single WU in the last 16hrs. The WU request either gets 0 tasks or a message saying no work is available.
ID: 72303 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
San-Fernando-Valley

Send message
Joined: 13 Apr 17
Posts: 256
Credit: 604,411,638
RAC: 0
Message 72304 - Posted: 29 Mar 2022, 14:20:32 UTC - in response to Message 72300.  

@ Tom:

My stats say 600 tasks in progress:
State: All (82650) · In progress (600) · Validation pending (28154) · Validation inconclusive (8569) · Valid (45223) · Invalid (0) · Error (104)
Application: All (82650) · Milkyway@home N-Body Simulation (0) · Milkyway@home Separation (82650)
I wish I could find out where they are.
None of my rigs show any tasks.
...

It looks like those 600 tasks in progress, but nowhere to be seen in BOINC Manager, are tasks solely with a minimum quorum of one.
ID: 72304 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom Donlon
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 10 Apr 19
Posts: 408
Credit: 120,203,200
RAC: 0
Message 72305 - Posted: 29 Mar 2022, 14:32:23 UTC

Hmm. That's really strange. I hope that things didn't get "desynced" with the DB when the drive failed. I'm not sure exactly what that would mean in terms of tech, but if the server thinks that there are 1.6M WUs that are ready to be sent out, but they are actually stuck somewhere, then it probably wouldn't make more WUs that could actually get sent out.

Maybe I can clear out the WUs that are waiting in the queue somehow. I suppose this could also happen with the WUs in the validation queue, which could explain the huge amount of WUs waiting for validation that apparently aren't getting sent out either.

I'll have to dig around. If I end up purging everything I'll have to figure out a way to give out the proper credits to everyone who has WUs that have been crunched and are sitting in limbo.

Today is busy for me, I have to train students on our observatory telescope, have several meetings, and am about to teach class. But I'll try to look into things when I get a chance.
ID: 72305 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 15 · Next

Message boards : News : Server Downtime March 28, 2022 (12 hours starting 00:00 UTC)

©2024 Astroinformatics Group