Welcome to MilkyWay@home

Server fails to download GPU tasks so long as there are tasks "ready to report"


Advanced search

Message boards : Number crunching : Server fails to download GPU tasks so long as there are tasks "ready to report"
Message board moderation

To post messages, you must log in.

AuthorMessage
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69366 - Posted: 19 Dec 2019, 15:23:27 UTC

Old problem... "No New Tasks" returned whenever any tasks are "Ready to report"
It seems that this is something that could be resolved rather easily by setting as flag allowing tasks to be "Fetched" until the MAX_TASK_LIMIT is reached??
Running 3 Titan Vs on a single rig, single client, and I would not know how to or look to spoof the gpu count. Each Titan V runs 6 tasks each with 0.5 CPU per completing tasks in 52-55sec, The problem is that the MW server will not send new GPU tasks to my client so long as any completed tasks are ready to report. Once GPU tasks download and start, there is never a 91sec period where there are no completed tasks. This is not an issue for the CPU tasks since it will not accumulate completed tasks every 91 sec.
ID: 69366 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69375 - Posted: 20 Dec 2019, 2:37:38 UTC

Fixed.
ID: 69375 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69377 - Posted: 20 Dec 2019, 18:19:34 UTC

Ugh! The fetch for GPU tasks returned back to the bugged behavior of not sending any new GPU tasks when there are any GPU tasks "ready to report". While, CPU tasks will continue to accumulate.
This is definitely a BUG with Milkyway GPU tasks.
ID: 69377 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileJoseph Stateson
Avatar

Send message
Joined: 18 Nov 08
Posts: 227
Credit: 1,230,792,150
RAC: 392,649
1 billion credit badge10 year member badge
Message 69382 - Posted: 20 Dec 2019, 23:02:55 UTC - in response to Message 69377.  
Last modified: 20 Dec 2019, 23:17:30 UTC

Unless I misread what you sent me, you have queued up and are still running CPU tasks from MW. I suspect they count against you in the total download allowed.

[EDIT] My program does not differentiate between Milkyway CPU tasks and GPU tasks when calculating time to wait. You need to stop running any of the CPU tasks from Milkyway. If that fixes the problem then I will look at how to improve my program although I don't have a 36 core cpu so cant do a real test

They need to be aborted, not just suspended and have the project send only nvidia stuff
ID: 69382 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69392 - Posted: 21 Dec 2019, 23:55:16 UTC

No worries, no cpu tasks are running or being fetched. Preferences set to not allow them on rigs called "home." I have separate preferences for other machines which do run cpu tasks.
Still holding "proper" gpu tasks on the 2 rigs running the mod. :thumb
ID: 69392 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69393 - Posted: 21 Dec 2019, 23:56:12 UTC

Issue is fixed. This thread should be closed.
ID: 69393 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profilemikey
Avatar

Send message
Joined: 8 May 09
Posts: 2319
Credit: 398,516,859
RAC: 196,727
300 million credit badge10 year member badgeextraordinary contributions badge
Message 69394 - Posted: 22 Dec 2019, 0:44:12 UTC - in response to Message 69393.  

Issue is fixed. This thread should be closed.


You said that yesterday too and it turns out it wasn't...why not give it a few days this time?
ID: 69394 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69395 - Posted: 22 Dec 2019, 2:51:32 UTC

it's been running as intended for a day now. Zero GPU idle time on a rig with 3 Titan Vs. I had initially not disabled CPU tasks as described in the other thread. Works great! Give Jstateson's solution a try.
ID: 69395 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profilemikey
Avatar

Send message
Joined: 8 May 09
Posts: 2319
Credit: 398,516,859
RAC: 196,727
300 million credit badge10 year member badgeextraordinary contributions badge
Message 69396 - Posted: 22 Dec 2019, 11:35:13 UTC - in response to Message 69395.  

it's been running as intended for a day now. Zero GPU idle time on a rig with 3 Titan Vs. I had initially not disabled CPU tasks as described in the other thread. Works great! Give Jstateson's solution a try.


I don't run cpu tasks at Projects where my gpu's can crunch so that's never been a problem for me, I have the mw back-off stuff in my files in the right places and STILL get the 'not sending tasks' BS so run Einstein as a zero resource share Project until MilkyWay refills my cache.
ID: 69396 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jpmboy

Send message
Joined: 29 Apr 17
Posts: 33
Credit: 2,461,970,320
RAC: 7,670,885
2 billion credit badge2 year member badge
Message 69397 - Posted: 23 Dec 2019, 3:36:33 UTC - in response to Message 69396.  
Last modified: 23 Dec 2019, 3:37:11 UTC

I don't run cpu tasks at Projects where my gpu's can crunch so that's never been a problem for me, I have the mw back-off stuff in my files in the right places and STILL get the 'not sending tasks' BS so run Einstein as a zero resource share Project until MilkyWay refills my cache.

just to followup. I also have the mod running on this rig. A single Radeon VII. No GPU idle time. Sorry for the large image. I did scale it to 80% original.
ID: 69397 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Server fails to download GPU tasks so long as there are tasks "ready to report"

©2020 Astroinformatics Group