Message boards :
Number crunching :
WUs not downloaded in time - rig is idling - doing no work ...
Message board moderation
Previous · 1 · 2
Author | Message |
---|---|
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
Anybody running MW exclusively on the BOINC 7.16.1 client and see very different pull requests? There are significant changes to work_fetch.cpp that eliminate a lot of the previous issues. ![]() |
![]() ![]() Send message Joined: 18 Nov 08 Posts: 291 Credit: 2,461,693,501 RAC: 0 ![]() ![]() |
Anybody running MW exclusively on the BOINC 7.16.1 client and see very different pull requests? There are significant changes to work_fetch.cpp that eliminate a lot of the previous issues. I cannot find a window 7.16.1 client. Poked around over at GitHub but didn't find an executable. I can no longer run 18.04 with my AMD s9000 boards (a long and sad story) and do not have the expertise to do the cross compile of the 16.1 source to windows. You know where to find the windows 16.1? |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
Anybody running MW exclusively on the BOINC 7.16.1 client and see very different pull requests? There are significant changes to work_fetch.cpp that eliminate a lot of the previous issues. Github contains the source code. It does not house any executables. You download the source code and compile it yourself. The client 7.16.1 branch can be found with the client tag at github. Just click on the branch arrow and scroll down to the client_release/7.16.1 https://github.com/BOINC/boinc/tree/client_release/7/7.16 The Windows build artifacts are over at AppVeyor. https://ci.appveyor.com/api/buildjobs/4bvvgoug1ej0x5mh/artifacts/deploy%2Fwin-client%2Fwin-client_master_2019-09-18_15ffc98a.7z ![]() |
![]() Send message Joined: 23 Feb 18 Posts: 26 Credit: 4,744,416,145 RAC: 0 ![]() ![]() |
Yes, this was reported in May. From the last result there needs to be a 10min period of no requests until the clients can get more work. That's not true.. if you report immediatly the last results and wait the first deferred communication interval of 1minute and 30seconds the new workunits will arrive.. Automatic: 102726 Milkyway@Home 19/09/2019 15:37:18 Sending scheduler request: To fetch work. 102727 Milkyway@Home 19/09/2019 15:37:18 Reporting 5 completed tasks 102728 Milkyway@Home 19/09/2019 15:37:18 Requesting new tasks for AMD/ATI GPU 102729 Milkyway@Home 19/09/2019 15:37:25 Scheduler request completed: got 0 new tasks 102730 Milkyway@Home 19/09/2019 15:48:00 Sending scheduler request: To fetch work. 102731 Milkyway@Home 19/09/2019 15:48:00 Requesting new tasks for AMD/ATI GPU 102732 Milkyway@Home 19/09/2019 15:48:22 Scheduler request completed: got 300 new tasks with manual intervention: 104225 Milkyway@Home 19/09/2019 17:53:10 update requested by user 104226 Milkyway@Home 19/09/2019 17:53:13 Sending scheduler request: Requested by user. 104227 Milkyway@Home 19/09/2019 17:53:13 Reporting 1 completed tasks 104228 Milkyway@Home 19/09/2019 17:53:13 Requesting new tasks for AMD/ATI GPU 104229 Milkyway@Home 19/09/2019 17:53:15 Scheduler request completed: got 0 new tasks 104230 Milkyway@Home 19/09/2019 17:53:15 Not sending work - last request too recent: 17 sec 104231 Milkyway@Home 19/09/2019 17:54:50 Sending scheduler request: To fetch work. 104232 Milkyway@Home 19/09/2019 17:54:50 Requesting new tasks for AMD/ATI GPU 104233 Milkyway@Home 19/09/2019 17:54:56 Scheduler request completed: got 300 new tasks This idle time can be manually minimized.. but this isn't the solution.. reported results must be replaced with new workunits.. not wait until the queue is empty and then wait for timeouts and then download a full stack.. Want your Kids stay off from Drugs? Get them building Crunching PC's and they'll never have enough money for drugs |
![]() ![]() Send message Joined: 18 Nov 08 Posts: 291 Credit: 2,461,693,501 RAC: 0 ![]() ![]() |
Just got around to downloading. It is 7.15.0 does that have the new feature that 16.1 has? Going to let it run for a while and see what happens. thanks! |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
It is made from the current master which is 7.15.0 and has the same work_fetch.cpp module that is in the client_release/7.16.1. So in that regard they are same. Mainly the client has the commit for the bugfix I requested #3076 https://github.com/BOINC/boinc/commit/0b5bae4cc98660538b76842dea8b5cf4a16d06f6 There are other changes in 7.16.1 compared to the master 7.15.0 but in areas I don't think has an impact on the inability of the client to request work for work turned in at MW. That is why I suggested running one of the later artifacts that has the latest code in work_fetch.cpp. All I ask is someone run it and see if anything changes for work scheduling. [Edit]Just wanted to point out that the changes to work_fetch.cpp didn't just cover the issue with using a max_concurrent statement. DA also implemented many lines of new code specifically calling rr_simulation routines. It's those routines that help determine the correct shortfall in work requested and the part that I think will have the greatest effect on work requested at MW. ![]() |
![]() ![]() Send message Joined: 18 Nov 08 Posts: 291 Credit: 2,461,693,501 RAC: 0 ![]() ![]() |
Did not affect the problem where I run out of data. I was hoping that after each upload there would be a few downloads, but no, my BT task had to issue an update as shown: between lines 2186 and 2190 2183 Milkyway@Home 9/19/2019 2:27:45 PM Computation for task de_modfit_86_bundle4_4s_south4s_bgset_2_1564052102_18863666_0 finished 2184 Milkyway@Home 9/19/2019 2:28:16 PM Sending scheduler request: To fetch work. 2185 Milkyway@Home 9/19/2019 2:28:16 PM Reporting 16 completed tasks 2186 Milkyway@Home 9/19/2019 2:28:16 PM Requesting new tasks for AMD/ATI GPU 2187 Milkyway@Home 9/19/2019 2:28:19 PM Scheduler request completed: got 0 new tasks 2188 Milkyway@Home 9/19/2019 2:30:59 PM update requested by user 2189 Milkyway@Home 9/19/2019 2:31:00 PM Sending scheduler request: Requested by user. 2190 Milkyway@Home 9/19/2019 2:31:00 PM Requesting new tasks for AMD/ATI GPU 2191 Milkyway@Home 9/19/2019 2:31:04 PM Scheduler request completed: got 900 new tasks 2192 Milkyway@Home 9/19/2019 2:31:07 PM Starting task de_modfit_80_bundle4_4s_south4s_bgset_2_1564052102_18598672_1 However, I could disable my script and see how long it takes before the client asks again on its own. I will try that. |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
Sorry to hear that didn't accomplish anything. I thought give it a shot, what the heck. I assume that if you ran the sched_ops_debug flag in logging options and it shows the client is asking for 0 seconds of gpu work? Have you used the work_fetch_debug flag to see what the shortfalls are for each component and project? I am beginning to think that the consensus opinion here that the server is misconfigured and doesn't allow requests for work at the same time work is reported is correct. ![]() |
![]() ![]() Send message Joined: 18 Nov 08 Posts: 291 Credit: 2,461,693,501 RAC: 0 ![]() ![]() |
Sorry to hear that didn't accomplish anything You spoke too soon (maybe) Just looked at results and at the very least it seems to be only in the order of 12 to 15 minutes, not a huge amount though some may consider that a lifetime ;<) 4360 Milkyway@Home 9/19/2019 5:00:25 PM Computation for task de_modfit_14_bundle5_testing_4s3f_2_1564052102_18633077_1 finished 4361 Milkyway@Home 9/19/2019 5:01:07 PM Sending scheduler request: To fetch work. 4362 Milkyway@Home 9/19/2019 5:01:07 PM Reporting 15 completed tasks 4363 Milkyway@Home 9/19/2019 5:01:07 PM Requesting new tasks for AMD/ATI GPU 4364 Milkyway@Home 9/19/2019 5:01:10 PM Scheduler request completed: got 0 new tasks 4365 Milkyway@Home 9/19/2019 5:12:50 PM Sending scheduler request: To fetch work. 4366 Milkyway@Home 9/19/2019 5:12:50 PM Requesting new tasks for AMD/ATI GPU 4367 Milkyway@Home 9/19/2019 5:12:53 PM Scheduler request completed: got 900 new tasks Note there is no user request for update so the scheduler took about 12 minutes and then issued a new request for data. At least it as not an hour or more. Looks like this on my data analysis graph. The 6 minute delay was from my BT update script which shaved off about 9 minutes of idle. ![]() |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
OK, that is encouraging. Still wondering where that 12 minute timeout is coming from. I doubt it is a hard coded delay in the client. Could it be that the reason was the one espoused by the project scientist that stated you could just be hitting the RTS buffer when it is empty? ![]() |
![]() Send message Joined: 23 Feb 18 Posts: 26 Credit: 4,744,416,145 RAC: 0 ![]() ![]() |
when you ron out of WU the first deferred communication is always more or less 1.40minutes.. in this first stage it uploads the latest results (and don't downloads anything) then begin the second deferred communication of more or less 12minutes (can't remember).. the gpu is idling and then since no results to report, the downloads of 300wu begin this loop is always the same on all hosts.. next time i'll run out of WU i will post the exact times.. Want your Kids stay off from Drugs? Get them building Crunching PC's and they'll never have enough money for drugs |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
Yes, it is hard for me to diagnose these issues as I have never seen the behavior described. But I don't solely run MW on my hosts. They always have around 900 tasks in their cache and just keep topping off to reach 900. The only time I ever ran out of tasks was when the project was offline and I crunched through the entire cache. But as soon as the project came back, the first scheduler connection started refilling back to 900. But I run a spoofed client compiled from the latest source so I don't know if that is what insulates me. I also don't remember this issue when I ran the bone stock 7.14.2 client either however. ![]() |
![]() ![]() Send message Joined: 18 Nov 08 Posts: 291 Credit: 2,461,693,501 RAC: 0 ![]() ![]() |
Yes, it is hard for me to diagnose these issues as I have never seen the behavior described Perhaps your Linux version is different enough from windows that the timing and scheduling functions are causing problem only in windows. I tried using the 7.16.1 Linux app with my AMD s9x00 boards but gave up after about 7 attempts using different AMD drivers and kernels. I had it running for a short time (an hour) on an old 18.04.1 with a 4.x kernel but an upgrade ruined things and I never got kernel 5.00x to work and was unable to go to go back to the original 18.04.1 as my usb install was only "live". I had failed to install the full package when downloading to the usb flash earlier in the year. Also, your FP64 2080 specs are still only 1/3 of my S9100 specs so possibly you are not in the timing area where the problem occurs. Just a guess. |
Send message Joined: 2 Oct 16 Posts: 167 Credit: 1,006,950,718 RAC: 686 ![]() ![]() |
when you ron out of WU the first deferred communication is always more or less 1.40minutes.. in this first stage it uploads the latest results (and don't downloads anything) There is nothing to actually upload. Set the client to no networking and the tasks go straight to Waiting to Report and not Uploading. There are no data files to download or upload for this project per task. |
![]() ![]() Send message Joined: 13 Dec 12 Posts: 101 Credit: 1,782,758,310 RAC: 8 ![]() ![]() ![]() |
The problem after 5 months is still present. Any update? Agree with Mikey. Other projects with quick work units don't have this issue. What makes M@H so unique? It is frustrating a little to have great crunching gear but it sits idle. One reason I like Einstein@Home. It just works. Never any hiccups. ![]() ![]() |
![]() ![]() Send message Joined: 13 Dec 12 Posts: 101 Credit: 1,782,758,310 RAC: 8 ![]() ![]() ![]() |
Wow gambatesa! I just noticed you have 1.8 Billion credits and only a 1 year badge! Then I read your signature about keeping kids off drugs and it all made sense! LOL Dedication. ![]() ![]() |
![]() ![]() Send message Joined: 8 May 09 Posts: 3339 Credit: 524,010,781 RAC: 14 ![]() ![]() ![]() |
Yes, it is hard for me to diagnose these issues as I have never seen the behavior described No I have the same problem in both my Linux and Windows pc's!! |
![]() ![]() Send message Joined: 24 Jan 11 Posts: 712 Credit: 551,844,720 RAC: 44,624 ![]() ![]() ![]() ![]() |
The client is up to 7.16.2 now and has about 60 more commits from the master added to it. A lot of polishing. The close with the red X issue seems to be fixed now. From what I hear the only thing hanging up this as the new master release is the translations are still waiting to come in. ![]() |
©2024 Astroinformatics Group