Welcome to MilkyWay@home

Waiting for GPU memory

Message boards : Number crunching : Waiting for GPU memory
Message board moderation

To post messages, you must log in.

AuthorMessage
tmath250

Send message
Joined: 15 Jan 12
Posts: 11
Credit: 89,558,167
RAC: 0
Message 53247 - Posted: 17 Feb 2012, 19:37:49 UTC

I hope this is not a duplicate thread (I did search FIRST:)). I have about 28 (and increasing) work units in various stages of completion. They say "Waiting to run (waiting for GPU memory). I have the 6xduo-core processor and (2) HD7900 gpus. any thoughts?
ID: 53247 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 8 May 09
Posts: 3315
Credit: 519,950,216
RAC: 22,077
Message 53260 - Posted: 18 Feb 2012, 12:36:43 UTC - in response to Message 53247.  

I hope this is not a duplicate thread (I did search FIRST:)). I have about 28 (and increasing) work units in various stages of completion. They say "Waiting to run (waiting for GPU memory). I have the 6xduo-core processor and (2) HD7900 gpus. any thoughts?


Go into Boinc Manager, down by the clock, and click on Tools, computing preferences and then the processor usage tab, I am using version 6.?.? of Boinc so bear with me and figure out what your version says that means the same thing. Anyway on that page you will see a line that says "while processor usage is less than [] percent (0 means no restriction)". If there is a number in that box change it to a zero and see if that helps, the default is 25. Another thing to check is under the disk and memory usage tab. Make sure you have "leave applications in memory while suspended". If this IS already checked leave it checked, you have 32gb of memory and that should be plenty.
ID: 53260 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
floyd

Send message
Joined: 13 Sep 11
Posts: 17
Credit: 3,263,835
RAC: 0
Message 53271 - Posted: 18 Feb 2012, 17:38:40 UTC - in response to Message 53247.  

Waiting to run (waiting for GPU memory)

I haven't experienced this myself but I've read you need to do a system reboot to clear the GPU memory.
ID: 53271 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tmath250

Send message
Joined: 15 Jan 12
Posts: 11
Credit: 89,558,167
RAC: 0
Message 53280 - Posted: 18 Feb 2012, 21:55:20 UTC

Thank you! I will try that and report back.
ID: 53280 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Werkstatt

Send message
Joined: 19 Feb 08
Posts: 350
Credit: 141,284,369
RAC: 0
Message 53284 - Posted: 18 Feb 2012, 23:39:14 UTC

A search in the BOINC forum brought this:

3) Diskussionsforen : Questions and problems : (Waiting on GPU) Returns!...Repeatable on demand! (Message 42273)
Posted 22 days ago by Charlie Fenton Thanks for your offers of help. We don't need any more information regarding the "waiting for gpu memory" issue; we are well aware of that. It means just what it says: there is not enough free memory in your gpu to run the task. If you want to get more information, set the coproc_debug flag in your cc_config.xml file as explained here.

Unfortunately, due to a bug in some drivers, BOINC currently cannot check available gpu RAM periodically, so the check is made only when you launch BOINC. That means that if more gpu RAM becomes available after BOINC starts, BOINC does not know about it. We are working on this problem.

My concerns are the several other issues you have raised, such as the md5 errors, spontaneous restarts, etc.
ID: 53284 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jimmy Gondek

Send message
Joined: 28 Sep 11
Posts: 60
Credit: 22,764,173
RAC: 0
Message 53384 - Posted: 23 Feb 2012, 3:17:10 UTC - in response to Message 53271.  

Waiting to run (waiting for GPU memory)

I haven't experienced this myself but I've read you need to do a system reboot to clear the GPU memory.


There is a reason for a reboot (though it is not required, as explained below) to clear the problem. The errant WUs giving the Waiting for GPU error are stuck in memory, merely Disconnecting Client... and quitting/restarting BOINC Manager will not release them from memory! One of two things can be done at this point...either restart the machine, or manually quit the errant processes.

To manually quit those processes (thus purging them from memory) use Task Manager in Windows or Activity Monitor in OSX. The errant WU processes will be obvious in those programs as they will be using zero (0) CPU!

If one then proceeds with a BOINC Manager restart, those WUs will return to the head of the queue and process normally...if one chooses to leave BOINC Manager running while doing the process memory purge, the Wus will run when they next get their turn in the queue.

Hope that helps! :)
ID: 53384 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jimmy Gondek

Send message
Joined: 28 Sep 11
Posts: 60
Credit: 22,764,173
RAC: 0
Message 53385 - Posted: 23 Feb 2012, 3:32:19 UTC - in response to Message 53284.  

A search in the BOINC forum brought this:

3) Diskussionsforen : Questions and problems : (Waiting on GPU) Returns!...Repeatable on demand! (Message 42273)
Posted 22 days ago by Charlie Fenton Thanks for your offers of help. We don't need any more information regarding the "waiting for gpu memory" issue; we are well aware of that. It means just what it says: there is not enough free memory in your gpu to run the task. If you want to get more information, set the coproc_debug flag in your cc_config.xml file as explained here.

Unfortunately, due to a bug in some drivers, BOINC currently cannot check available gpu RAM periodically, so the check is made only when you launch BOINC. That means that if more gpu RAM becomes available after BOINC starts, BOINC does not know about it. We are working on this problem.

My concerns are the several other issues you have raised, such as the md5 errors, spontaneous restarts, etc.


That was my thread...

(Waiting on GPU) Returns!...Repeatable on demand!:
http://boinc.berkeley.edu/dev/forum_thread.php?id=7212

...I was trying to troubleshoot this with Charlie Fenton but was informed that the folks at BOINC were no longer devoting time and resources to pre-version 7 issues.

The best I can figure is that some hiccup occurs when BOINC Manager toggles between running single-CPU tasks and multi-CPU tasks in the queue while using MWAH. (There is a repeatable Waiting for GPU issue when trying to run WUs from another project while MWAH is running as explained in that thread.)

The hiccup sometimes causes the single-CPU MWAH tasks (that got put on hold to run the multi-CPU MWAH tasks) to get paused in memory with something akin to a sort of erroneous GPU flag being set...when those tasks are called upon to run again after the multi-CPU tasks have finished they process for about 30-seconds and then trip with the Waiting for GPU error.

Those errant WU tasks will never, never, never clear from that error and process normally without the user intervention I describe above.

Hopefully this anomaly has been addressed and solved in v.7 of BOINC Manager, though I still have my suspicions that there is either a problem with how some MWAH WUs talk to BOINC Manager, or that there is a MWAH WU scheduling issue with BOINC Manager.

Again, I hope this is helpful! :)
ID: 53385 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Waiting for GPU memory

©2024 Astroinformatics Group