I'm getting a few "Maximum disk usage exceeded" errors. I don't think I've ever seen this here.

For my tasks, the client_state file shows:

<rsc_disk_bound>15000000.000000</rsc_disk_bound>

Example task result shows Peak disk usage 5,741.01 MB.

---

These things keep coming. Updating to the latest driver doesn't seem to have helped and others running the same tasks don't seem to have any problems, either. I'm thinking I have a hardware problem... :-(

---

Additional info, in case it matters...

I have local preferences set that are pretty wide-open, I think. The host has an 1TB HDD with only about 250GB used. Local disk limits are set to:

Use at most -- 150GB (most restrictive)
Leave at least -- 0.1 GB (least restrictive)
Use at most -- 50% of total (less restrictive)

The BOINC manager says that 26 GB is used for BOINC with 124GB available and that MS@H is using less than 240MB.

---

Additional info, in case it matters...

I have local preferences set that are pretty wide-open, I think. The host has an 1TB HDD with only about 250GB used. Local disk limits are set to:

Use at most -- 150GB (most restrictive)
Leave at least -- 0.1 GB (least restrictive)
Use at most -- 50% of total (less restrictive)

The BOINC manager says that 26 GB is used for BOINC with 124GB available and that MS@H is using less than 240MB.

It probably won't help but change the top line "use at most" to 500gb, if it works back it off to just above where you get the errors again. If it doesn't work, as I suspect it won't but who knows, then you can switch it back to where it is now.

This is not a client configuration problem, it is a workunit problem.

The project can define a max. disk usage per workunit, each workunit that fails to stay below this limit will be aborted by the BOINC core client.

The only thing you could do against it would be to patch your core client so it ignores this limit - but then a workunit running wild would not be recognized.

p.s.: usually all workunits of the same batch have the same disk limitation. Even though it is possible to calculate an individual limit for each single workunit, I doubt that too many projects do that.

OOPS, sorry ... I thought your problem was related to the rsc_disk_bound value (as mentioned in the starting post) but this doesn't seem to be the case.

Several years ago I made this thing, maybe it helps. Note that BOINC will probably still not get along well with NTFS compression, i.e. BOINC uses the physical disk space for the calculation, ignoring the compression potential.

I'm with Ananas' original thought - I think it's to do with <rsc_disk_bound>. There's a parallel thread at Einstein - Maximum disk usage exceeded - where ritterm cross-posted, and some related discussion in Results showing "Aborted by user".

...I think it's to do with <rsc_disk_bound>...

That's what I was thinking, too, of course. However, I now think I might have a GPU hardware problem -- all the tasks I've checked that errored out for me have been completed by another host without a problem. If the tasks I ran had bad parameters, would the same task work for another host?

When I upgraded the video driver, I went to the AMD website, downloaded and ran their auto-detect tool, and let it pick and install a new driver. Is there anything else I need to install?

---

...I think it's to do with <rsc_disk_bound>...

That's what I was thinking, too, of course. However, I now think I might have a GPU hardware problem -- all the tasks I've checked that errored out for me have been completed by another host without a problem. If the tasks I ran had bad parameters, would the same task work for another host?

When I upgraded the video driver, I went to the AMD website, downloaded and ran their auto-detect tool, and let it pick and install a new driver. Is there anything else I need to install?

A bad driver by itself wouldn't cause a disk limit error - unless it's spewing out yards and yards of error messages. Look in the slot directory, as I said at Einstein.

A bad driver by itself wouldn't cause a disk limit error - unless it's spewing out yards and yards of error messages. Look in the slot directory...

I'm afraid I'm not sure I know what to look for. These tasks are failing right away, after only 1-2 seconds of run time. I don't see anything changing in the slot directory when this happens (\ProgramData\BOINC\slots, right?).

---

A bad driver by itself wouldn't cause a disk limit error - unless it's spewing out yards and yards of error messages. Look in the slot directory...

I'm afraid I'm not sure I know what to look for. These tasks are failing right away, after only 1-2 seconds of run time. I don't see anything changing in the slot directory when this happens (\ProgramData\BOINC\slots, right?).

Well, each task gets allocated to one particular numbered folder in there as it starts - which one is visible via the 'properties' button while it's active, but a couple of seconds doesn't give you much time to investigate.

Each slot should be empty, unless there's a running task using it. Might be worth (re-)starting BOINC with GPU activity disabled, and emptying any slots which should be empty but aren't. Then, the next task you allow to run should occupy the lowest-numbered empty slot - watch that, and see if anything (big) appears in it.

I suspended all tasks then resumed them one at a time and waited for one to crash. It didn't take too long, but all that's left in the directory is the stderr output file and it's only 4KB. If something "big" was generated before being deleted, I wasn't able to see anything.

The only message in the BOINC manager log related to the task is something similar to this:

Aborting task de_80_DR8_Rev_8_5_00004_1429700402_4384432_0: exceeded disk limit: 5115.01MB > 14.31MB

I just don't understand what's going on. Only about 5% of the tasks are failing for me and others don't seem to be having any problem with them.

---

I suspended all tasks then resumed them one at a time and waited for one to crash. It didn't take too long, but all that's left in the directory is the stderr output file and it's only 4KB.

THAT'S the file you want to look at, post it here if you can please.

THAT'S the file you want to look at, post it here if you can please.

Well, other than the initial remarks about the "Maximum disk usage exceeded" error and what appears to be the lack of results data at the end, the rest of the file looks virtually identical to the stderr output of a valid task.

However, I think my problem is solved. Following the suggestion of a forum post about a similar problem at another project, I checked my host's slots directories and found two "stray" VM image files left by one of the VM projects (probably CERN's CMS-dev), each of which was over 5GB. I deleted those files and slots and have been running trouble free for almost 12 hours. I'm not sure, though, that I understand why that was the problem.

---

THAT'S the file you want to look at, post it here if you can please.

Well, other than the initial remarks about the "Maximum disk usage exceeded" error and what appears to be the lack of results data at the end, the rest of the file looks virtually identical to the stderr output of a valid task.

However, I think my problem is solved. Following the suggestion of a forum post about a similar problem at another project, I checked my host's slots directories and found two "stray" VM image files left by one of the VM projects (probably CERN's CMS-dev), each of which was over 5GB. I deleted those files and slots and have been running trouble free for almost 12 hours. I'm not sure, though, that I understand why that was the problem.

I'm glad you got it fixed, that is strange. Could they be trying to use the same file name?

I'm glad you got it fixed, that is strange. Could they be trying to use the same file name?

Me, too! :-) I'm really not sure what's going on with the other project, but you can read more in this thread over at CERN/CMS-dev.

---

I reported this problem to the BOINC developers, and got this reply from David Anderson:

I looked at this and couldn't immediately see the problem.
The BOINC client deletes everything in a slot directory before using it for a new job.
If a deletion fails (e.g. because a file is in use by another app) it doesn't use
that slot directory.
I verified this by opening some Word docs in slot directories.

Notes:

* There's a "slot_debug" log flag for messages related to slot directories.
Unfortunately it doesn't print messages about failed file deletions; I'll add this.
* The "disk limit exceeded" errors refer to the per-job disk limit, not the user's
disk usage preferences; I'll change the message to clarify this.
* Apps aren't responsible for cleaning out their slot dirs; BOINC does this. It
may be that BOINC is failing to delete VM images because they're still in use by
the VirtualBox executive.

Bottom line: I'll need some more info to debug this.
If anyone is seeing this reproducibly, let me know.
Otherwise we'll release a client with more debugging output to help us investigate.

-- David

So, help needed.

Under what circumstances does the CMS .vdi image get left behind? Is there a difference between successful task completions and abnormal (error) exits?

Can the .vdi be deleted manually? Immediately? Later? After BOINC restart? After reboot?

Does BOINC ever clean it up by itself, say after a client restart?

And anything else you can think of.

Could somebody pass David's message over to CERN/CMS-dev, please? I don't even have an invitation code to create a posting account.

Could somebody pass David's message over to CERN/CMS-dev, please? I don't even have an invitation code to create a posting account.

Done.

---