Welcome to MilkyWay@home

SLOT Directory errors on new system try2


Advanced search

Message boards : Number crunching : SLOT Directory errors on new system try2
Message board moderation

To post messages, you must log in.

AuthorMessage
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64278 - Posted: 29 Jan 2016, 1:54:04 UTC

I had posted this in Seti's Number Crunching forum as I thought the problem with this system and the previous system was with Seti; but now I am pretty sure it is with Milky Way as the massive number of higher and higher folders in the SLOTS directory are filled with a file called nbodyseparation. After reseting both projects, uninstalling BOINC, etc. then reinstalling BOINC with no Seti optimized, joining both projects but suspending Milky Way for days just running Seti, I saw no more folders in the SLOTS folder than 0-8. I resumed Milky Way also and it jumped to 0-11 in the SLOTS folder. Now a few days after that, I have 0-227 folders in the SLOTS folder. Do any of you have any suggestions as I am sure that this will simply go back to eventually hitting 800 folders and locking out both Milky Way and Seti. The post from the Seti forums is below as it has all of the details, and I am simply running Milky Way stock (though in my preferences I have N-body simulation turned off)and right now Seti is still stock too.

This is a continuation of a thread I started back 1 April 2015. For some reason the system will not let me reply or continue it in any way so I am going on. Message 1660332 - Posted: 1 Apr 2015, 14:44:43 UTC

Claggy - I apologize that I didn't get back to you in any way on the previous system. I ended up just totally uninstalling BOINC and Lunatics off the system at least 3 times, clearing out the folders, etc. and eventually it stopped happening and that system has been going strong even on the SSD.

Now, I am having the EXACT same issue with another system.

It is a new system with a Gigabyte (GA-78LMT-USB3 6.0)mother board with the latest bios, AMD FX-8350 (not overclocked), 8GB single stick of A-Data DDR3 RAM, an eVGA Nvidia GTX 750 TI 2GB (version 353.54 driver)video card, a SanDisk SSD drive for OS and most apps, a regular Toshiba 1TB HDD, and Windows 10 Pro (latest patches - new system clean install of 8.1 with immediate upgrade to 10 Pro).

It has the latest version of BOINC. It is running only Seti and Milkyway.

I originally had the lunatics installed but with the 8.0 change, I removed it.

It was fine with the stock apps for a bit but then started the same Slot Directory Errors except that it tries 800.

1/21/2016 12:23:59 PM | SETI@home | [error] Can't create task for 11oc11ab.3018.14427.10.37.50_0
1/21/2016 12:23:59 PM | SETI@home | [error] exceeded limit of 800 slot directories
1/21/2016 12:23:59 PM | SETI@home | [error] Can't create task for 15oc11ae.7220.361061.3.30.49_0
1/21/2016 12:24:00 PM | SETI@home | [error] exceeded limit of 800 slot directories

I have uninstalled BOINC and moved it from the SSD to the HDD, copying the data directory and it seemed to have no issues finding/using it.

I have RESET Seti multiple times and as per the last system with the issue, it works fine for 2 to 36hours, then starts again.

This is a totally different system so it is something weird but kind of replicatable (real word?). The only thing really in common is a AMD proc and Nvidia video card.

Here is a copy of the initial startup of BOINC on the system:

1/21/2016 12:49:58 PM | | cc_config.xml not found - using defaults
1/21/2016 12:49:58 PM | | Starting BOINC client version 7.6.22 for windows_x86_64
1/21/2016 12:49:58 PM | | log flags: file_xfer, sched_ops, task
1/21/2016 12:49:58 PM | | Libraries: libcurl/7.45.0 OpenSSL/1.0.2d zlib/1.2.8
1/21/2016 12:49:58 PM | | Data directory: E:\ProgramData\BOINC
1/21/2016 12:49:58 PM | | Running under account trever
1/21/2016 12:50:01 PM | | CUDA: NVIDIA GPU 0: GeForce GTX 750 Ti (driver version 353.54, CUDA version 7.5, compute capability 5.0, 2048MB, 1668MB available, 1606 GFLOPS peak)
1/21/2016 12:50:01 PM | | OpenCL: NVIDIA GPU 0: GeForce GTX 750 Ti (driver version 353.54, device version OpenCL 1.2 CUDA, 2048MB, 1668MB available, 1606 GFLOPS peak)
1/21/2016 12:50:01 PM | | OpenCL CPU: AMD FX(tm)-8350 Eight-Core Processor (OpenCL driver vendor: Advanced Micro Devices, Inc., driver version 2.0 (sse2,avx,fma4), device version OpenCL 1.2 AMD-APP (937.2))
1/21/2016 12:50:01 PM | | Host name: BlueJacket
1/21/2016 12:50:01 PM | | Processor: 8 AuthenticAMD AMD FX(tm)-8350 Eight-Core Processor [Family 21 Model 2 Stepping 0]
1/21/2016 12:50:01 PM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 htt pni ssse3 fma cx16 sse4_1 sse4_2 popcnt aes f16c syscall nx lm avx sse4a osvw xop wdt fma4 topx rdtscp bmi1
1/21/2016 12:50:01 PM | | OS: Microsoft Windows 10: Professional x64 Edition, (10.00.10586.00)
1/21/2016 12:50:01 PM | | Memory: 7.98 GB physical, 9.23 GB virtual
1/21/2016 12:50:01 PM | | Disk: 931.17 GB total, 929.31 GB free
1/21/2016 12:50:01 PM | | Local time is UTC -6 hours
1/21/2016 12:50:01 PM | Milkyway@Home | URL http://milkyway.cs.rpi.edu/milkyway/; Computer ID 642423; resource share 75
1/21/2016 12:50:01 PM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 7848126; resource share 100
1/21/2016 12:50:06 PM | SETI@home | General prefs: from SETI@home (last modified 01-Apr-2015 15:37:51)
1/21/2016 12:50:06 PM | SETI@home | Computer location: home
1/21/2016 12:50:06 PM | SETI@home | General prefs: no separate prefs for home; using your defaults
1/21/2016 12:50:06 PM | | Preferences:
1/21/2016 12:50:06 PM | | max memory usage when active: 4086.76MB
1/21/2016 12:50:06 PM | | max memory usage when idle: 7764.85MB
1/21/2016 12:50:06 PM | | max disk usage: 10.00GB
1/21/2016 12:50:06 PM | | don't compute while active
1/21/2016 12:50:06 PM | | don't use GPU while active
1/21/2016 12:50:06 PM | | suspend work if non-BOINC CPU load exceeds 75%
1/21/2016 12:50:06 PM | | (to change preferences, visit a project web site or select Preferences in the Manager)
1/21/2016 12:50:07 PM | | Suspending computation - computer is in use

In the SLOTS folder under Program Data/BOINC, there are folders from 0 to 801. Some have amounts of 1 to 4MB but a lot seem to have only 229 bytes. The only file in the folder is a separation_checkpoint that is 1KB in size and gives an access denied error if you try to open in say notepad. Even if I close out of BOINC and kill all processes with BOINC, I cannot delete most of those folders. Even if I uninstall BOINC and try to delete the BOINC folder in Program Data, I get an access denied error on the SLOTS folder. I have to reboot the system and then I can delete the folders.

Again, this is extremely weird and definitely seems to be an issue with more modern AMD procs/motherboards with an Nvidia card as I have an old XP system with a AMD Athlon II x2 and the same eVGA GTX750ti and it has had no issues)

Any help/suggestions you can give would be appreciated.


Thanks for any help.
I gotta fever and the only prescription is more cowbell.
ID: 64278 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64300 - Posted: 5 Feb 2016, 18:10:34 UTC

<sigh>

Well, I was right and it did it again. 0-801 slot folders in the SLOTS directory and pretty much most of them are nbody so this is a Milky Way issue not a Seti issue.

It seems that while the system is processing Milky Way tasks, it is creating the SLOT folders for the tasks but not completely removing them in any kind of short time and if you have a system which is processing a lot of tasks fast, it quickly runs out of folders ... unless there is some way to increase the maximum the system allows. I know that the numbers are different as the other system that had this same issue for a while was originally giving this error at 200 slots and this one is 800 slots. Plus weird that it is only happening on the two systems I have with fairly modern AMD procs - none of my Intel have this issue nor the older AMD Athlon II.

Very confused and wishing for any help though right now, my only solution is to remove/suspend Milky Way from this system.
I gotta fever and the only prescription is more cowbell.
ID: 64300 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profilemikey
Avatar

Send message
Joined: 8 May 09
Posts: 2920
Credit: 476,755,814
RAC: 259,906
300 million credit badge13 year member badgeextraordinary contributions badge
Message 64307 - Posted: 6 Feb 2016, 12:35:21 UTC - in response to Message 64300.  

<sigh>

Well, I was right and it did it again. 0-801 slot folders in the SLOTS directory and pretty much most of them are nbody so this is a Milky Way issue not a Seti issue.

It seems that while the system is processing Milky Way tasks, it is creating the SLOT folders for the tasks but not completely removing them in any kind of short time and if you have a system which is processing a lot of tasks fast, it quickly runs out of folders ... unless there is some way to increase the maximum the system allows. I know that the numbers are different as the other system that had this same issue for a while was originally giving this error at 200 slots and this one is 800 slots. Plus weird that it is only happening on the two systems I have with fairly modern AMD procs - none of my Intel have this issue nor the older AMD Athlon II.

Very confused and wishing for any help though right now, my only solution is to remove/suspend Milky Way from this system.


Have you tried to stop running the n-body units and run the regular cpu units instead. The only difference is that the n-body units will sometimes use all your cpu core's for one work unit, while the regular cpu units will only use one cpu core per unit.
ID: 64307 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64316 - Posted: 7 Feb 2016, 5:16:13 UTC

I had already turned off n-body a long time back (I found it interesting that it was showing n-body in the slot folders):

Run only the selected applications MilkyWay@Home: yes
MilkyWay@Home N-Body Simulation: no
Milkyway@Home Separation: yes
Milkyway@Home Separation (Modified Fit): yes

Should I turn off any of the others?
I gotta fever and the only prescription is more cowbell.
ID: 64316 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profilemikey
Avatar

Send message
Joined: 8 May 09
Posts: 2920
Credit: 476,755,814
RAC: 259,906
300 million credit badge13 year member badgeextraordinary contributions badge
Message 64325 - Posted: 8 Feb 2016, 12:27:22 UTC - in response to Message 64316.  

I had already turned off n-body a long time back (I found it interesting that it was showing n-body in the slot folders):

Run only the selected applications MilkyWay@Home: yes
MilkyWay@Home N-Body Simulation: no
Milkyway@Home Separation: yes
Milkyway@Home Separation (Modified Fit): yes

Should I turn off any of the others?


No I would run all the ones that work and don't give me errors. Make sure you have a 'NO' in this section though:

If no work for selected applications is available, accept work from other applications? yes/no
ID: 64325 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64333 - Posted: 13 Feb 2016, 22:33:07 UTC

It is interesting that it simply seems to be that Milky Way running on this system simply creates the slots folders for tasks as needed; but doesn't delete those folders when the task is done nor does it reuse some of those folders already created. If I stop Milky Way from getting more tasks, it will eventually clear out most of the folders but I will be left with a bunch of empty folders that I can actually delete.
It is 100% Milky Way because if I stop Milky Way from getting tasks or remove Milky Way from this system, eventually (after a few days) I will be left with just folders 0-8 for Seti.
I gotta fever and the only prescription is more cowbell.
ID: 64333 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64334 - Posted: 13 Feb 2016, 22:36:05 UTC - in response to Message 64325.  

Done.

Run only the selected applications MilkyWay@Home: yes
MilkyWay@Home N-Body Simulation: no
Milkyway@Home Separation: yes
Milkyway@Home Separation (Modified Fit): yes
If no work for selected applications is available, accept work from other applications? no

Let's see if that helps.
I gotta fever and the only prescription is more cowbell.
ID: 64334 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileThunder
Avatar

Send message
Joined: 9 Jul 08
Posts: 85
Credit: 44,842,651
RAC: 0
30 million credit badge14 year member badge
Message 64359 - Posted: 5 Mar 2016, 0:10:34 UTC

FINALLY a clue! I've been driving myself nuts trying to figure out why every few days BOINC was unable to start new tasks and giving me an error about (sorry it's not copied here) a limit of 500 slots.

(I have no idea why yours barfs at 800 and mine at 500)

A browse through the (100's of) sub-directories under slots reveals that nearly all of them contain a file called "separation_checkpoint".

At the moment, this machine has about 25 tasks (mostly GPU), so there's no good reason for 100's of slots being still "filled".

Here's the host that is the constant offender:

http://milkyway.cs.rpi.edu/milkyway/show_host_detail.php?hostid=683009
ID: 64359 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64406 - Posted: 23 Mar 2016, 17:53:58 UTC

Well, I do have a bit more information as I got smart and changed the Message Log options to include SLOT and STATEfile.

It is setting the STATEfile to dirty but seems to be able to do its work; but tries to clean up the SLOTS and fails with an ERROR 5.

I have no idea what ERROR 5 in SLOT file cleanup is but at least I have a start.

Also, I have let BOINC finish all tasks, uninstall BOINC completely, reboot, then reinstall and still getting this.

Log when first starting BOINC:

3/23/2016 12:46:43 PM | | Starting BOINC client version 7.6.22 for windows_x86_64
3/23/2016 12:46:43 PM | | log flags: file_xfer, sched_ops, task, slot_debug, statefile_debug, task_debug
3/23/2016 12:46:43 PM | | log flags: work_fetch_debug
3/23/2016 12:46:43 PM | | Libraries: libcurl/7.45.0 OpenSSL/1.0.2d zlib/1.2.8
3/23/2016 12:46:43 PM | | Data directory: E:\ProgramData\BOINC
3/23/2016 12:46:43 PM | | Running under account trever
3/23/2016 12:46:46 PM | | CUDA: NVIDIA GPU 0: GeForce GTX 750 Ti (driver version 353.54, CUDA version 7.5, compute capability 5.0, 2048MB, 1668MB available, 1606 GFLOPS peak)
3/23/2016 12:46:46 PM | | OpenCL: NVIDIA GPU 0: GeForce GTX 750 Ti (driver version 353.54, device version OpenCL 1.2 CUDA, 2048MB, 1668MB available, 1606 GFLOPS peak)
3/23/2016 12:46:46 PM | | OpenCL CPU: AMD FX(tm)-8350 Eight-Core Processor (OpenCL driver vendor: Advanced Micro Devices, Inc., driver version 2.0 (sse2,avx,fma4), device version OpenCL 1.2 AMD-APP (937.2))
3/23/2016 12:46:46 PM | SETI@home | Found app_info.xml; using anonymous platform
3/23/2016 12:46:46 PM | | [statefile] set dirty: Set mode
3/23/2016 12:46:46 PM | | [statefile] set dirty: Set mode
3/23/2016 12:46:46 PM | | [statefile] set dirty: Set mode
3/23/2016 12:46:46 PM | | Host name: BlueJacket
3/23/2016 12:46:46 PM | | Processor: 8 AuthenticAMD AMD FX(tm)-8350 Eight-Core Processor [Family 21 Model 2 Stepping 0]
3/23/2016 12:46:46 PM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 htt pni ssse3 fma cx16 sse4_1 sse4_2 popcnt aes f16c syscall nx lm avx sse4a osvw xop wdt fma4 topx rdtscp bmi1
3/23/2016 12:46:46 PM | | OS: Microsoft Windows 10: Professional x64 Edition, (10.00.10586.00)
3/23/2016 12:46:46 PM | | Memory: 7.98 GB physical, 9.23 GB virtual
3/23/2016 12:46:46 PM | | Disk: 931.17 GB total, 223.63 GB free
3/23/2016 12:46:46 PM | | Local time is UTC -5 hours
3/23/2016 12:46:46 PM | | [statefile] Writing state file
3/23/2016 12:46:46 PM | | [statefile] Done writing state file
3/23/2016 12:46:46 PM | Milkyway@Home | URL http://milkyway.cs.rpi.edu/milkyway/; Computer ID 642423; resource share 75
3/23/2016 12:46:46 PM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 7848126; resource share 100
3/23/2016 12:46:51 PM | SETI@home | General prefs: from SETI@home (last modified 01-Apr-2015 15:37:51)
3/23/2016 12:46:51 PM | SETI@home | Computer location: home
3/23/2016 12:46:51 PM | SETI@home | General prefs: no separate prefs for home; using your defaults
3/23/2016 12:46:51 PM | | Preferences:
3/23/2016 12:46:51 PM | | max memory usage when active: 4086.76MB
3/23/2016 12:46:51 PM | | max memory usage when idle: 7764.85MB
3/23/2016 12:46:51 PM | | max disk usage: 10.00GB
3/23/2016 12:46:51 PM | | don't compute while active
3/23/2016 12:46:51 PM | | don't use GPU while active
3/23/2016 12:46:51 PM | | suspend work if non-BOINC CPU load exceeds 75%
3/23/2016 12:46:51 PM | | (to change preferences, visit a project web site or select Preferences in the Manager)
3/23/2016 12:46:51 PM | | [work_fetch] Request work fetch: Prefs update
3/23/2016 12:46:51 PM | | [work_fetch] Request work fetch: Startup
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/0: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/0/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/1: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/1/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/10: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/10/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/100: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/100/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/101: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/101/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/102: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/102/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/103: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/103/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/104: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/104/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/105: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/105/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/106: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/106/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/107: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/107/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/108: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/108/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/109: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/109/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/11: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/11/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/110: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/110/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/111: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/111/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/112: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/112/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/113: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/113/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/114: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/114/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/115: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/115/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/116: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/116/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/117: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/117/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/118: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/118/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/119: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/119/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/12: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/12/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/120: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/120/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/121: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/121/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/122: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/122/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/123: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/123/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/124: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/124/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/125: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/125/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/126: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/126/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/127: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/127/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/128: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/128/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/129: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/129/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/13: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/13/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/130: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/130/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/131: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/131/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/132: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/132/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/133: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/133/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/134: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/134/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/135: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/135/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/136: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/136/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/137: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/137/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/138: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/138/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/139: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/139/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/14: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/14/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/140: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/140/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/141: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/141/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/142: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/142/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/143: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/143/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/144: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/144/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/145: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/145/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/146: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/146/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/147: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/147/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/148: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/148/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/149: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/149/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/15: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/15/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/150: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/150/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/151: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/151/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/152: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/152/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/153: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/153/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/154: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/154/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/155: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/155/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/156: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/156/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/157: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/157/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/158: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/158/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/159: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/159/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/16: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/16/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/160: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/160/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/161: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/161/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/162: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/162/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/163: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/163/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/164: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/164/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/165: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/165/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/166: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/166/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/167: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/167/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/168: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/168/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/169: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/169/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/17: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/17/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/170: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/170/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/171: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/171/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/172: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/172/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/173: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/173/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/174: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/174/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/175: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/175/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/176: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/176/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/177: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/177/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/178: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/178/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/179: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/179/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/18: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/18/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/180: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/180/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/181: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/181/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/182: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/182/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/183: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/183/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/184: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/184/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/185: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/185/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/186: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/186/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/187: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/187/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/188: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/188/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/189: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/189/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/19: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/19/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/190: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/190/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/191: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/191/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/192: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/192/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/193: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/193/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/194: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/194/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/195: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/195/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/196: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/196/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/197: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/197/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/198: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/198/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/199: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/199/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/2: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/2/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/20: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/20/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/200: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/200/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/201: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/201/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/202: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/202/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/203: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/203/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/204: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/204/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/205: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/205/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/206: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/206/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/207: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/207/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/208: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/208/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/209: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/209/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/21: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/21/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/210: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/210/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/211: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/211/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/212: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/212/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/213: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/213/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/214: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/214/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/215: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/215/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/216: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/216/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/217: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/217/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/218: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/218/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/219: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/219/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/22: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/22/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/220: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/220/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/221: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/221/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/222: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/222/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/223: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/223/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/224: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/224/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/225: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/225/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/226: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/226/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/227: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/227/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/228: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/228/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/229: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/229/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/23: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/23/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/230: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/230/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/231: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/231/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/232: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/232/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/233: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/233/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/234: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/234/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/235: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/235/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/236: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/236/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/237: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/237/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/238: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/238/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/239: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/239/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/24: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/24/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/240: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/240/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/241: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/241/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/242: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/242/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/243: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/243/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/244: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/244/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/245: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/245/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/246: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/246/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/247: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/247/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/248: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/248/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/249: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/249/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/25: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/25/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/250: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/250/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/251: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/251/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/252: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/252/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/253: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/253/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/254: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/254/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/255: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/255/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/256: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/256/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/257: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/257/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/258: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/258/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/259: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/259/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/26: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/26/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/260: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/260/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/261: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/261/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/262: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/262/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/263: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/263/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/264: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/264/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/265: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/265/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/266: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/266/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/267: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/267/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/268: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/268/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/269: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/269/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/27: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/27/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/270: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/270/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/271: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/271/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/272: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/272/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/273: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/273/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/274: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/274/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/275: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/275/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/276: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/276/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/278: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/278/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/279: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/279/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/28: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/28/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/280: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/280/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/281: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/281/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/282: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/282/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/283: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/283/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/284: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/284/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/285: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/285/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/286: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/286/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/287: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/287/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/288: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/288/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/289: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/289/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/29: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/29/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/290: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/290/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/291: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/291/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/292: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/292/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/293: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/293/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/294: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/294/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/295: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/295/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/296: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/296/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/297: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/297/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/298: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/298/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/299: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/299/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/3: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/3/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/30: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/30/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/300: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/300/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/302: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/302/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/304: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/304/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/307: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/307/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/31: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/31/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/310: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/310/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/311: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/311/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/314: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/314/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/32: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/32/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/33: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/33/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/34: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/34/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/35: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/35/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/36: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/36/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/37: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/37/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/38: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/38/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/39: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/39/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/4: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/4/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/40: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/40/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/41: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/41/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/42: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/42/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/43: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/43/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/44: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/44/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/45: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/45/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/46: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/46/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/47: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/47/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/48: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/48/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/49: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/49/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/5: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/5/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/50: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/50/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/51: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/51/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/52: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/52/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/53: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/53/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/54: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/54/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/55: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/55/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/56: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/56/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/57: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/57/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/58: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/58/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/59: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/59/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/6: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/6/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/60: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/60/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/61: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/61/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/62: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/62/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/63: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/63/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/64: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/64/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/65: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/65/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/66: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/66/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/67: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/67/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/69: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/69/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/7: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/7/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/70: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/70/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/71: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/71/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/72: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/72/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/73: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/73/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/74: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/74/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/75: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/75/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/76: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/76/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/77: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/77/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/78: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/78/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/79: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/79/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/8: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/8/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/80: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/80/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/81: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/81/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/82: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/82/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/83: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/83/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/84: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/84/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/85: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/85/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/86: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/86/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/87: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/87/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/88: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/88/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/89: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/89/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/9: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/9/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/90: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/90/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/91: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/91/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/92: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/92/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/93: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/93/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/94: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/94/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/95: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/95/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/96: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/96/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/97: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/97/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/98: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/98/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [slot] cleaning out slots/99: delete old slot dirs
3/23/2016 12:46:51 PM | | [slot] failed to remove file slots/99/separation_checkpoint: Error 5
3/23/2016 12:46:51 PM | | [statefile] set dirty: init
3/23/2016 12:46:51 PM | | Suspending computation - computer is in use
3/23/2016 12:46:51 PM | | [statefile] Writing state file
3/23/2016 12:46:51 PM | | [statefile] Done writing state file
3/23/2016 12:46:53 PM | | [statefile] set dirty: set_language
3/23/2016 12:46:53 PM | | [statefile] Writing state file
3/23/2016 12:46:53 PM | | [statefile] Done writing state file

Any ideas from anyone on the ERROR 5?
I gotta fever and the only prescription is more cowbell.
ID: 64406 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileThunder
Avatar

Send message
Joined: 9 Jul 08
Posts: 85
Credit: 44,842,651
RAC: 0
30 million credit badge14 year member badge
Message 64407 - Posted: 23 Mar 2016, 20:06:23 UTC

I have no idea if this relates in any way, but I was having the same issue and the solution was so simple (for me) that I gave myself a good forehead slap.

Remembering (from long ago) that anti-malware software can cause issues with BOINC, I created an exception in mine for the BOINC data directories.

I've not had the slots folders issue since.

Again... may not relate in any way to yours, but since I posted that I was having the same issue in the same thread, I just wanted to share the (embarrassingly simple) solution.
ID: 64407 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
trever

Send message
Joined: 30 Jun 10
Posts: 7
Credit: 113,059,296
RAC: 0
100 million credit badge12 year member badgeextraordinary contributions badge
Message 64414 - Posted: 24 Mar 2016, 15:23:16 UTC - in response to Message 64407.  

Thanks Thunder and sadly/happily (I've been in IT for over 20 years so I'm a bit embarrassed) it seems to be working.

After I set the SLOTS folder to be excluded by my A/V, and rebooted the system, BOINC is able to clear out those slot folders and seems to not be getting anymore.

We shall see if this is a permanent fix; but I feel it will be.

I do feel a bit stupid for not remembering to set the exception: but happy to find the solution.

THANKS AGAIN!!!
I gotta fever and the only prescription is more cowbell.
ID: 64414 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileThunder
Avatar

Send message
Joined: 9 Jul 08
Posts: 85
Credit: 44,842,651
RAC: 0
30 million credit badge14 year member badge
Message 64421 - Posted: 25 Mar 2016, 13:47:08 UTC - in response to Message 64414.  

That's great news, so I'm happy my suggestion was of value. :-)

I have no idea why MW@H is the only project that has issues, but I suspect it relates to the fact that it creates many, many more tasks (with short times) than others.

For what it's worth, I started in IT in roughly '88, so this was one of those situations where experience didn't make a bit of difference. I remind myself that thankfully even a blind dog finds a bone once in a while. ;-)
ID: 64421 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profilemikey
Avatar

Send message
Joined: 8 May 09
Posts: 2920
Credit: 476,755,814
RAC: 259,906
300 million credit badge13 year member badgeextraordinary contributions badge
Message 64425 - Posted: 26 Mar 2016, 10:15:34 UTC - in response to Message 64421.  

That's great news, so I'm happy my suggestion was of value. :-)

I have no idea why MW@H is the only project that has issues, but I suspect it relates to the fact that it creates many, many more tasks (with short times) than others.

For what it's worth, I started in IT in roughly '88, so this was one of those situations where experience didn't make a bit of difference. I remind myself that thankfully even a blind dog finds a bone once in a while. ;-)


Several Boinc projects have workunits that a/v programs think, due to the way they do things, are problematic so alot of us have just excluded the whole set of Boinc directories from our a/v scanning altogether. In effect that means that any real virus, as well as all false positives which most Boinc stuff is, are ignored UNTIL they try to move out into the rest of our pc's, then they get caught and confined or deleted depending on how our a/v program handles them. As long as any real virus stays confined to the Boinc directories it is each projects problem not mine or yours, and most have some very good a/v stuff that should preclude that anyway.
ID: 64425 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : SLOT Directory errors on new system try2

©2022 Astroinformatics Group