I have work units starting with 3732, 606 and 607 here and everything seems to be fine.

Thank you for the work you've put in to get everything running smoothly again.

... that when the purge starts, it's so expensive that it slows down the rest of the system and can't catch up, while pretty much makes everything slow and unresponsive. so to get the purge done we have to stop the assimilator and work generation.

Can't you automatically delete any WU that's been done for 24 hours or more? So rather than doing a full purge every 24 hours you'd be purging WUs continuously at roughly the same rate that they're coming in..

Perhaps keep a log of completed WUs (sorted by reported completion time by definition), and purge old (>24h) entries whenever you add a new one (i.e. a WU is reported)

if(thisWU was completed more than 24 hours ago) delete and examine next item;
else exit;

To put it in pseudo-code <.< (had a brain-fart, so that was the best I could do)

i'll try and give this a shot and see if the server gets overloaded again. a day of WUs (at ~6 WUs a second) is quite a lot... which is part of the problem

---

... that when the purge starts, it's so expensive that it slows down the rest of the system and can't catch up, while pretty much makes everything slow and unresponsive. so to get the purge done we have to stop the assimilator and work generation.

Can't you automatically delete any WU that's been done for 24 hours or more? So rather than doing a full purge every 24 hours you'd be purging WUs continuously at roughly the same rate that they're coming in..

Perhaps keep a log of completed WUs (sorted by reported completion time by definition), and purge old (>24h) entries whenever you add a new one (i.e. a WU is reported)

if(thisWU was completed more than 24 hours ago) delete and examine next item;
else exit;

To put it in pseudo-code <.< (had a brain-fart, so that was the best I could do)

i'll try and give this a shot and see if the server gets overloaded again. a day of WUs (at ~6 WUs a second) is quite a lot... which is part of the problem

Are you sure you want to do that just yet Travis? It's working really good right now. I'd sleep on it for the night, and celebrate Pint Night at the Ruck. :D :D :D :D

i'll try and give this a shot and see if the server gets overloaded again. a day of WUs (at ~6 WUs a second) is quite a lot... which is part of the problem

As long as you make a variable out of the, er.. completed WU retention time, you should just be able to start with something relatively short like an hour, then increase it if you think it can take more. Like Stefan Ver3 said, take it slow! We're all enjoying the current flow of WUs :)

i'll try and give this a shot and see if the server gets overloaded again. a day of WUs (at ~6 WUs a second) is quite a lot... which is part of the problem

As long as you make a variable out of the, er.. completed WU retention time, you should just be able to start with something relatively short like an hour, then increase it if you think it can take more. Like Stefan Ver3 said, take it slow! We're all enjoying the current flow of WUs :)

I agree. Do we need 24 hours? A few hours would be nice for those who want to check. But what's the point of being able to check if there are no WUs to check? ;)

i'll try and give this a shot and see if the server gets overloaded again. a day of WUs (at ~6 WUs a second) is quite a lot... which is part of the problem

As long as you make a variable out of the, er.. completed WU retention time, you should just be able to start with something relatively short like an hour, then increase it if you think it can take more. Like Stefan Ver3 said, take it slow! We're all enjoying the current flow of WUs :)

I agree. Do we need 24 hours? A few hours would be nice for those who want to check. But what's the point of being able to check if there are no WUs to check? ;)

right now the purge function has it built only for days. if 1 day starts to make the server run slowly, i'll recode it for 3 or 4. right now things seem to be working fine so i'm hoping everything will be ok and WUs will be retained for a day.

---