Clearing results from database quickly?

Since the change to the longer units I have been noticing a few things that are a little strange. Firstly, in my results I am seeing a lot of tasks supposedly not returned by deadline, but I have not seen any evidence of this actually happening on my machines. Indeed, one machine supposedly had 17 results not returned by deadline, but it had been set to no new work for several weeks and hasn't had anything to return! The others have all been going through work steadily and returning promptly, yet all are shown as having work not returned by deadline. I cannot work it out.

Secondly, are you removing finished results from the database very quickly now? I just completed a few tasks, and within 5 minutes went to see the credit that they received but could find no trace of ANY completed tasks in my results.

How long are you planning to leave the results in the database?

---

Proud member of BOINC@AUSTRALIA

Since the change to the longer units I have been noticing a few things that are a little strange. Firstly, in my results I am seeing a lot of tasks supposedly not returned by deadline, but I have not seen any evidence of this actually happening on my machines. Indeed, one machine supposedly had 17 results not returned by deadline, but it had been set to no new work for several weeks and hasn't had anything to return! The others have all been going through work steadily and returning promptly, yet all are shown as having work not returned by deadline. I cannot work it out.

Secondly, are you removing finished results from the database very quickly now? I just completed a few tasks, and within 5 minutes went to see the credit that they received but could find no trace of ANY completed tasks in my results.

How long are you planning to leave the results in the database?

G'Day Wang Solutions,
From what I have seen and timed, you will get about 5 minutes from the time you return your result till they are purged from the database.
So you will have to be quick.
I have not seen evidence of the first part of the problem you have reported with any of my computers, no results that are reporting past deadline.

---

I'd like to see the results being available for more than a few minutes too, like many here, i run several projects and not being able to analyse the results to make the best use of my systems by assigning them to more appropriate projects is a bit of a headache at the moment.......no point having machines crunching when they'll return the work after the deadline is up!!
Surely there's a way to archive the results for more than a few minutes.

---

I'd like to see the results being available for more than a few minutes too, like many here, i run several projects and not being able to analyse the results to make the best use of my systems by assigning them to more appropriate projects is a bit of a headache at the moment.......no point having machines crunching when they'll return the work after the deadline is up!!
Surely there's a way to archive the results for more than a few minutes.

BoincView and BoincLogX can both log results.

Stats sites like BOINCstats can also give you data about your recent results about 24 hr after they were returned.

---

The only problem is you don't get the Granted Credit for the tasks, be able to review the stderr data (if any and may not apply to BoincLogX.), and have no simple way to relate what's in the logs to what is shown on the Host Summary.

Alinator

I think when Travis is back he will change the purge time....the only reason it is so fast now was the load on the server when they were short.....Travis is the only person with permissions to change this and other parameters so we are just waiting for him to get back and adjust the server to the new task length ;)

---

I have now had time to look into the first problem a little more, and I still cannot explain it. According to the host record here:

http://milkyway.cs.rpi.edu/milkyway/results.php?hostid=11188

I still have 5 tasks in cache, with the other ones listed having not been returned by deadline. But this PC actually has NO Milky Way work on it currently, and has not done so for several days.

I suspect that the database might have been corrupted and that these tasks are actually from some other host, or have in fact been completed and returned by this host but not recognised for some reason (or were listed twice).

Does anyone have any other thoughts?

---

Proud member of BOINC@AUSTRALIA

Most likely they are just 'ghosts'.

Ghosts are tasks which the project thinks it sent to you, but the scheduler reply which would have told your host about them never makes it back to the host.

Given the severe overload the backend was experiencing recently, it's not too surprising that some tasks ended up getting ghosted.

In any event, they are pretty much harmless and there isn't very much you can do about them yourself anyway. Usually the best thing to do is just ignore them and let them timeout on deadline.

HTH,

Alinator

Thanks, I was not really worried about it from my point of view, but just wondering if anyone else was experiencing it and if in fact it needed to be checked on for reasons of data integrity.

---

Proud member of BOINC@AUSTRALIA

Hi, I'm new to Milky Way.
I also find it peculiar that the stats for each WU are removed so quickly.
Why can't they be removed after24/48hrs?

*burp* ... erm, I mean *bump*

---

BOINC.BE: For Belgians who love the smell of glowing red cpu's in the morning
Tutta55's Lair

I also noticed completed work disappearing from the work unit list in less than 5 min... As others have said would be nice to have them be available for 24/48 hrs.

---

Okay, I've held off on commenting about this and a few other subjects because I figured that eventually Travis would come along, read and take care of the many outstanding issues, but...

This is getting completely rediculous. It's great that the server can seemingly run without intervention now, but good grief! Travis, we've not seen a single post addressing anything for over a MONTH now. :-P

The database is only keeping 14k-15k tasks in progress at a time right now which is a tiny, miniscule fraction of what it was before. The odds of keeping results in the database for a bit longer affecting the operation of the project at this point is virtually nil. The setting to increase the time to keep workunits in the database before purging is annoyingly simple... it would probably take you 1 minute to make the change.

If the whole project is going to take a sabbatical for a month, the least you can do is grant your volunteers the common courtesy of spending 2 minutes to make a post saying so. Otherwise, please, PLEASE get off your rear-ends and demonstrate to us that you still have a pulse and can fog a mirror. 5+ years of volunteering for BOINC projects has shown me repeatedly that NOTHING will drive volunteers away from a BOINC project faster than a sense that the project staff isn't responsive and it's bloody OBVIOUS from the posts lately that this is what your volunteers are feeling.

[edit]Sorry if this post sounds a bit overly harsh, but I'm sincerely getting sick of the projects I volunteer for just mysteriously curling up and dying lately. It's really starting to grate on my nerves. It's not just MW, so my fuse is being shortened by more than just this project. [/edit]

---

Hi, I'm new to Milky Way.
I also find it peculiar that the stats for each WU are removed so quickly.
Why can't they be removed after24/48hrs?

Hi and welcome, but to what I'm not sure now ;)

A brief history of what happenend before you joined.
This project use to have very short WUs - my 3GHz P4 completed them in about 10m33s and another host of mine in around 7min. These short WUs meant that the database had many (many!) entries and this caused problems. I think they were related to the purging process. (Sorry, don't know the programming side, but the forums will have it all). In an effort to to sort this out, settings were changed to what we have now - I think the WUs remain for a maximum of 5min.
Since then, the WU duration was increased, initially by about 60 times, and soon afterwards by about 30 times the original duration. My P4: 10h17m and 5h32m.

Now at this point the database purging could have been returned to its old setting, but at the same time the admins have disappeared.

So, the real problem here is the lack of admins, which is a pity as this project had very responsive admins until that moment.

Hoping this sheds some light on the situation, and that people realize that there's not much point in complaining about the problems here until there someone to respond to it.

One thing's for sure, no response will lose a lot of crunchers. I'll stay while the project works, in the hope for a return of the admins. (Good movie title that: "Return of the Admins") :)

Rod

So, the real problem here is the lack of admins, which is a pity as this project had very responsive admins until that moment.

Hoping this sheds some light on the situation, and that people realize that there's not much point in complaining about the problems here until there someone to respond to it.

One thing's for sure, no response will lose a lot of crunchers. I'll stay while the project works, in the hope for a return of the admins. (Good movie title that: "Return of the Admins") :)

Rod

I guess a lot of us are out and about looking for either a new primary or secondary project to crunch on with all the change overs/updates on equipment, server versions, kredit kops, etc. Is this a normal policy with the admins here to just go missing? Whose minding the store when the servers go down, etc.?

[edit] I don't know why I bothered to post this even after looking through the messages for an hour. So... Does this mean that we are part of the experiment titled "How to Attract and Lose Participants!"? One more thing, I've seen a lot of information about the computing aspects of the project. Is there any publications or anything to show what's actually been done along the astronomy aspect yet?
[/edit]

---

Is this a normal policy with the admins here to just go missing? Whose minding the store when the servers go down, etc.?

It's quite normal @ a lot of the projects once they've been running for awhile, especially during the summer month's, maybe this fall things will pick up as far as communications from the Dev's goes, but who knows ... ????

Is this a normal policy with the admins here to just go missing?

Not at this project. This project was one of the nicest projects with regular responses from the admins. This strange disappearance has really got me puzzled!

Oh well, lets hope they'll be back, because it doesn't take long for the Boinc community to turn against a project.
Rod

Some projects are run by grad students who disappear on summer vacation.

---

me@rescam.org

Some projects are run by grad students who disappear on summer vacation.

If they are on summer vacation when would they likely to be back? Our summer vacation here in South Africa is over Christmas, so I've no idea when their vacation ends.

But still very strange that they would suddenly vanish after being so active, without at least saying goodbye...

Rod

Some projects are run by grad students who disappear on summer vacation.

shh.

school starts back up in 3 weeks or so.

we're clearing results from the database quickly because the server would get overloaded and things would come to a screeching halt. it seems that having the purge running constantly was the only way to fix that.

on another note, nate and i have been pretty busy working on a new version of the application and assimilator, which will mean even more serious workunits; and hopefully less server load. there'll be more about this in the next week or two when we start it up.

---