June 10, 2008 Hey everyone, it's Dave. We're having a great deal of trouble with the database at this point. We've been trying to create new work for you but the database keeps losing connection. This may affect the forum as well. We're working towards a fix diligently and we apologize to everyone for any time we may have wasted
May 29, 2008 There have been some problems with a recent search having WUs that freeze or don't work. Currently i've shut down the server to remove the bad workunits from the database -- I'm hoping to have everything fixed sometime this
evening.
March 24, 2008 Due to many complaints that the reduction in credit was a bit too severe, we've increased the generated credit a bit. It should be around 4 credit per WU
now. Please feel free to keep complaining about credit on the forums and we'll eventually get it right. :)
March 23, 2008 Due to the fact that the application has been highly optimized in the last few versions, most people agree that the granted credit is way too high for the
project by a factor of 3. We've scaled credit down to keep it in line with other projects, so if you notice reduced credit this is why.
March 20, 2008 I've updated the assimilator to deal with some of the performance issues we've been having with it. You should see more work availability and hopefully we wont have
results piling up that aren't assimilated. I've started up a bunch of new searches and hopefully the work will keep flowing faster than before.
March 20, 2008 I just upgraded the BOINC server software. The database is a little shaky right now as we're trying to upgrade to PHP5 to facilitate it better. Therefore the forums will be
down until we can get PHP5 up, hopefully it won't take more than a day. However, the assimilator problem seems to be fixed with this release and hopefully so will the freezeups.
March 19, 2008 We're running a purge on the database to try and get things sped up. During this time we wont be generating any new work, but this should be completed sometime tonight and there
will be new work after that.
March 15, 2008 Good news everyone! The paper we submitted to the GECCO 2008 conference was accepted, this paper is all about results we've gotten using the genetic search/simplex hybrid on BOINC -- so we
couldn't have done it without you :) for more information on the conference: GECCO 2008.
March 7, 2008 We've updated the windows and 32 bit linux binaries. Hopefully this will fix the problem win2k users were having with the apps crashing.
March 6, 2008 We noticed a computational error in the code, so we've updated all the binaries to 1.21. This way everyone has the same version number which makes keeping track of everything better. We've also added some
optimization to more platforms so you might notice improved performance. Let us know if you're having any problems with the new binaries. We should have a 32 bit linux version soon (this was the only platform not updated
yet).
March 6, 2008 Linux binaries for 32 and 64 bit have been updated and optimized, let us know if you have any problems with 1.20 of these.
March 5, 2008 We've updated windows, ppc-apple and 32 bit linux binaries to 1.19. Let us know if these work. We should be adding some new platforms soon as well.
March 4, 2008 We've updated windows, ppc-apple and 32 bit linux binaries to 1.18 to address the problems people have been seeing. Please let us know if these are still causing any problems.
March 4, 2008 I'm trying to roll the application back to 1.13 because 1.15 is still having problems. We'll re-update as soon as we figure out what the problem is.
March 4, 2008 We've updated the applications again, hopefully these will work for everybody. Let us know if there are any problems.
March 3, 2008 There was a stray printf in the code that was used to compile the binaries, which might make any of you using the 1.14 version to be seeing a bunch of computation errors. We're recompiling and signing and hope
to have a fixed version out tonight or tomorrow morning. Until then i've stopped the daemons so everyone isn't seeing a bunch of errors when they're trying to run milkyway. Sorry about the inconvenience for everyone.
March 3, 2008 I've updated the applications for a few versions of the binaries, we should be adding more on wednesday. Let us know if there are any problems with the new binaries. We're hoping these should fix the progress
bug, memory leaks, and also report why windows is giving popups -- so we can fix that in the next version.
February 25, 2008 Sorry for having everything so quiet but we've been working on getting everyone up to speed and a new version of the application. Right now it's looking like you can expect v1.14 on wednesday. We're just working on getting the
new windows binaries compiled (with code that should help us track down why it's giving that popup error). Additionally, I think we fixed the memory leak -- so let us know if you're still getting any of that. So hopefully we'll be able
to put the new binaries on the server without too many problems :)
On another note, we've found what we think might be a problem in how our searches are running (two of the parameters might be dependent) so we're trying a few new things on the server end to see whats going on with this. The
newer searches are using some intentionally difficult data to help us find the root of some of the problems we've been having -- so thats what you're crunching on right now. The genetic search seems to have some problems with this,
and i'm going to start up some particle swarm searches to see if they do a little better with the problem. Thanks again for all your crunching time!
January 30, 2008 Sorry i've not been able to have as much time in the forums lately -- we had the camera ready version of the HCW paper due monday, and the GECCO paper due today,
so i've been quite busy writing and putting all the results you've given us down into the paper.
Hopefully the GECCO paper will be accepted and when it is, i'll give you all a link to it to read. The results are quite good - you're crunching as fast as the large
partition of the bluegene we have access to (1,024 processors). better yet, unlike the bluegene you're available 24/7 and we don't have to compete with other researchers
:D
The asynchronous genetic search/simplex hybrid is performing really well, and while not as fast as it's synchronous counterpart it is still quite good. The results also
show that by with some fine tuning we'll be able to improve the performance even more.
I've started up two new searches (gs_170..174 and gs_180..184) that further the results we have in the GECCO paper. They've refined the search a bit and are using more
points in the hybrid simplex operator we've been using.
I also hope to start some particle swarm searches up and running again shortly.
We've also been in the process of looking at some new undergraduates to work with our project to take care of the server and help with compiling binaries and that type of
thing, so hopefully that will also help clean up a bunch of the bugs we've been having.
cheers everybody, and thanks for continuing to work with us :D
January 27, 2008 We've submitted a camera ready version of our HCW paper, and due to an increased page limit, we've added the first results for running the double shot genetic search on BOINC. You can access the paper here if you'd like to
read it HCW Paper and see the results of your work. The results are interesting, as they tend to suggest that slower processors aid in the explorative functions of the
search (or the global search aspects), while faster processors aid in the exploitative aspects (or local search). This is good news because it means every computer that joins counts! We again like to thank you for all your shared
CPU cycles.
January 8, 2008 Our labstaff is going to be taking down milkyway for awhile to run some tests to see why it's been crashing so frequently. Hopefully it will be back soon running smoothly.
January 7, 2008 It looks like there have been some more power problems with the server, which is why it's been offline. We're working our labstaff to get these problems fixed.
January 3, 2008 It looks like things are running smoothly on our end, however we've had some users report that they're still having problems. Whats worked for them is a detach and reattach to the project. If you're still having
problems try giving that a shot -- if it doesn't work let me know.
January 3, 2008 While I was gone, it looks like our upload and download directory permissions got messed up -- read/write/execute were removed for anyone other than root. I'm hoping this was the reason that
you couldn't upload or download files for the workunits. Please let me know if things are working correctly now. I have the assimilator/validator running manually again so i can watch to see
if this problem happens again.
December 26, 2007 I'm going to be out of town until new years eve, so if i don't get back to any questions on the forum or email that's why. I've set up the assimilator and validator to run as daemons in the
config.xml now, so if the machine goes down and comes back up, these should start back automatically even if i'm still out of town -- so hopefully things will run smoothly while I'm gone. I hope
everyone is having a happy holidays!
--Travis
December 19, 2007 It looks like we've found and fixed the problem with stats export and update, so as of now, stats should be updated and exported every 6 hours.
December 19, 2007 For an update on the research front, Professor Carlos Varela presented the work for our eScience paper at the conference in Bangalore, India, last week, and I'm
glad to let you all know that it was awarded best paper (of 206 submissions and 60 acceptances, only 2 received best paper). Additionally, a paper we submitted to the 17th International Heterogeneity in Computing Workshop
(HCW 2008) was just accepted, which contains prelimiary results for the asynchronous genetic search we've been using.
December 19, 2007 A week or so ago, the server crashed more than a few times. It seems that there was an overloaded circuit causing these crashes, and our labstaff has fixed it so hopefully things should be running much smoother
now.
December 14, 2007 Some of the first convolution work units generated are granting an incorrect amount of credit (~4 credit), this has been fixed and now all newly created convolution work units should be granting the correct amount (~6.25 credit)
December 14, 2007 I'm starting up some convolution searches, let me know how they run.
gs_50 ... gs_54 are running convolution
gs_40 ... gs_44 are running non-convolved
both are running the hybrid simplex/genetic search optimization method. I've stopped the particle swarm searches for the time being because they don't seem to be working very well and I'm trying to figure out why. We should
have some simulated annealing work units up and running next week.
the convolution workunits should take a lot longer (mabye 30+ minutes), but should grant the same amount of credit per time as unconvolved work units. let me know if anything is off.
December 13, 2007 There was a rather big snowstorm here in the northeast and the server went down. The Internet still seems a bit weird, so please bear with us until everything is up and running correctly.
December 12, 2007 We're uploading new binaries with will be able to calculate the convolution code (although they won't be doing this quite yet), please let
us know if you have any problems with the new binaries in the forums. We're hoping these should also fix some memory leak issues as well.
December 11, 2007 Things have been a little quiet for awhile so here's an update as to what we've been working on:
1. Nate has committed the convolution code, and we've finished debugging it. Hopefully today there'll be some new binaries as soon as I can get Joe to take a look at it (unfortunately he's the only one that can sign them at the moment). The
convolution should take about 4-8x more time per work unit (which should make the people who want longer work units happy), because instead of calculating the fitness of the astronomy model against fixed star points, it calculates the fitness
against probabilistic distributions of star points (ie, instead of a star being a fixed point, it has a probability distribution in a range of space). Nate is going to make a post about what exactly is going on with this in the next couple
days.
2. The particle swarm searches seem to be running, however they're not performing nearly as well as I expected, so i'm looking into ways to optimize this.
3. I implemented a hybrid simplx/genetic search optimization and have been testing it out. The new genetic search work units you've been seeing are doing this. Where our previous genetic search reproduced parameters based on the double shot
approach (see our escience paper), these generate child parameters using the simplex method. This is interesting because instead of generating child parameters based on two parents, the simplex method can use N parents. Right now i'm testing 4
(the next batch will test 5) and i'll keep increasing to see what effect N has on the convergence rate.
4. I'm working on implementing a simulated annealing optimization, so if you start seeing work units starting with 'sa' it will be for that.
So that's what we've been up to in the last few days, thanks to everyone for continuing to crunch our numbers :)
--Travis
December 5, 2007 The forum problem is fixed, so continue to keep updating us about where our site is breaking :) We're looking into the
PHP problems we're getting all over the website so hopefully they will be fixed soon.
December 5, 2007 It looks like we're having forum troubles, we're taking a look at it.
December 2, 2007 The server locked up again, so we're taking a look into why it's doing this. Hopefully it will be sorted out shortly.
November 30, 2007 If you've noticed that some of the workunits start with 'ps' those are for the new particle swarm
search that we just got working (genetic search starts with 'gs'). With the last update to the
binaries the resulting work units return a lot more information about what kind of search they
were run from, when they were generated and how they were generated, so we're going to be able
to really start studying whats going on in depth now.
As to what we're going to be working on while getting these new results, the plan is:
1. fix the remaining memory leaks in the astronomy code
2. Nate is going to be updating the code with the 'convolution' which will make work units a bit longer but more accurate
3. i'm going to be implementing adaptive simulated annealing and maybe parallel tempering and stochastic tunneling search methods
4. we'll be doing a lot of data analysis about the particle swarm searches and comparing those to the asynchronous genetic search
5. there's an upcoming genetic/evolutionary algorithms conference (paper submissions due by mid january) and we're hoping to submit
the results for genetic search (and particle swarm if we have enough) to that.
thanks again everyone for crunching our numbers and putting up with all the server troubles last week. :D
November 30, 2007 The server totally crashed last evening, but things should be back up and running now.
November 29, 2007 After a recompile and restart, the server appears to be serving work again (finally) and you should be getting the newest binaries.
November 28, 2007 For some reason work isn't being uploaded, even though there are workunits available in the system and the new binaries
seem to be recognized by the server. I've sent an email off the the boinc projects list and hopefully someone will be
able to respond and help us out. We'll also be doing a reboot of the server sometime this evening in hopes that might
help things out as well. We'll keep you posted on the news and in the forums.
November 28, 2007 A user posted a question on the forums asking what we've done so far, and I figured the response might be newsworthy:
We've actually already gotten some interesting results. We've crunched through 5-10 asynchronous genetic searches (AGS) for two different sizes of volumes. We're comparing this data to how the AGS converges on the Rensselaer
grid and BlueGene (results we presented in the eScience paper linked on the main page), and hope to have another publication out this month comparing Boinc, the RPI Grid and the BlueGene for the astronomy project. Before the
server issues, the project was assimilating about 2-3 workunits a second, which is comparable to the large partition of the BlueGene we have used (1024 processors). So far what we've done is pretty promising :)
Once we get the new binaries, we'll be able to test all different kinds of searches like particle swarm, simulated annealing and some others to see which work best for the astronomy application and which scale the best and are
most resistant to the heterogeneity of BOINC, just to mention a few things. Nate should also be updating the code in a week or so with the 'convolution' which makes the calculation a lot more complicated - but more accurate.
Instead of testing the astronomy model to fixed star points, it's tested against star points that are probabilistic distributions, increasing the computation by almost an order of magnitude. How that works out will be really
interesting to see :) I've talked to Nate and hopefully he'll be posting a bit more about how the convolution works and what it'll be doing.
--Travis
November 28, 2007 Awhile back some workunits got awarded massive amounts of credit due to some benchmarking issues in boinc (ie, 200,000+ credit). While we've waiting to get
new binaries signed and uploaded, I've gone through the database and removed credit from these workunits, so our toplists should be looking the way they're
supposed to. I apologize if any users/teams lost more credit then they think they deserved (send me a message if it's a problem), but I did the best I could
with my limited knowledge of MYSQL. However, with the new validator and fixed credit scheme up and running, I think things will even themselves out once
we start crunching numbers again.
--Travis
November 28, 2007 After fixing the shared memory problem, it looks like the server isn't recognizing the new binaries. We'll be signing new ones tonight and putting them on the
server and hopefully this will fix the problem of not being able to download any work. As far as we know, there is work available, but the server doesn't think
there are correct binaries to upload for it.
November 27, 2007 Sorry for all the errors as of late, but we've had another problem with our server. Something went wrong with the shared memory while updating the binaries and
the feeder isn't starting back up. We hope to have things working soon, and the new binaries available.
November 27, 2007 We've completed updating the binaries so we should be able to do particle swarm searches and gather a lot more information about how searches are being performed.
Hopefully these new binaries will fix some of the memory leak problems and incompatibilities with linked libraries as well.
November 27, 2007 We're currently in the process of updating the binaries, so there won't be any work available for the next hour or so while we get them ready. The new binaries
will allow us start performing particle swarm optimization in addition to genetic search (as well as simulated annealing and a few others when they get implemented
server side). Additionally, we'll be able to track different statistics about how these different optimization algorithms work on such a heterogeneous environment
as BOINC, and be able to use this to perform different types of optimization.
November 27, 2007 There was a power outage this morning and I had to manually restart the BOINC demons after the machine was turned back on. Everything should be working again.
November 27, 2007 Server problem should be fixed now and work should be availble.
November 26, 2007 We've had some problems with server processes not shutting down correctly so the server needs to do a hard reboot. There might not be any work
available while this is happening. Things should be working again as soon as our labstaff gets back to us.
November 26, 2007 We've updated the validator which should now be granting fixed credit depending on how much work the workunit will do (determined server side). We hope this
will fix any problem with 'cheaters' or people with benchmarks that didn't run correctly being awarded way too much credit. You might notice that credit granted
might have gone down a bit -- for the current size workunits we've made that 1.0 credit. Workunit size should be starting to go up, so this should increase as well
for longer running workunits.
November 25, 2007 Sorry about the flakyness of work availability the last few days. We've been having some problems with the server but hopefully they will all be fixed shortly.
November 19, 2007 Here's an update about what we're currently doing (taken from the forums):
Right now we're looking into two different things:
1. The effect of the asynchronous return time of work units on the quality of the genetic searches we're running. If you'd like to read more about the asynchronous genetic
search we're running, we have some papers we sent out recently to workshops/conferences detailing the search -- but we only had results using the new BlueGene supercomputer
here at RPI, and our local RPI Grid of clusters.
The papers can be found here:
http://www.cs.rpi.edu/~deselt/desell_genetic_ppam_2007.pdf
http://www.cs.rpi.edu/~deselt/hcw2008.pdf
What we're hoping to do with this information is determine what effect a really heterogeneous computing environment has on our asynchronous genetic search, and figure out ways
to optimize the search for these types of environments.
2. The astronomy program we're running currently calculates an integral over the wedge of the sky we're modeling. Since it's not possible to actually integrate the function
we're interested in, we have to calculate the integral the old fashioned (and expensive way) of calculating a bunch of points and calculating the area between them. The next
few searches we plan on doing will calculate the integral with varying accuracy - so expect work units of varying times (probably 8 minutes to over an hour). With these results
we'll be able to figure out what effect the integral accuracy has on the rate our genetic search converges to a solution, and what effect the integral accuracy has on the
quality of the answer.
Some other things that we have in the works is that i've been working on an asynchronous particle swarm algorithm to use (as opposed to asynchronous genetic search). For a
brief overview of what particle swarm optimization is, wikipedia isn't too shabby: http://en.wikipedia.org/wiki/Particle_swarm_optimization . We'll be able to contrast and
compare these two searches, which should be interesting.
Also, we're using the results to compare doing this work on BOINC to the BlueGene supercomputer at RPI. Who knows, with enough participants in the project -- you guys might
beat out the 7th fastest supercomputer in the world! :D (see: http://www.top500.org/lists/2007/06 )
--Travis Desell
November 6, 2007 Sorry about the lack of updates, but I have just been informed that our genetic search work generator has
been tested and works. All that is needed to be done is to update the binaries and create a system for starting new
searches. The plan is to meet tomorrow, and have this up and running to get some results. We are planning on
putting out a paper soon, so we will be looking forward to see how everything works out, and I'll try to put the
paper on here, so anyone interested can check it out.
October 17, 2007 We're in a bit of a dry spell right now as far as updates. It's been midterm period for all the developers
here, so we haven't had much time to work on this project. In addition, it appears that we are having some technical
difficulties with the server. The server was temporarily down, and some of the services did not reload properly on
reboot. We do have our genetic search implemented on MPI now, so this means we should be able to port that code over
to our BOINC server soon, so I will be working on fixing the server as well as getting that automated search up this
week.
October 7, 2007 I just released version 1.06 of the application, and this will now run on windows unlike before. In addition
to working on windows, it also correctly handles checkpointing. This means it won't crash if you suspend it and resume
it later. The only platform that has yet to upgrade to 1.06 is mac, but that should be up tomorrow. Now everyone should
be able to get some credits. I'll do my best to get some work up on the server real soon, and we should be working on
some real problems really soon!
October 4, 2007 I just wanted to let everyone know that we've got one of the project scientists checking out the boards for
now, and he has posted what exactly our project is doing. You can check it out in the MilkyWay@home Science message
board. I'll try to prod them into making a web page with all that info on there so I can put a link to it on the main
page, but at least for now you can check that out to get an idea of what we are doing.
October 3, 2007 I've got all the error and warning messages I see fixed on the web page, and I've submitted new versions
of the software to the server also. In addition I added some mac binaries, so if you have a mac you should be able
to compute some stuff for us now. I've fixed the errors that have to do with CPU usage and disk usage, and the
run time should be much better for the application. There still seems to be a bug in the Windows version of the
application; something to do with BOINC thinking there is a memory leak, and the app will continue running after
100%. In response to some of the requests of the users. I believe the RSS feed is working now, so you should try
it out. You should be able to attatch to the project now, you should be able to use either http://milkyway.cs.rpi.edu/
or http://milkyway.cs.rpi.edu/milkyway/. I'm gonna try to get an automated system up in the next week or two, which
means I'll have some real work up there, and it will probably last more than 3 minutes. Those workunits were only
meant for tests. Once that is done, I'll release a new version of the app and we should be good to go!
October 2, 2007 It looks like the message boards are at least functional. There are still some errors to be fixed, but
at least they are useable. This means I can start getting some feedback from you guys! Also, I found the bug
in the code, so if you have gotten work and it ran really slowly and caused your computer to do some strange things
I should be able to fix those very soon.
September 29, 2007 I was just able to upload some binaries for windows_intelx86 platforms. I've been running some tests
on them all night, and it seems like the windows version is using significantly more disk space than the UNIX
versions. For the time being I will be increasing the bound on the disk so I can verify some results, so if
you're running windows on a 32 bit processor and you get some errors while running, this is due to these tests.
On the forum front, it appears that we have old html files on our server and I'm going to need to do some serious
updating there, so it may be a bit longer before we see the forums up. I tried to get them working yesterday and
today, and all I managed to do was break the site, so when we finially get things figured out for the forums,
you may see the page go down from time to time... but don't worry, that means we're working on it!!! Finally,
I granted all the claimed credit, so you should all be getting small amounts of credit, but credit to say the least.
I'm going to look into automating it. Things seem to be looking good right now, and once we have a genetic
search implemented I will put some major effort inot getting the forums up so I can get some feedback. Hopefully
things keep moving at this pace, and we will have a fully functional project up very soon!
September 27, 2007 I've added some new binaries and have just realeased version 1.04 of the application. I found a small
bug, but the bigger news is that we can now support four platforms. The platforms include: amd64-unknown-freebsd,
i386-unknown-freebsd, i686-pc-linux-gnu, and sparc-sun-solaris. I realize I'm missing a few key platforms, but
thats next on the schedule. I'm also going to try to get those forums up soon, so stay tuned for that... Maybe
they'll be up this weekend, who knows? Also our research group has joined with the biology department, and we
will be releasing another BOINC project soon called L1F3@home. I will have a link for their page once it's up.
We may even try to grant bonus credits if users are registered for both projects, so keep an eye out for that too.
September 20, 2007 Today we finally were able to recieve results from a client!! Currently the only platform that has bug
free code is i686-pc-linux-gnu, but look for updates when I add new platforms, it shouldn't be too long. I will
now start developing an autonomous system for processing incoming results. That's when we will start having some
real work to do, for now we are still in testing stages. Thanks for your continuing support and patients.
September 16, 2007 This is just a warning that if you have joined this project, there are still problems with the program
and we are still technically in the early stages of development. We hope to have some real work up on the server
soon, but as of right now things are still a little broken. I will do my best to keep up withrequests and needs.
I will post again when we have things working, and once that's finished I will seriously get to some of your
requests.
September 6, 2007 I notice there are some concerns about forums and the like, but unfortunately I'm the only person
working on this project and I am only a college undergrad doing some research on the side. I appreciate your
interest in this project, and also your commitment to keep things respectable while I am away. Currently I'm
working on fixing some bugs in the actual program, and once I get something working I will add binaries for new
platforms and update the web interface. Hopefully I can get everything running smoothly, and I hope to have
project working correctly sometime this weekend or during next week. Once that is finished I will get right
on top of web stuff. Thanks for your patients!!
August 28, 2007 I have a couple more updates for you today. There is now some work on the server, but as of right now,
the upload directory is not readable or writable by users that are outside of our network here, so clients can't
upload their results. I have created binaries for three platforms: i386-unknown-freebsd, i686-pc-linux-gnu, and
sparc-sun-solaris. If you are connected to the project, but are getting errors about having an outdated signature
file, then you need to detach from the project then reattach; there was a small error in our code signing procedure
which is now fixed. Lastly, we are having a little bit of trouble with clients reading from input files. I will
be working on a solution, but until then the results will be insignificant. Thank you for your participation and
your patience, and hopefully we will have a working product up soon.
August 26, 2007 I just wanted to update everyone, and show that this project hasn't died. First of all, we just got
a brand new server machine that will be dedicated to our project. Everything has been loaded onto this machine,
but we still have a couple of bugs to iron out a couple of bugs. We should have new work to be done in the next
couple of days.
July 8, 2007 A new web interface is up for Milkyway@home
July 7, 2007 Milkyway@home project is up and running, and account creation is now enabled