October 23, 2009 I've also made a powerpoint version of the slides from our BOINC workshop presentation here. October 22, 2009 Hello from the 5th Pan-Galactic BOINC Workshop in Barcelona! I had my presentation today so I'm making the slides available to everyone. You can download them here (sorry they're in keynote for all the Windows users out there). I'll be back next week so try not to break anything before then. --Travis October 5, 2009 One of our astronomers started up some searches today, but it looks like the workunit size was too small and the server couldn't keep up. We're going to start up some new searches tomorrow which should take longer to process and not make the website unusable and server unaccessible. --Travis October 2, 2009 Server went down last night but I've fixed the problem and everything should be up and running smoothly again. --Travis September 21, 2009 Things should be running again. September 21, 2009 Everything should be running again shortly. No one informed me about the server crash and I've been sick in bed with the flu :( --Travis September 15, 2009 I've decreased the sleep time on the feeder again (down to 1 second now), so people who were having trouble getting workunits hopefully won't any more. I'm hoping this might speed up the webpages a little as well. September 15, 2009 We've had our recent submission to the 2009 e-science conference accepted. I've made a forum post with a link to a preliminary version of the paper for everyone to read and comment on here. Thanks again for all your support in making this research possible! September 14, 2009 I have some idea what's causing the workunit errors right now. I've started a few new searches and upgraded our assimilator/validator so hopefully that will fix it. Let me know if you're having any issues with the *_6 workunits. September 14, 2009 Looks like something weird happened to one of our searches over the weekend. I've made a couple changes to see if it fixed the problem. Let us know if you're still having WU errors. September 11, 2009 There were some other errors in the WU parameter files, but I think everything should be cleaned up now. Newly generated workunits should have the correct size and credit. September 11, 2009 Milkyway's credit awarded has been a bit too high for awhile now, and we're bringing it down to be in line more with other projects. We've been using a credit multiplier of 7.5 per double precision FLOP and other projects are using around 5.4 per double precision FLOP. We're reducing our multipler to this, so credit should be down around 20-30%. I know everyone hates credit reductions, but it was really needed. We also have some performance improvements for CUDA and ATI applications on the way out, so hopefully it won't be too bad. As usual, try to keep the forum flaming about the credit changes somewhat civil. September 11, 2009 On 9/12/2009 from about noon until 2 pm EDT the ATI application will be available through BOINC in order to assist the BOINC developers with any issues relating to the new 6.10 client that will have ATI support. Please see this thread for more information. August 28, 2009 I've implemented a new search method, differential evolutuon, so if you are crunching workunits starting with 'de' that's what these are for. I'll be putting a forum post describing what these are doing in more detail up sometime today. --Travis August 26, 2009 The CUDA application has been released on BOINC, please see this thread for more information. The Milkyway@Home team appreciates your patience in reaching this milestone. This code has been implemented and tested on the GeForce GTX 285 donated by NVIDIA to the Milkyway@Home research team. August 21, 2009 A little more news on the server software upgrade. We're upgrading it to put allow us to correctly recognize NVidia and ATI graphics cards, which means the applications we supply will be able to support those GPUs. August 20, 2009 A planned BOINC server side software upgrade will take place on August 21 at 4:00pm EDT (America/New_York). August 12, 2009 Professor Newberg has gone through all the steps to allow MilkyWay@Home to actually accept donations and has written a letter of request, which I've posted here. The letter discusses the progress we've made in both scientific computing and astronomy, as well as our current funding situation. You can directly access the page to make donations here. August 8, 2009 We're having a small problem with the assimilator, it should be back up and running shortly. July 30, 2009 An alpha version of the CUDA App has been released for community preview, please see the following forum post. July 29, 2009 Since some users are still reporting missing credit, I've re-run the stats export which should hopefully update their credit to what it should be after the restore. This might take a little while to show up correctly, but let me know if this has fixed your missing credit, if you're still missing credit. The only credit that you should be missing would be from 4am this morning til around 2pm this afternoon. July 29, 2009 We restored the server to this mornings backup. Let us know if you're still having any problems. July 29, 2009 Looks like labstaff is in the process of restoring the lost data right now. Hopefully we didn't lose too much after the restore is done. July 29, 2009 Our database crashed or got corrupted this morning and it's looking we lost a lot of user accounts and credits. For now if you lost your account just make a new one. We're looking into this right now and are trying to restore all that we can. --Travis July 21, 2009 I'm testing a genetic search right now with the new verification. Let me know how things are working. Work should be flowing now July 21, 2009 I'll be updating the server daemons tonight with better result verification and updated particle swarm and genetic searches. There will probably be some downtime while I get everything up and running correctly. We've also been examining some issues we're having with our fitness calculation, so we'll be releasing a new application and its code to deal with that as well. It also seems we're having some excessive credit issues again so things are going to have to be tweaked (have fun flaming our forums :P). June 22, 2009 Do to some rather unsavory activity in the Cafe, it now has a minimum RAC of 5 to enable posting messages. --Travis June 15, 2009 John has made the following post in the science section. It's an in depth explanantion of the current astronomy work being done here and quite readable and easy to understand (even I can get it). I recommend it if you'd like to know a bit more about what's going on here on the astronomy end of things. --Travis June 11, 2009 Travis here. I'm going to be camping until Sunday so don't break anything while I'm gone. Hopefully Matt and John will keep the server running and filled with work for you guys :) I'm still having a few problems with the CUDA application for regular milkyway, so it'll probably be out Monday or Tuesday after I get back. June 11, 2009 I noticed a problem in the binaries and code I put on the code download page. I removed them to do some fixes, but they should be available tonight or at the latest tomorrow morning. June 11, 2009 Matts searches had the wrong sized integral so they were crunching much faster than expected. We've stopped these so there shouldn't be any new short workunits. June 11, 2009 Since it looks like the server is handling things fine (no work shortages), I've put out the CUDA code for milkyway@home. If this works and the server holds up we won't need milkywayGPU@home. The code can be downloaded in the usual code release directory. I've also added two CUDA binaries for intel versions of OS X, single and double precision if anyone would like to test them out. They're also in the code release directory. If anyone would like to test the code for linux and windows that would be appreciated. I've started a new forum thread about this here. June 11, 2009 A few more optimizations to the search process have been made, so we're running some of the old stripes again to get better fits. The new runs are ps_new_11_1, for example. June 11, 2009 We have started a thread in the science forums relating to the astrophysics portion of the project for those who are interested. June 10, 2009 If you look below the news, the plots describing the progress of the different searches we're running is now on the main page. They're updated every five minutes. June 10, 2009 Nate finished up his PhD thesis and we've made it available here. Its also in our publications section. Thanks everyone for your help! June 9, 2009 There was just a power outage which took out the server, it should be back up and running now. I also did an update to the feeder which has it running more frequently. I think this should improve the work availability. Let us know how things are working. June 9, 2009 I made a little update to the search progress forum thread. You can now see how the searches are progressing here. June 8, 2009 We are once again attempting 3 stream searches. If you are getting errors relating to the search ps_sgr_235_3s_4 you can get a fixed version of the code here ( http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=886#24282 ) compliments of Cluster Physik June 2, 2009 Here is a post relating to the new ps_sgr_208_* runs you may have been seeing and the Sagittarius Geometry that makes them different. ( http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=881 ) June 1, 2009 Looks like I fixed the problem. Things should hopefully be running smoothly again. June 1, 2009 I broke the server doing some updates to get our new astronomers running their new searches. It should be back up shortly. June 1, 2009 I've increased the minimum sendwork time server side to 1 minute (this might be a bit high for some of our GPU clients so if it is let me know). I'm hoping this should increase work availability for everyone, so let me know how it works. June 1, 2009 Looks like the gpu code release was missing yet another file. I've updated the v0.05 code and it really should work this time :) June 1, 2009 I've updated the code to v0.05 and its in the code release directory. I've combined the linux and osx makefiles into a single makefile, which can also compile the GPU applications. I've included a new (faster) kernel with this release as well. As always, the new code is for discussion in the following forum post and I've also started a thread for makefile discussion. May 22, 2009 I've updated v0.04 to the code release directory. This should work correctly with the compile script. The next steps on our to-do list are getting checkpointing working correctly for it (i'm pretty sure its not working), and compile time specified single or double floating point precision. In the next couple days I should get the milkyway_gpu forums up and running and some test work units available. May 22, 2009 I updated another version of the GPU code. It should have the evaluation directory now. May 21, 2009 I missed a couple files in the previous release, so theres a v0.02 in the code release directory now. May 21, 2009 I've released a preliminary version of the CUDA application in the code release directory. For more detail, there's a forum post here. This should run on the same test samples, however I'm having some issues getting a working makefile/build script using both BOINC and the CUDA libraries, so if anyone has some knowledge about how to do this it would be really appreciated. May 20, 2009 Looks like I've finally gotten the CUDA application working. I'm in the process of cleaning up the code, and it should be available for you to play around with tonight. It's my first crack at a CUDA application, so it's a bit rough, but currently it's crunching the current stripe 82 WUs in around a minute using a GeForce 9600M GT. I'm sure there's quite a bit more optimization we can to do get it running even faster, as this is the first version and not optimized at all. Part of the reason this has taken so long was that I was doing quite a bit of testing comparing double and single precision math on the GPU. I've come up with some schemes that will allow us to use older graphics cards which don't support double precision math (like the GeForce I've been using); which should let a lot more of our users be able to use the GPU application. More news later when I post the code. --Travis May 19, 2009 Looks like the power is going to be out until tomorrow morning with another outage tomorrow afternoon. I'll post more information as we know about it. May 19, 2009 There's another power outage today at 4PM EST, which will bring the server down again. I'll try to get it up and running right after the power comes back. May 18, 2009 The reason we went down today a few times was that there were power outages on campus while they were doing things to the electrical system. Hopefully things should be up and running smoothly now. May 18, 2009 Looks like the assimilator and validator is back up and working. Let us know if you have any problems. May 18, 2009 I'm working on the assimilator and validator right now so hopefully I'll have them running shortly. Except a large news update this week by wednesday -- our CUDA application is almost working so it should be out by then. I'm just doing some last tests before I put out the alpha code for everyone. --Travis April 11, 2009 So I've been working on making the new project specifically for GPUs. This is will alleviate the no work situation because we will be giving much longer workunits to GPU clients. However, I've discovered a bug in a python script that was made when compiling. I need labstaff to give me permission to overwrite the file, but am not sure they work on weekends. So hopefully by early next week we can have a new project up! April 8, 2009 There might be some downtime this afternoon as we should be meeting with labstaff to get the MilkywayGPU@Home project up and running. I'll also be posting CPU code for the GPU application this afternoon so the people who have made the GPU application for the ATI cards can get their hands on it and modify it so it works with ATI cards. We should have the CUDA code out sometime this week. March 29, 2009 It looks like the assimilator is working again, so work should be available shortly. March 29, 2009 I'm working on getting work generation working again, for some reason the last crash is causing the assimilator to not run. Hopefully we'll have it figured out tonight. March 25, 2009 Sorry for the recent lack of communication, but we've been working on quite a few things behind the scenes here.
First, my new laptop just came in the mail, so I'll be able to seriously start working on a CUDA version of the milkyway application. We'll be developing this for OS X, Linux and Windows.
Second, we're going to be doing a few server-side changes to how things are run, which should significantly help with workunit reporting and work availability. We'll be splitting MilkyWay@Home into two separate applications/backends, Milkyway regular and Milkyway GPU. This will allow us to keep current workunit sizes for non-GPU applications, while farming a lot more work out to the GPUs (so they need to contact the server less frequently). We'll be doing different optimization methods on the regular Milkway and the GPU Milkyway so they'll both be doing equally valid research. In the next day or two I'll be posting the code that the application will be using for the GPU version. After testing to make sure that it's working correctly we'll swap to only awarding credit to the GPU applications for GPU milkyway (that way our servers wont be as bogged down, which will mean better work availability). Also, before a credit firestorm erupts again, we have no plans to award less credit/work to the GPU applications, so you don't need to worry about that. March 11, 2009 Sorry for the lack of communication recently. I'm back from my flu so it should be a bit better now :) I started up some new searches, which should hopefully do a better job avoiding the edges of the search space which was causing some of the weird acting very long workunits. Should be starting up a few new types of searches in the next day or so as well, more on that later. February 28, 2009 I started up some new searches, these should take about twice as long to crunch (which should help server load and work availability). I've also started searches on new stripes (24, 25, and 26) which are areas of the sky we haven't looked at before, so we're real interested to see how these come out. February 27, 2009 I'm reducing the workunit queue to 6 per core. Increasing it to 20 seemed to make the work availability problem worse. I'm thinking that with a smaller number it will take more work requests to clear out the scheduler's work queue. This is just a temporary fix so don't get too worked up about it. Once we get the scheduler's queue increased we'll up the WU queue again. February 27, 2009 I'm dropping the purge back to instant, because there are going to be a whole lot more workunits floating around in the system with the increased WU queue. If nothing bad happens I'll increases it back to 3-6 hours. February 27, 2009 I'm going to be updating the assimilator/validator today, so there might be some work shortages. I'm putting some very limited redundancy into our particle swarm search, which should hopefully fix the problem we've been seeing with outliers messing up the search direction. I'll also be putting in a few changes which should help the work availability problem: first I'm raising the workunit queue to 20 (from 12) and I'll also be increasing the minimum number of available workunits on the server. I'll be starting up a couple new searches this afternoon, and have modified the awarded credit slightly upwards, so you might see a better RAC with these. February 25, 2009 I just wanted to let everyone know that we're looking into the problem people are having with not getting work from the scheduler. I'm thinking it might be that our server isn't accepting enough incoming http requests or something along those lines. Hopefully we'll have this sorted out in the next couple days. February 20, 2009 The server seems to be running fine, so I've increased the purge time from 3 to 6 hours. February 19, 2009 We've updated our workunit quota, so people using GPU apps should be able to do a lot more crunching. February 19, 2009 Some of the applications with the tag 'mindc_linux: 0.16' are returning invalid results. This might be due to the fact they're not using the latest checkpointing code, or for some other reason. If you're using one of these please update your application to the version compiled with the 0.18 code. I'll stop awarding credit for these tonight (Feb 19th) at midnight. February 18, 2009 I just noticed that the s22 and s23 were generating the wrong amount of credit. They should be around 12-13 each, all the newly issued ones should reflect this change. February 18, 2009 We started up some searches for stripes 22 and 23, which are all new. These also have multiple streams AND multiple cuts, so they'll take a bit longer to crunch. Please let us know if you have any problems with these. February 18, 2009 I think the drop to 6/9 credits was a bit much (for now), so I've upped the credits to 8/12 credits for stripes 79/82/86/20/21. I'm really gonna let this sit for awhile (promise) and see how things pan out, unless something drastic happens. February 18, 2009 The server won't be awarding credits to applications with a version number less than 0.16. If you're not receiving credit for your optimized application then you probably need to update it to a more recent version. There are links to valid optimized applications in our code discussion and number crunching forums. If any valid applications are having problems with this, please let me know. February 17, 2009 I've set the purge to 3 hours now, so you should be able to look at your workunits for awhile. If it doesn't kill the server in the next few days I'll look into increasing it to 6 or 12 hours. We might even be able to get it back to a day. February 17, 2009 Sorry guys, but in order to have some semblance of cross project parity in this (at the request of other projects), we had to reduce credit again. New WUs should now be generating around 6.1 credit for stripes 79, 82 and 86, and around 9.2 credit for stripes 20 and 21. Other projects have been getting requests from their users to increase credit to match ours from their users, because they felt they couldn't compete -- and this is exactly the situation we'd like to avoid. February 17, 2009 Given some discussion about credit rates, and the fact that the standard is 1GFLOPs/day = 100 credits. 1GFLOPs/day = 100 credits would mean 79, 82 and 86 WUs would be generating ~1.4 credits, while 20 ad 21 would be generating ~2 credits per WU. I think this is too low. What I'm thinking about dropping credits to around 4-6 credit per 79, 82 and 86 WU, and 6-9 credit per 20 and 21 WU. I'm open to a bit of discussion on this because given that we're an alpha project I think our credit shouldn't be too low considering what you guys have to put up with here, however it definitely shouldn't be 10x what the supposed rate should be. February 17, 2009 I started up some searches for stripes 20 and 21, let us know how they're crunching. February 17, 2009 We're going to be working on something that will let you guys see how the searches we're currently running are progressing, in somewhat real time. I've started a thread about this (with some preliminary images) here. Please feel free to comment with suggestions on how we can make this more interesting for you :) February 17, 2009 It's time to get all the older applications out there updated. As of February 17, midnight EST, we won't be awarding credit for applications flagged as v0.15 or earlier. These applications have a checkpointing problem and can frequently return invalid results to the server. So nows a good time to update to the newest optimized applications if you're using an older one. There are links to them in the number crunching and code discussion forums. February 16, 2009 I've updated the credit rates, WUs should be generating around 12 credit each for the new searches. I've also lifted the credit limit. This is a pretty big change so there's probably going to be a bit more tweaking to be done, but I'm going to see how this goes for awhile. February 16, 2009 Dave didn't update the version number in the code, so the windows apps are now at v0.19... Please direct all flames his way :) February 16, 2009 The stock windows application has been updated to v0.18 as well. February 15, 2009 I've updated the stock application for linux and os x to v0.18. The code is in the code release directory as mw_v0.18d. This has a couple new performance improvements (thanks to our users) which has it running 30% faster, or more. This should bring our stock application (and some of the other optimized versions of it) in line with the fastest optimized apps we're seeing out there. Because of this, in the next day or so we'll have another (much loved) credit modification. I'll most likely be dropping the credit rate by about 30-50% but at the same time removing the credit limit. Due to the nature of the performance improvements, older hosts should see better performance improvements than newer ones, so for most people the credit/hr should be the same, while faster hosts might lose some credit or gain some credit depending on where they are compared to the speed limit. February 14, 2009 I started some new searches, these are doing particle swarm optimization (as opposed to the newton method), so let me know how the WUs are crunching. These workunits should start with 'ps' instead of 'nm'. February 13, 2009 I updated the code for v0.18 again, this should fix the problem with stripe 86 and the problem where the probabilities were a little off. February 13, 2009 I've upated the alpha release of the v0.18 application. It should now compile correctly. It has some changes for a small checkpointing bug and more performance improvements. I'm going to let other people try it out for a bit before I update the stock application. So if all goes well stock should upgrade to 0.18 in a couple days. After we've updated the stock application to v0.18 and we see that it's working correctly and we have optimized versions of it available, I'm going to stop awarding credit to pre 0.18 compiled applications, because we don't have any other way to get people to upgrade their optimized applications. I'll post a news item as a warning and provide links to appropriate v0.18 applications so that everyone has time to upgrade their applications before I make the switch. February 8, 2009 It looks like the server has been running smoothly all day, so I think we're in the clear. Thanks for everyone putting up with the downtime and lost workunits! February 7, 2009 It looks like we're having some kind of problem with the transitioner. It's running very slowly due to the fact that something else is hammering our mysql server. I'm hoping this is just because of people returning results which aren't
in the database. We're looking into the problem to try and figure out what's wrong, so things are going to be a bit slow until we figure it out. February 7, 2009 Work should be flowing now, let me know if you have any problems. February 7, 2009 I'm not generating any new work tonight, because the server is being really loaded down with all the old workunits being reported. I'm hoping by morning things should die down and I'll be able to start up work generation again. Thanks
again for bearing with us through the downtime. We're trying to fix all the problems and make sure it doesn't happen again. February 6, 2009 Sorry about the server downtime. Our mysql database got corrupted and we had to rebuild it. Unfortunately we had to clean out the workunit and result tables to get things working again, so your best bet right now is to do a reset. I'm
still working out a few things, but work should start flowing sometime this evening. February 5, 2009 Part of the problem with our server being slow is we've been crunching quite a few workunits and it's getting bogged down. I've recompiled a modified version of the db_purge which will let me purge in increments of less than a day, so
WUs will be purged every 6 hours (instead of 24). Hopefully this should decrease the load on the database and speed things up. Once the purge catches up tonight I expect we'll see a bit of an improvement in server performance. February 4, 2009 I've started 5 new searches for stripes 79, 82 and 86; each starts from a different set of parameters so we're seeing how close these get to each other. We should be using these for an upcoming publication, more on that later :) After
these are done we'll probably be focusing on stripes 20 and 21. Feburary 2, 2009 It seems that our estimate for floating point operations was quite a bit off. I've updated our estimates so this might help some of the problem people are having not being able to download new work. February 2, 2009 I've released the preliminary code for v0.18. I'm still testing it but I figured I'd get it out there for more eyes to look at. There's been a few changes in the code for multiple streams and better checkpointing, so I'd like to have
some more thorough testing of it before fully releasing it to the wild. It also has new sample workunits to test on for streams 20 and 21. Hopefully this will make the transition to v0.18 a bit smoother. It can be found in the usual spot. February 2, 2009 I've increased the in-progress workunit limit from 8 to 12, so this should help a little bit in keeping your queues filled. The newton method searches we've been running are a bit less sensitive to this than the genetic searches we were
doing so hopefully the server will be able to handle the extra load. January 28, 2009 It was a bit low compared to what you were getting before, so I've upped it to be in line with what it was. However I think we might have to do some kind of credit adjustment because the project's awarded credit as a whole is pretty high
in comparison to the number of users. January 28, 2009 I had some issues generating some workunits today when starting up some new searches using different stripes. The formula for credit also changed a bit because the new stripes have multiple streams and the function to calculate awarded
credit needed to be changed to reflect that. A couple WUs ending with l36 might calculate very fast, and some of them might have odd credit values. The problem has been fixed though so WUs generated with l38 and l39 should work nicely, without
weird credit. January 25, 2009 It looks like Dave had a problem with the v0.16 signature file for windows, so I've resigned them and updated the windows app to v0.17. Let me know if these are working. January 25, 2009 Since the v0.16 version of the application has fixed the problem with the application returning bad result, tomorrow afternoon I'll be turning back on the error checking that detects impossible results. To ensure that you will be getting
credits for all your workunits, update your optimized applications to a version compiled with v0.16. Stock users have nothing to worry about. January 25, 2009 This is tentatively speaking, but the v0.16 apps have been running all day without any errors. So I think we've fixed the checkpointing bug! Hopefully we won't have to update the application for awhile now. :) January 25, 2009 I really think we've found the last error in checkpointing this time, honestly! I've released v0.16 for osx and linux, and windows should be along the way. The source is in the code release directory. January 25, 2009 I bet another application version is just what you've all been waiting for, and here it is! (Just kidding). I've put v0.15 on the server for OSX and Linux, and as usual, Windows will be on the way when Dave gets to it. Anyways, the update
to v0.14 reduced the number of bad results from a couple in a hundred to a couple in a thousand. I'm hoping v0.15 will be the last of them. I've turned off the code that is denying credits for these bad results, because you guys are still doing
the work and it's a problem on our end. In fact if you get a bad result it helps us figure out whats going wrong. :)
The code is in the code release
directory as usual. I'd really like to thank our users (and the people making their own compiled code) for putting up with the changes this last week. Once this last issue is sorted out I think the code should remain stable for awhile as Nate
will be doing a lot of optimizations across different stripes. January 24, 2009 What it's looking like is MW is probably a bit more sensitive to floating point calculations than some other project (in fact thats almost entirely what it's doing), so the credit changes might have been a bit extreme for older
architectures. I'm upping the credit a bit to ~70% of the older level, newly generated WUs should reflect this. January 24, 2009 Dave has updated windows application to v0.14 as well. Let us know how they're performing. Januray 24, 2009 We've had a lot of performance improvements since when started with the new version of our application (v0.1), and I know this will upset some people, but even our un-optimized stock application is awarding more credit than most other
BOINC projects (even their optimized versions). So I'm reducing the awarded credit by half, which should put us in line with most BOINC projects. Feel free to complain in our forums and try to change my mind if this is too much of a change. This
should be reflected in the next set of searches we start up. January 24, 2009 I've updated the OSX and Linux apps to 0.14, and the code is in the code release directory. Dave should be updating the Windows apps when he
gets to it. These should fix the error with checkpoint which was causing apps to return invalid results. After these have a chance to get out to everyone and everything looks ok (probably Monday or Tuesday) I'll turn back on the flag
that will not award credit to widely invalid results (which was what the checkpointing error was causing). This should give everyone time to update their stock app (or a newly compiled optimized app).
On another note, this thread has more information about all the changes in v0.14. I was able to remove a multiply and divide from the inner loop of the app, as well as
some conditional statements, which resulted in a performance improvement of ~4%. January 21, 2009 There's a windows application placing 'time: X' in the result where it should be have the application name and version. This application also is returning bad results, so it will not be awarded credit. I've also noticed an issue in the
Gipsel app, however I think it might be showing up in the stock app and other optimized applications (but these are less common so it's less frequent). I think it is due to the fact that checkpointing is being done incorrectly, so I'm looking
into this problem. Hopefully we'll have a code update shortly which will fix this problem. January 21, 2009 We've made the change to the new assimilator+validator. All newly generated WUs will not award credit for applications not compiled as per previous posts. In the next few days, we should be doing another update with automatic error
checking. As we go through this process I'll post in the news which compiled applications are generating correct results, and which are generating bad results so they can be updated before the automatic error checking comes into place. January 21, 2009 The update will be tonight at 11PM EST. All WUs generated after this time will not award credit to applications not compiled with the instructions here. This is
your last chance to update your application to stock or a different recently compiled application. January 20, 2009 I'm pushing the update time until sometime tomorrow, because we'd like to have one more update to the windwos application before making the swap. It will happen sometime tomorrow so I'd get your apps updated today. January 20, 2009 The changes are in place for our new assimilator+validator, so this evening at 8pm EST we'll be making the change. The new assimilator+validator is stricter when it comes to checking for WU correctness. Workunits returned with
impossible results will be marked invalid and not awarded credit. Additionally, as per the previous news post, non-stock applications not compiled with the latest instructions detailed here, will not be awarded credit. Please update your application to the stock app, or a verified recently compiled app to prevent any loss of credit. The new
assimilator+validator contains changes which will allow us to automatically detect bad results, so when we have completed debugging this, credit will also not be awarded to applications which are not returning correct results. January 19, 2009 I've recompiled these yet again -- because people still have been getting strange performance issues. I've used the exact same compilation flags and used the same code from 0.7 (with the exclusion of the code that fixed the memory leak).
They've tested fine for me, so let me know if you're having any performance issues with v12. January 19, 2009 A significant number of hosts using optimized windows apps are returning bogus results. We're recommending our users return to using the stock app (until updated, correct versions of these optimized apps are released), because starting
tomorrow these optimized apps will NOT be granting any credit. I've released v11 of the source code, with compilation instructions that'll ensure that an optimized app will be granted credit in the future here. The new released code also has multiple test WUs to check the code against, so hopefully newer optimized versions will not return bogus results; they will also let us
track what applications are and aren't working so we are able to debug our searches and let users know which applications are working.
Unfortunately, the bogus windows apps are so prevalent that it's preventing us from implementing our dynamic WU checking; because the number of bad results is skewing our detection which would result in correct results being labeled as invalid.
We also have no way to update these apps automatically, unlike our own stock app on the server. This is why we've had to take such a drastic measure (not granting credit to any older optimized apps) until this situation is resolved. Again, our
users have until tomorrow to either revert to our stock app or upgrade to an application that uses the new compilation instructions and code, or risk not being granted any credit. January 18, 2009 We've updated the x86_64 OS X application because it still wasn't running as fast as it should be. Let us know how the new version is working. January 18, 2009 We're having a discussion about how we would like to have the milkyway code be licensed (open source of course). I was wondering if any of you had any information as to this process.
I think the thing we are most concerned about is that we want our code to be openly available for use and modification -- however if someone distributes their own binary, that code must be made available as well.
It's a little frustrating when people are releasing binaries to the general public and we have no access to the code being used to generate these binaries; so we can't put these optimizations into our own stock application so everyone can benefit
from them. It can also make debugging things a pain because we don't know exactly where our results are coming from.
I also think some of our users should be a bit concerned about this as well because they don't know exactly what they're running on their machines.
Anyways, we'd like your input and information so we can select an appropriate open source license for the milkyway code. I've made a forum post for this discussion here. January 18, 2009 Dave updated the windows binaries with the latest code, so let us know how they're running. January 18, 2009 I've updated the osx and linux binaries to 0.10, because it seemed that some of the compiler flags i had used in 0.9 were causing the application to run about twice as slow. Let me know how the new ones are doing. January 17, 2009 If you didn't notice already, the home page has been updated with publications and support information, since these have been requested by our users. Enjoy! --Travis January 17, 2009 I've made some small changes with the compiler flags re-released the new version of the linux app. Hopefully these should run a little bit faster than 0.8. Let us know
if you have any problems with these in the forums. I also made a small change in the output file that'll let me know what WUs are being returned to the stock app, because I'm
trying to debug an issue with the search and want to know if it's coming from our stock app, our algorithm, or non-stock apps. January 16, 2009 I've updated the application to version 0.8 for linux and OS X. This should fix the memory leak issue the application was having, and has a few more performance optimizations, so
it should run a bit faster. Also the problem where a few checkpoint values were not being saved to the correct accuracy has also been fixed. The new code is in the code release directory. The windows app should be updated as soon as Dave gets to it. --Travis January 15, 2009 Part of the assimilator update allows us to track users that repeatedly return invalid results and stop awarding credit to them.
If you're using a non-stock app you might want to make sure it's returning the correct results, or you might stop getting credit from it. January 15, 2009 I'll be updating the assimilator today, so work might be a little sporadic as I iron out the bugs. December 11, 2008 It's come to my attention (looking at the results that are coming in), that a few of our users are using applications that are incorrect. If you're going to compile/modify our code please check it against
the test files we've provided to make sure that it's correct. The searches we're running right now are highly sensitive to bad results, so this is pretty important. Either way I'm watching the users that are
giving back bad results, and when our validator gets updated in the next couple days these bad results will not be awarded any credit. I'm hoping this isn't malicious action on the part of these users, and rather
just a failure to double check their compiled apps. December 9, 2008 I've made a change in workunit generation that allows workunits across searches to share star files/parameter files (they weren't beforehand). The downside of this is you'll probably
be downloading new star files and parameter files because of the new change. The upside is after those star and parameter files are downloaded you won't have to download more until we start
work on different wedges. December 9, 2008 I've stopped the old assimilator and validator (they will be missed *sniff*). I apologize if anyone has a WU out there that hasn't still been crunched for the old
assimilator/validator, but it was holding steady at 5 workunits left in the database for the last 3-4 days, so I figured it was about time. December 3, 2008 [B^S] Astral Walker kindly suggested an code optimization which seems to be getting about a 10% speedup. I've updated the application again to v0.7 (at least the linux and osx
binaries). Windows should be on its way when Dave gets it on the server. I've updated the code release
directory as well. December 1, 2008 We've put the 0.6 windows binary on the server so it should be available now. Sorry for the wait. This should fix the checkpointing problems. December 1, 2008 There was a problem where the v0.6 code wasn't up to date. It's been fixed and the zip and tar files updated. November 30, 2008 There was a bug in milkyway v0.5 that was causing it to crash when it started up. I've fixed it and released the code for 0.6 and am updating the binaries. November 30, 2008 I've released the version 0.5 code to the code release directory. Note: this is just if you want to
compile the code yourself (for example, to try and come up with your own optimizations). The new application should automatically download for you if you want to use our stock app. November 30, 2008 We're starting the move to only using the new application. No new workunits will be generated for the old application (however the old assimilator and validator will keep
running until all the old workunits have been processed (so you'll still get credit for those). We'll be updating with a version 0.5 tonight which will fix the checkpoint error that's
being output (it's just cosmetic). We'll also put out a binary for FreeBSD along with that. Since we've made the source code available, we'll only be supporting binaries for FreeBSD,
Linux, OS/X and Windows, so if you're using something more obscure than that you'll have to compile it yourself because we don't have regular access to those types of systems. We've had
some discussion and the new WUs should be a bit more computationally intense, which should help with the server problems we've been having. Not to mention that moving away from the old
app and it's server side daemons will also help our server load issues. November 25, 2008 We've updated a bug or two in version 0.4, and also compiled with various optimization flags which should improve performance further. The code has been
released for this version as well. We've also compiled a binary for 32 bit linux, so let us know how that works. Also, if you've compiled your own version of the application, don't use
-ffast-math because this reduces the accuracy of the likelihood value we're trying to calculate. November 24, 2008 I've released a newer version of the code, which has some more performance improvements. Feel free to download it and any other previous code releases here. November 22, 2008 I've released the code so feel free to download it and take a look. There's some more
information about the code in the forums here. Please use the code discussion forum to post suggestions
so we can further optimize and improve the code. November 22, 2008 I've updated the code of the new app to fix the progress bar errors, so let me know how it's working (when you get version 0.2 of the app). I've also compiled an x86_64 linux
binary (and I think Dave will be compiling an i686 version), so there should be a new version (and linux versions) available whenever dave wakes up and has time to sign the binaries and
put them on the server. I'll be posting the new code online sometime today or tomorrow, and I'll update the news when it's available. November 21, 2008 I've started a few WUs for the new application. Please let us know if these are working. November 21, 2008 We've flagged the new application as a test application, so you can opt out of testing it. Default preferences are to NOT run test applications, so if you want to help us
test it you can enable this in your project preferences. November 21, 2008 We've fixed the problem with the application and have recompiled it for windows and OSX. I'll be starting up some workunits tonight (in the next few hours) and hopefully
they'll work ok. If the workunits don't fail, I'll start up the new assimilator. --Travis November 20, 2008 We've put the new version of the application up on the server, currently we only have it for windows. I'm working on compiling
the application for OSX at the moment. Afterwards I'm going to try to start a few workunits tonight and see how horribly things
crash. -Travis November 20, 2008 Nate came to us with some problems in the current application code, so we've been working on fixing that. Everything is looking right (or very close to it) at the moment, so we'll be trying to
get the new application out this afternoon (around 2pm EST). November 19, 2008 I just implemented another round of optimizations, and now on the AMD Linux cluster it's running 8x-9x faster (almost 4 times as fast as the previous version). The machine I'm testing on is really
high end, so I'm hoping that older archiectures will get even more improvement from this. Also, we're still on schedule to put the new app on the server this afternoon and start with the new
workunits. --Travis November 18, 2008 I've just finished implementing a bunch of optimizations suggested by our users in the new milkyway application (we're hoping to roll it out on wednesday). On our AMD Linux cluster it's
running about 2-3x faster. I'm currently getting some numbers for OSX and Dave is working on some numbers for windows. We should be releasing the code wednesday when we put the app on our
server. --Travis November 14, 2008 I've had to start purging results immediately (as opposed to leaving them on the server for a day) because our server has been getting overloaded with the new faster applications out there. I
also just finished the new assimilator finally, so hopefully within the next couple days the new application will be uploaded to the server and we'll be able to begin testing the new app. Dave should
post the new code to the forum once we have things working, and after that we'll start tweaking the awarded credit so we're in line with other projects -- which i know is our users favorite thing
:D --Travis November 3, 2008 After some discussion, we've decided to move the credit generated per workunit back to where it was, however reduce the maximum possible credit generated per hour in half -- this should keep
our project in line with other BOINC projects, but not penalize people not running optimized apps. These changes should effect all newly generated work so it might take ahwile for everything to
propagate out. --Travis November 2, 2008 I'm bringing the assimilator down for a bit to make some updates. Unfortuantely one of those is we're lowering the credit awarded. After talking to Dave Anderson we're generating about
twice the amount of credit per work compared to SETI and that's the benchmark we're going to be using. Especially in light of new faster applications, we're reducing the credit generated by about
half to keep things in line with other projects. We'll probably doing some more adjustment (if this turns out to be too harsh), and will definitely be doing some more when the new version of the
application comes out (hopefully this week). I'm gonna start up a search with some more expensive WUs so hopefully that will help keep the server load down and work flowing better. --Travis October 6, 2008 We know that our application code is inefficient, theres no need to be offensive or highly negative about it. The vast majority of the code was written by astronomy graduate
students, not computer scientists, whose main goal was to get an application that spits out the right numbers -- not one that did so efficiently. The whole reason we've released our
source code is so that you can help us make it faster. There's only Nate and myself working on the code, and I'm working on more of the distribution and machine learning aspects, not
going through all the legacy astronomy code (theres a lot of it and most of it neither Nate nor myself have written). So please, if you find code thats inefficient, redundant or outright wrong
please let us know instead of attacking the project and what we're trying to do here. Thats what the forums are for. Please try and work with us, instead of against us and hopefully in
a few iterations we'll have an application thats running a lot more efficiently. --Travis August 19, 2008 We've made a change to our validator which basically puts a speed limit to the amount of credit any given host can be awarded (per cpu). While this shouldn't effect
the vast majority of our users (as the speed limit is quite high), it will help prevent the project from giving out more credit than intended. August 16, 2008 The deadline has been increased to 3 days (up from 2), mainly for computers that need to go offline for weekends. Hopefully this should work well for everyone. August 15, 2008 Due to some problems with multiple processor machines, I've increased the WU queue back to 20, but I've lowered the deadline for WUs to two days. Hopefully this
will help keep
peoples machines crunching without any downtime. We're still looking into a way to get a per-core queue and hope to have that going soon. July 30, 2008 Due to the longer WU calculation time we've reduced the WU queue to 8 (from 20). Let us know if this causes any problems. June 18, 2008 I'm taking down the assimilator to let the purge get the database back to running smoothly and to let the validator catch back
up as well. After the downtime hopefully there'll be better WU availability and things will be running smoothly, as we'll be running the
purge as a daemon. June 18, 2008 The server is overloaded (yet again) so while we're generating new WUs, people are grabbing them faster than they're generated.
I'm purging the database -- and going to leave it on as a daemon, so once the database is back to a managable size hopefully it will stay
there. We're also going to look into getting new hardware for a faster server to help as well. June 12, 2008 We've started a new search to test and see if the server has been fixed, let us know how things are working. June 10, 2008 Another update on the database -- we've emailed labstaff so hopefully they'll have some answers for us tomorrow (because everything we're trying to do now is either timing out or seg
faulting). Hopefully we'll have the mess fixed up soon, and new searches up and running right after that. --Travis June 10, 2008 Hey everyone, it's Dave. We're having a great deal of trouble with the database at this point. We've been trying to create new work for you but the database keeps losing connection. This may affect the forum as well. We're working towards a fix diligently and we apologize to everyone for any time we may have wasted May 29, 2008 There have been some problems with a recent search having WUs that freeze or don't work. Currently i've shut down the server to remove the bad workunits from the database -- I'm hoping to have everything fixed sometime this
evening. March 24, 2008 Due to many complaints that the reduction in credit was a bit too severe, we've increased the generated credit a bit. It should be around 4 credit per WU
now. Please feel free to keep complaining about credit on the forums and we'll eventually get it right. :) March 23, 2008 Due to the fact that the application has been highly optimized in the last few versions, most people agree that the granted credit is way too high for the
project by a factor of 3. We've scaled credit down to keep it in line with other projects, so if you notice reduced credit this is why. March 20, 2008 I've updated the assimilator to deal with some of the performance issues we've been having with it. You should see more work availability and hopefully we wont have
results piling up that aren't assimilated. I've started up a bunch of new searches and hopefully the work will keep flowing faster than before. March 20, 2008 I just upgraded the BOINC server software. The database is a little shaky right now as we're trying to upgrade to PHP5 to facilitate it better. Therefore the forums will be
down until we can get PHP5 up, hopefully it won't take more than a day. However, the assimilator problem seems to be fixed with this release and hopefully so will the freezeups. March 19, 2008 We're running a purge on the database to try and get things sped up. During this time we wont be generating any new work, but this should be completed sometime tonight and there
will be new work after that. March 15, 2008 Good news everyone! The paper we submitted to the GECCO 2008 conference was accepted, this paper is all about results we've gotten using the genetic search/simplex hybrid on BOINC -- so we
couldn't have done it without you :) for more information on the conference: GECCO 2008.
Here's a link to our paper so you can all read it: An Asynchronous Hybrid Genetic-Simplex Search for Modeling the Milky Way Galaxy using Volunteer
Computing. This isn't the camera ready version, so we'll let you know when we have it updated with all our comments -- and of course acknowledgements to all of you who are doing so much to help our
research! March 7, 2008 We've updated the windows and 32 bit linux binaries. Hopefully this will fix the problem win2k users were having with the apps crashing. March 6, 2008 We noticed a computational error in the code, so we've updated all the binaries to 1.21. This way everyone has the same version number which makes keeping track of everything better. We've also added some
optimization to more platforms so you might notice improved performance. Let us know if you're having any problems with the new binaries. We should have a 32 bit linux version soon (this was the only platform not updated
yet). March 6, 2008 Linux binaries for 32 and 64 bit have been updated and optimized, let us know if you have any problems with 1.20 of these. March 5, 2008 We've updated windows, ppc-apple and 32 bit linux binaries to 1.19. Let us know if these work. We should be adding some new platforms soon as well. March 4, 2008 We've updated windows, ppc-apple and 32 bit linux binaries to 1.18 to address the problems people have been seeing. Please let us know if these are still causing any problems. March 4, 2008 I'm trying to roll the application back to 1.13 because 1.15 is still having problems. We'll re-update as soon as we figure out what the problem is. March 4, 2008 We've updated the applications again, hopefully these will work for everybody. Let us know if there are any problems. March 3, 2008 There was a stray printf in the code that was used to compile the binaries, which might make any of you using the 1.14 version to be seeing a bunch of computation errors. We're recompiling and signing and hope
to have a fixed version out tonight or tomorrow morning. Until then i've stopped the daemons so everyone isn't seeing a bunch of errors when they're trying to run milkyway. Sorry about the inconvenience for everyone. March 3, 2008 I've updated the applications for a few versions of the binaries, we should be adding more on wednesday. Let us know if there are any problems with the new binaries. We're hoping these should fix the progress
bug, memory leaks, and also report why windows is giving popups -- so we can fix that in the next version. February 25, 2008 Sorry for having everything so quiet but we've been working on getting everyone up to speed and a new version of the application. Right now it's looking like you can expect v1.14 on wednesday. We're just working on getting the
new windows binaries compiled (with code that should help us track down why it's giving that popup error). Additionally, I think we fixed the memory leak -- so let us know if you're still getting any of that. So hopefully we'll be able
to put the new binaries on the server without too many problems :)
On another note, we've found what we think might be a problem in how our searches are running (two of the parameters might be dependent) so we're trying a few new things on the server end to see whats going on with this. The
newer searches are using some intentionally difficult data to help us find the root of some of the problems we've been having -- so thats what you're crunching on right now. The genetic search seems to have some problems with this,
and i'm going to start up some particle swarm searches to see if they do a little better with the problem. Thanks again for all your crunching time! January 30, 2008 Sorry i've not been able to have as much time in the forums lately -- we had the camera ready version of the HCW paper due monday, and the GECCO paper due today,
so i've been quite busy writing and putting all the results you've given us down into the paper.
Hopefully the GECCO paper will be accepted and when it is, i'll give you all a link to it to read. The results are quite good - you're crunching as fast as the large
partition of the bluegene we have access to (1,024 processors). better yet, unlike the bluegene you're available 24/7 and we don't have to compete with other researchers
:D
The asynchronous genetic search/simplex hybrid is performing really well, and while not as fast as it's synchronous counterpart it is still quite good. The results also
show that by with some fine tuning we'll be able to improve the performance even more.
I've started up two new searches (gs_170..174 and gs_180..184) that further the results we have in the GECCO paper. They've refined the search a bit and are using more
points in the hybrid simplex operator we've been using.
I also hope to start some particle swarm searches up and running again shortly.
We've also been in the process of looking at some new undergraduates to work with our project to take care of the server and help with compiling binaries and that type of
thing, so hopefully that will also help clean up a bunch of the bugs we've been having.
cheers everybody, and thanks for continuing to work with us :D January 27, 2008 We've submitted a camera ready version of our HCW paper, and due to an increased page limit, we've added the first results for running the double shot genetic search on BOINC. You can access the paper here if you'd like to
read it HCW Paper and see the results of your work. The results are interesting, as they tend to suggest that slower processors aid in the explorative functions of the
search (or the global search aspects), while faster processors aid in the exploitative aspects (or local search). This is good news because it means every computer that joins counts! We again like to thank you for all your shared
CPU cycles. January 8, 2008 Our labstaff is going to be taking down milkyway for awhile to run some tests to see why it's been crashing so frequently. Hopefully it will be back soon running smoothly. January 7, 2008 It looks like there have been some more power problems with the server, which is why it's been offline. We're working our labstaff to get these problems fixed. January 3, 2008 It looks like things are running smoothly on our end, however we've had some users report that they're still having problems. Whats worked for them is a detach and reattach to the project. If you're still having
problems try giving that a shot -- if it doesn't work let me know. January 3, 2008 While I was gone, it looks like our upload and download directory permissions got messed up -- read/write/execute were removed for anyone other than root. I'm hoping this was the reason that
you couldn't upload or download files for the workunits. Please let me know if things are working correctly now. I have the assimilator/validator running manually again so i can watch to see
if this problem happens again. December 26, 2007 I'm going to be out of town until new years eve, so if i don't get back to any questions on the forum or email that's why. I've set up the assimilator and validator to run as daemons in the
config.xml now, so if the machine goes down and comes back up, these should start back automatically even if i'm still out of town -- so hopefully things will run smoothly while I'm gone. I hope
everyone is having a happy holidays!
--Travis December 19, 2007 It looks like we've found and fixed the problem with stats export and update, so as of now, stats should be updated and exported every 6 hours. December 19, 2007 For an update on the research front, Professor Carlos Varela presented the work for our eScience paper at the conference in Bangalore, India, last week, and I'm
glad to let you all know that it was awarded best paper (of 206 submissions and 60 acceptances, only 2 received best paper). Additionally, a paper we submitted to the 17th International Heterogeneity in Computing Workshop
(HCW 2008) was just accepted, which contains prelimiary results for the asynchronous genetic search we've been using. December 19, 2007 A week or so ago, the server crashed more than a few times. It seems that there was an overloaded circuit causing these crashes, and our labstaff has fixed it so hopefully things should be running much smoother
now. December 14, 2007 Some of the first convolution work units generated are granting an incorrect amount of credit (~4 credit), this has been fixed and now all newly created convolution work units should be granting the correct amount (~6.25 credit) December 14, 2007 I'm starting up some convolution searches, let me know how they run.
gs_50 ... gs_54 are running convolution
gs_40 ... gs_44 are running non-convolved
both are running the hybrid simplex/genetic search optimization method. I've stopped the particle swarm searches for the time being because they don't seem to be working very well and I'm trying to figure out why. We should
have some simulated annealing work units up and running next week.
the convolution workunits should take a lot longer (mabye 30+ minutes), but should grant the same amount of credit per time as unconvolved work units. let me know if anything is off. December 13, 2007 There was a rather big snowstorm here in the northeast and the server went down. The Internet still seems a bit weird, so please bear with us until everything is up and running correctly. December 12, 2007 We're uploading new binaries with will be able to calculate the convolution code (although they won't be doing this quite yet), please let
us know if you have any problems with the new binaries in the forums. We're hoping these should also fix some memory leak issues as well. December 11, 2007 Things have been a little quiet for awhile so here's an update as to what we've been working on:
1. Nate has committed the convolution code, and we've finished debugging it. Hopefully today there'll be some new binaries as soon as I can get Joe to take a look at it (unfortunately he's the only one that can sign them at the moment). The
convolution should take about 4-8x more time per work unit (which should make the people who want longer work units happy), because instead of calculating the fitness of the astronomy model against fixed star points, it calculates the fitness
against probabilistic distributions of star points (ie, instead of a star being a fixed point, it has a probability distribution in a range of space). Nate is going to make a post about what exactly is going on with this in the next couple
days.
2. The particle swarm searches seem to be running, however they're not performing nearly as well as I expected, so i'm looking into ways to optimize this.
3. I implemented a hybrid simplx/genetic search optimization and have been testing it out. The new genetic search work units you've been seeing are doing this. Where our previous genetic search reproduced parameters based on the double shot
approach (see our escience paper), these generate child parameters using the simplex method. This is interesting because instead of generating child parameters based on two parents, the simplex method can use N parents. Right now i'm testing 4
(the next batch will test 5) and i'll keep increasing to see what effect N has on the convergence rate.
4. I'm working on implementing a simulated annealing optimization, so if you start seeing work units starting with 'sa' it will be for that.
So that's what we've been up to in the last few days, thanks to everyone for continuing to crunch our numbers :)
--Travis December 5, 2007 The forum problem is fixed, so continue to keep updating us about where our site is breaking :) We're looking into the
PHP problems we're getting all over the website so hopefully they will be fixed soon. December 5, 2007 It looks like we're having forum troubles, we're taking a look at it. December 2, 2007 The server locked up again, so we're taking a look into why it's doing this. Hopefully it will be sorted out shortly. November 30, 2007 If you've noticed that some of the workunits start with 'ps' those are for the new particle swarm
search that we just got working (genetic search starts with 'gs'). With the last update to the
binaries the resulting work units return a lot more information about what kind of search they
were run from, when they were generated and how they were generated, so we're going to be able
to really start studying whats going on in depth now.
As to what we're going to be working on while getting these new results, the plan is:
1. fix the remaining memory leaks in the astronomy code
2. Nate is going to be updating the code with the 'convolution' which will make work units a bit longer but more accurate
3. i'm going to be implementing adaptive simulated annealing and maybe parallel tempering and stochastic tunneling search methods
4. we'll be doing a lot of data analysis about the particle swarm searches and comparing those to the asynchronous genetic search
5. there's an upcoming genetic/evolutionary algorithms conference (paper submissions due by mid january) and we're hoping to submit
the results for genetic search (and particle swarm if we have enough) to that.
thanks again everyone for crunching our numbers and putting up with all the server troubles last week. :D November 30, 2007 The server totally crashed last evening, but things should be back up and running now. November 29, 2007 After a recompile and restart, the server appears to be serving work again (finally) and you should be getting the newest binaries. November 28, 2007 For some reason work isn't being uploaded, even though there are workunits available in the system and the new binaries
seem to be recognized by the server. I've sent an email off the the boinc projects list and hopefully someone will be
able to respond and help us out. We'll also be doing a reboot of the server sometime this evening in hopes that might
help things out as well. We'll keep you posted on the news and in the forums. November 28, 2007 A user posted a question on the forums asking what we've done so far, and I figured the response might be newsworthy:
We've actually already gotten some interesting results. We've crunched through 5-10 asynchronous genetic searches (AGS) for two different sizes of volumes. We're comparing this data to how the AGS converges on the Rensselaer
grid and BlueGene (results we presented in the eScience paper linked on the main page), and hope to have another publication out this month comparing Boinc, the RPI Grid and the BlueGene for the astronomy project. Before the
server issues, the project was assimilating about 2-3 workunits a second, which is comparable to the large partition of the BlueGene we have used (1024 processors). So far what we've done is pretty promising :)
Once we get the new binaries, we'll be able to test all different kinds of searches like particle swarm, simulated annealing and some others to see which work best for the astronomy application and which scale the best and are
most resistant to the heterogeneity of BOINC, just to mention a few things. Nate should also be updating the code in a week or so with the 'convolution' which makes the calculation a lot more complicated - but more accurate.
Instead of testing the astronomy model to fixed star points, it's tested against star points that are probabilistic distributions, increasing the computation by almost an order of magnitude. How that works out will be really
interesting to see :) I've talked to Nate and hopefully he'll be posting a bit more about how the convolution works and what it'll be doing.
--Travis November 28, 2007 Awhile back some workunits got awarded massive amounts of credit due to some benchmarking issues in boinc (ie, 200,000+ credit). While we've waiting to get
new binaries signed and uploaded, I've gone through the database and removed credit from these workunits, so our toplists should be looking the way they're
supposed to. I apologize if any users/teams lost more credit then they think they deserved (send me a message if it's a problem), but I did the best I could
with my limited knowledge of MYSQL. However, with the new validator and fixed credit scheme up and running, I think things will even themselves out once
we start crunching numbers again.
--Travis November 28, 2007 After fixing the shared memory problem, it looks like the server isn't recognizing the new binaries. We'll be signing new ones tonight and putting them on the
server and hopefully this will fix the problem of not being able to download any work. As far as we know, there is work available, but the server doesn't think
there are correct binaries to upload for it. November 27, 2007 Sorry for all the errors as of late, but we've had another problem with our server. Something went wrong with the shared memory while updating the binaries and
the feeder isn't starting back up. We hope to have things working soon, and the new binaries available. November 27, 2007 We've completed updating the binaries so we should be able to do particle swarm searches and gather a lot more information about how searches are being performed.
Hopefully these new binaries will fix some of the memory leak problems and incompatibilities with linked libraries as well. November 27, 2007 We're currently in the process of updating the binaries, so there won't be any work available for the next hour or so while we get them ready. The new binaries
will allow us start performing particle swarm optimization in addition to genetic search (as well as simulated annealing and a few others when they get implemented
server side). Additionally, we'll be able to track different statistics about how these different optimization algorithms work on such a heterogeneous environment
as BOINC, and be able to use this to perform different types of optimization. November 27, 2007 There was a power outage this morning and I had to manually restart the BOINC demons after the machine was turned back on. Everything should be working again. November 27, 2007 Server problem should be fixed now and work should be availble. November 26, 2007 We've had some problems with server processes not shutting down correctly so the server needs to do a hard reboot. There might not be any work
available while this is happening. Things should be working again as soon as our labstaff gets back to us. November 26, 2007 We've updated the validator which should now be granting fixed credit depending on how much work the workunit will do (determined server side). We hope this
will fix any problem with 'cheaters' or people with benchmarks that didn't run correctly being awarded way too much credit. You might notice that credit granted
might have gone down a bit -- for the current size workunits we've made that 1.0 credit. Workunit size should be starting to go up, so this should increase as well
for longer running workunits. November 25, 2007 Sorry about the flakyness of work availability the last few days. We've been having some problems with the server but hopefully they will all be fixed shortly. November 19, 2007 Here's an update about what we're currently doing (taken from the forums):
Right now we're looking into two different things:
1. The effect of the asynchronous return time of work units on the quality of the genetic searches we're running. If you'd like to read more about the asynchronous genetic
search we're running, we have some papers we sent out recently to workshops/conferences detailing the search -- but we only had results using the new BlueGene supercomputer
here at RPI, and our local RPI Grid of clusters.
The papers can be found here:
http://www.cs.rpi.edu/~deselt/desell_genetic_ppam_2007.pdf
http://www.cs.rpi.edu/~deselt/hcw2008.pdf
What we're hoping to do with this information is determine what effect a really heterogeneous computing environment has on our asynchronous genetic search, and figure out ways
to optimize the search for these types of environments.
2. The astronomy program we're running currently calculates an integral over the wedge of the sky we're modeling. Since it's not possible to actually integrate the function
we're interested in, we have to calculate the integral the old fashioned (and expensive way) of calculating a bunch of points and calculating the area between them. The next
few searches we plan on doing will calculate the integral with varying accuracy - so expect work units of varying times (probably 8 minutes to over an hour). With these results
we'll be able to figure out what effect the integral accuracy has on the rate our genetic search converges to a solution, and what effect the integral accuracy has on the
quality of the answer.
Some other things that we have in the works is that i've been working on an asynchronous particle swarm algorithm to use (as opposed to asynchronous genetic search). For a
brief overview of what particle swarm optimization is, wikipedia isn't too shabby: http://en.wikipedia.org/wiki/Particle_swarm_optimization . We'll be able to contrast and
compare these two searches, which should be interesting.
Also, we're using the results to compare doing this work on BOINC to the BlueGene supercomputer at RPI. Who knows, with enough participants in the project -- you guys might
beat out the 7th fastest supercomputer in the world! :D (see: http://www.top500.org/lists/2007/06 )
--Travis Desell November 6, 2007 Sorry about the lack of updates, but I have just been informed that our genetic search work generator has
been tested and works. All that is needed to be done is to update the binaries and create a system for starting new
searches. The plan is to meet tomorrow, and have this up and running to get some results. We are planning on
putting out a paper soon, so we will be looking forward to see how everything works out, and I'll try to put the
paper on here, so anyone interested can check it out. October 17, 2007 We're in a bit of a dry spell right now as far as updates. It's been midterm period for all the developers
here, so we haven't had much time to work on this project. In addition, it appears that we are having some technical
difficulties with the server. The server was temporarily down, and some of the services did not reload properly on
reboot. We do have our genetic search implemented on MPI now, so this means we should be able to port that code over
to our BOINC server soon, so I will be working on fixing the server as well as getting that automated search up this
week. October 7, 2007 I just released version 1.06 of the application, and this will now run on windows unlike before. In addition
to working on windows, it also correctly handles checkpointing. This means it won't crash if you suspend it and resume
it later. The only platform that has yet to upgrade to 1.06 is mac, but that should be up tomorrow. Now everyone should
be able to get some credits. I'll do my best to get some work up on the server real soon, and we should be working on
some real problems really soon! October 4, 2007 I just wanted to let everyone know that we've got one of the project scientists checking out the boards for
now, and he has posted what exactly our project is doing. You can check it out in the MilkyWay@home Science message
board. I'll try to prod them into making a web page with all that info on there so I can put a link to it on the main
page, but at least for now you can check that out to get an idea of what we are doing. October 3, 2007 I've got all the error and warning messages I see fixed on the web page, and I've submitted new versions
of the software to the server also. In addition I added some mac binaries, so if you have a mac you should be able
to compute some stuff for us now. I've fixed the errors that have to do with CPU usage and disk usage, and the
run time should be much better for the application. There still seems to be a bug in the Windows version of the
application; something to do with BOINC thinking there is a memory leak, and the app will continue running after
100%. In response to some of the requests of the users. I believe the RSS feed is working now, so you should try
it out. You should be able to attatch to the project now, you should be able to use either http://milkyway.cs.rpi.edu/
or http://milkyway.cs.rpi.edu/milkyway/. I'm gonna try to get an automated system up in the next week or two, which
means I'll have some real work up there, and it will probably last more than 3 minutes. Those workunits were only
meant for tests. Once that is done, I'll release a new version of the app and we should be good to go! October 2, 2007 It looks like the message boards are at least functional. There are still some errors to be fixed, but
at least they are useable. This means I can start getting some feedback from you guys! Also, I found the bug
in the code, so if you have gotten work and it ran really slowly and caused your computer to do some strange things
I should be able to fix those very soon. September 29, 2007 I was just able to upload some binaries for windows_intelx86 platforms. I've been running some tests
on them all night, and it seems like the windows version is using significantly more disk space than the UNIX
versions. For the time being I will be increasing the bound on the disk so I can verify some results, so if
you're running windows on a 32 bit processor and you get some errors while running, this is due to these tests.
On the forum front, it appears that we have old html files on our server and I'm going to need to do some serious
updating there, so it may be a bit longer before we see the forums up. I tried to get them working yesterday and
today, and all I managed to do was break the site, so when we finially get things figured out for the forums,
you may see the page go down from time to time... but don't worry, that means we're working on it!!! Finally,
I granted all the claimed credit, so you should all be getting small amounts of credit, but credit to say the least.
I'm going to look into automating it. Things seem to be looking good right now, and once we have a genetic
search implemented I will put some major effort inot getting the forums up so I can get some feedback. Hopefully
things keep moving at this pace, and we will have a fully functional project up very soon! September 27, 2007 I've added some new binaries and have just realeased version 1.04 of the application. I found a small
bug, but the bigger news is that we can now support four platforms. The platforms include: amd64-unknown-freebsd,
i386-unknown-freebsd, i686-pc-linux-gnu, and sparc-sun-solaris. I realize I'm missing a few key platforms, but
thats next on the schedule. I'm also going to try to get those forums up soon, so stay tuned for that... Maybe
they'll be up this weekend, who knows? Also our research group has joined with the biology department, and we
will be releasing another BOINC project soon called L1F3@home. I will have a link for their page once it's up.
We may even try to grant bonus credits if users are registered for both projects, so keep an eye out for that too. September 20, 2007 Today we finally were able to recieve results from a client!! Currently the only platform that has bug
free code is i686-pc-linux-gnu, but look for updates when I add new platforms, it shouldn't be too long. I will
now start developing an autonomous system for processing incoming results. That's when we will start having some
real work to do, for now we are still in testing stages. Thanks for your continuing support and patients. September 16, 2007 This is just a warning that if you have joined this project, there are still problems with the program
and we are still technically in the early stages of development. We hope to have some real work up on the server
soon, but as of right now things are still a little broken. I will do my best to keep up withrequests and needs.
I will post again when we have things working, and once that's finished I will seriously get to some of your
requests. September 6, 2007 I notice there are some concerns about forums and the like, but unfortunately I'm the only person
working on this project and I am only a college undergrad doing some research on the side. I appreciate your
interest in this project, and also your commitment to keep things respectable while I am away. Currently I'm
working on fixing some bugs in the actual program, and once I get something working I will add binaries for new
platforms and update the web interface. Hopefully I can get everything running smoothly, and I hope to have
project working correctly sometime this weekend or during next week. Once that is finished I will get right
on top of web stuff. Thanks for your patients!! August 28, 2007 I have a couple more updates for you today. There is now some work on the server, but as of right now,
the upload directory is not readable or writable by users that are outside of our network here, so clients can't
upload their results. I have created binaries for three platforms: i386-unknown-freebsd, i686-pc-linux-gnu, and
sparc-sun-solaris. If you are connected to the project, but are getting errors about having an outdated signature
file, then you need to detach from the project then reattach; there was a small error in our code signing procedure
which is now fixed. Lastly, we are having a little bit of trouble with clients reading from input files. I will
be working on a solution, but until then the results will be insignificant. Thank you for your participation and
your patience, and hopefully we will have a working product up soon. August 26, 2007 I just wanted to update everyone, and show that this project hasn't died. First of all, we just got
a brand new server machine that will be dedicated to our project. Everything has been loaded onto this machine,
but we still have a couple of bugs to iron out a couple of bugs. We should have new work to be done in the next
couple of days. July 8, 2007 A new web interface is up for Milkyway@home July 7, 2007 Milkyway@home project is up and running, and account creation is now enabled