Posts by Nathan
|
1)
Message boards :
Number crunching :
8 Workunit limit
(Message 4721)
Posted 15 Aug 2008 by  Nathan
Post: I'm thinking that a good time to shoot for would be something that on average will be around the 373* series. However, I don't know if there will ever be a set length. As Travis brought up, it really depends on what part of the sky we're looking at and how complicated it is. I've been really busy the past couple weeks making the code more robust to deal with these more complicated pieces of the sky. The problem is that with each complication comes a significant increase in runtime... For example, if there is two pieces of tidal debris in the piece of sky we're looking at it increases the runtime by 75%, also if there happens to be a piece of the sky that has bad data or something that we need to ignore that also increases the runtime by some nontrivial amount (I know it seems backwards that if we need to ignore something that the runtime gets longer, but we have to calculate the effects of it). So as you can see, the runtime can ramp up very quickly. To keep the WUs similar lengths we'll need to be scaling the number of integral points by an amount that we think will compensate the other changes, but obviously it will not be an exact science. However, I think I can say that we'll do our best to keep them around the length of the 373* series, and that there should never be anything as long as the 372* series, no anything as short as the 371* series. Unfortunately, I don't think I can be more specific than this. |
|
2)
Message boards :
MilkyWay@home Science :
Goal
(Message 4686)
Posted 14 Aug 2008 by Nathan
Post: Milkyway@home is based at the Rensselaer Computer Science Department. This particular project is being developed to better understand the power of volunteer computer resources. The primary goal of Milkyway@home is to study tidal debris through the use of a maximum likelihood technique that utilizes volunteer computing. However, the goal is also to understand the power of volunteer computing. This latter part is possible because we have developed the code such that it can be ran on BOINC, on a supercomputer, and on a cluster of machines. Therefore, we can easily compare these results and determine a good measure of how the resources compare. Therefore, the goal is truly two-fold. |
|
3)
Message boards :
Number crunching :
New WU Length?
(Message 4424)
Posted 23 Jul 2008 by Nathan
Post: Has the MilkyWay work units been extended again? No they have not been extended again. The 3720282 and 3721282 are the long WUs from when I first increased it, but after that they have only been shortened. The 3730382 and 3731382 should run between half and a quarter of the time of those above. |
|
4)
Message boards :
Number crunching :
Guidence from Project Team Requested
(Message 4421)
Posted 23 Jul 2008 by Nathan
Post: I can attest that the science has not taken a backseat to the server, however, the science cannot get done without the server performing well. So without catering to the needs of the community and the server (at least with our current hardware) there would be no science getting done. Yes, I will full well admit that I increased the times by too much with the first jump. However, I wasn't sure how much was a good amount. I chose the "go big and then scale back" approach as opposed to the "scale up slowly" approach. Was it correct, maybe not; however, it instantly fixed a lot of the server issues. Thus, it was not all for not. As to the question, of do we get 30-60 times the science by increasing the length by 30-60. That is an entirely subjective question, and the only real answer I can give is that from what we've seen, a factor of 8 increase gives approximately one digit added precision, so the initial WU increase (the 372... series) would give us about 2 more digits of precision to all the calculations. Is this worth it? I don't know yet from the science aspect, but as with any instrument it takes a lot of calibration and fine-tuning to get it working the way you need it to. Currently, I'm still doing a lot of this calibration, so the science can get done. |
|
5)
Message boards :
MilkyWay@home Science :
New Math Formula
(Message 4420)
Posted 23 Jul 2008 by Nathan
Post: Well you just need to get your flux capacitor calibrated. Yeah, there are some timings that need ironed out with WU number and deadlines, we're working on that. As for the math used to calculate the WU work: the length of the WUs are almost entirely determined by a numerical integral within the code. Basically what happens is that we take the volume that we are studying and subdivide it into many smaller volumes and calculate what the probabilities are for the spheroid and the stream within each sub-volume and add them up to get a total integral value for the big volume. The number of subdivisions that we use effectively controls the run-time of the algorithm. The actual math is determined by the models of our stream and spheroid (which can be found in the paper here) and the shape of the volume we are looking at (in this case a wedge, so we create a number of small wedges in which the volume can be calculated using trigonometry). We multiply the subvolumes by probabilities and sum all these values to get our total integral. |
|
6)
Message boards :
Number crunching :
Guidence from Project Team Requested
(Message 4217)
Posted 17 Jul 2008 by Nathan
Post: After seeing some of the times, I'm thinking that the "medium" length ones will probably work out the best for everyone, so figure on timings roughly around what you're getting with the 373... series. As for the rest, I got my times wrong and Travis is flying back today, so tonight or tomorrow we should have some of the timing issues fixed. I need to talk to him and confirm things, but what I've been thinking is upping the deadline to about a week (this way we get results back on a reasonable schedule) and reducing the number of max work units (so you can actually finish all that you get). If this sounds unreasonable in any way please let us know. Also if you can suggest a good number of WUs given the new runtimes associated with teh 373 series, it would be very helpful. |
|
7)
Message boards :
MilkyWay@home Science :
New figure!
(Message 4214)
Posted 17 Jul 2008 by Nathan
Post: Excellent! I'm going to give a short presentation in about 5 weeks to my astronomers' league and THAT is a graphic and explanation that I can use that should make it pretty clear to just about anyone what you're doing! That's awesome! Great! |
|
8)
Message boards :
Number crunching :
New WU Length?
(Message 4196)
Posted 16 Jul 2008 by Nathan
Post: Adding to what Thunder said.....Nate has already said that Travis will increase the deadlines soon,so these short deadlines won't last long ;) Yeah, we'll be making some changes to the deadlines and number of WUs downloaded at a time ASAP. We'll just have to deal with it until then, sorry. |
|
9)
Message boards :
MilkyWay@home Science :
New figure!
(Message 4195)
Posted 16 Jul 2008 by Nathan
Post: So I just had to make a new figure for a newsletter and figured I'd share the results with you guys as it gives a good description of what we're working on. So what you see below is a plot of the plane of the Sagittarius dwarf orbit which is almost perpendicular to the Milky Way's disk. An image of an edge on disk galaxy was taken from the SDSS image archive and used to simulate the Milky Way. The coloration has been slightly altered to make it stand out more, but the dust lane can still be seen as the dark band that obscures the core of the galaxy. The Sun's position is marked with the green star. The white points are from a simulation modeling the disruption of the Sagittarius (Sgr) dwarf galaxy with the current position of the core indicated. The red dashed line indicates the directionality of the orbit. The cyan (blue) points are a subsampling of the stars we have found to be part of the Sgr tidal stream, while the magenta (pink) arrows indicate the directionality of the stream we found by analyzing the SDSS data. We have currently analyzed 2 wedges thus we have points and directions for those two volumes. The goal of this project, at least at current, is to be able to make all the white areas covered with cyan points such that we don't have a simulation but real data that defines the tidal disruption. That's where you guys come in; we need your help in analyzing all that area!  |
|
10)
Message boards :
MilkyWay@home Science :
What's on the agenda?
(Message 4194)
Posted 16 Jul 2008 by Nathan
Post: Wow! You certainly did! Heck, I'm so happy to get such a fast and thorough response from a BOINC project scientist that I got off my rear-end and attached 3 more computers and raised my resource share for MW! :) Thank you! You also caused me to go out looking for more information about BHB stars. If I'm interpreting what I read and the H-R diagram correctly, then they make a better standard candle since their absolute magnitude tends to very tightly correlate to their color temperature? (I know this is a bit off topic since you're not actually using them) Yes I'm now much clearer on the reason for F turnoffs. I sort of suspected that one reason might be how numerous they are, but the diagram really helped me see the gaussian distribution of their magnitudes, so I'm sure clearer on that aspect of the algorithm. I found the differences in the metallicity of the spheroid v disks very interesting. You alluded to that difference on page 11, but I didn't make the connection of WHY the spheroid stars were bluer. Glad to have cleared that up. :-) Now I'm wondering all sorts of things about evolutionary differences between spirals and other galaxies.... why more metal rich in the disk? any connection to past mergers in the formation of the disk v spheroid, etc. ? (don't feel the need to answer this stuff... I'm just musing in type) :) The disk is more metal rich because that's where all (read that all means most) the star formation happens because it's where all the dust/gas is. Therefore stars are born and die within the disk and when they die (and to some extent during their lifetimes) they eject material (most notably during supernovae) which is comprised of all of the heavier elements they made during the fusion process. Therefore when a new star is formed using this material they have a higher initial metalicity. Lather, rinse, repeat. :-) The formation of the spheroid is a topic of hot debate. Some say that the spheroid is entirely composed of past mergers, but at any rate there is ton of debris in the spheroid from many different things (3 that have been found are assumed to be dwarf galaxy mergers). As for the evolutionary differences in galaxies, I can't help you much. I am a Galactic astronomer, not an extra-Galactic astronomer; so my knowledge of the subject is not very verbose. I did want to make one clarification for anyone else reading, lest they think M55 is a galaxy. (You did make me reach for my lone Messier book to make sure my memory hadn't failed me) M55 is a visually wonderful globular cluster in Sagittarius. One of the few that really look like an obvious star cluster in even a pair of binoculars! Tonight would be a terrible night to see it since the moon is too near, but anyone with any decent pair of binoculars can find it pretty easily in about a week. Go out before the moon rises and find the "teapot" shape of Sagittarius (for help visualizing it without spending any $$'s I recommend a free open source program called stellarium (www.stellarium.org). Look for the two stars that make the 'handle' of the teapot and look in a line going south from them about 2x the distance between them and you should see a fuzzy patch. Spend some time looking at it and you'll start to make out the fact that the "grainy" appearance is actually clumps of stars within a large cluster of them. You won't see any of the color variations that the diagram Nathan provided indicates without a decent sized telescope (sadly our eyes need a LOT of light to distinguish color), but it's still pretty cool! Enjoy! I apologize for saying it was a galaxy, it is indeed a globular cluster. Thunder gives a good description of how to find it in the sky. I would like to add another nice little trick you can perform: if you are able to see Sagittarius (the teapot described above) and you put your arm out and with a closed fist over the teapot, the amount of area taken up by your fist is approximately the amount of area on the sky that the core of the Sagittarius dwarf galaxy takes up on the sky. That's how close it is to us!!! The only reason that you can't "see" it and that it took until 1994 to discover is that you are also looking straight through the disk of the Milky Way, as the dwarf is on the opposing side from us. |
|
11)
Message boards :
Number crunching :
New WU Length?
(Message 4145)
Posted 15 Jul 2008 by Nathan
Post: Sorry for all the confusion, I did make a post about the increased time for WUs when I did it, but I realize that "at the same time" wasn't enough time for some of you guys to realize what was going on. Thanks to those that pointed the others in the right direction. The answer to the big question is YES, the deadlines will be changed. The only reason it hasn't been done yet is that I do not know how, and Travis is out of town until today or tomorrow. However, it will be fixed ASAP. As for all the credit issues, my personal belief is that there should be a standard BOINC implementation that gives 1 credit per minute crunched. Simple sweet and no worrying by the project admins if there is too much or too little. But, that's just my opinion. However, I do understand why it's not universal and, so we deal with it. |
|
12)
Message boards :
Number crunching :
Hopefully...
(Message 4144)
Posted 15 Jul 2008 by Nathan
Post: Again, thanks for all the feedback guys. I do apologize for the suddenness of the longer WUs, but weighing all the problems with the server and the lack of work I figured it was worth the risk. As has been mentioned the server is performing much better than before, which was a big problem when you couldn't even connect to the site for how bogged down it was. As was commented on by someone above, we seem to have found a happy place with the length of the WUs now, so there shouldn't be any more "big" instances in time change like this. To address a couple of the comments above: yes the science will benefit, we will gets increased accuracy within all of our numerical calculations. Yes slower machines still contribute a lot to the project. Some tests have shown that even the slowest machines have a relatively high chance of their work bettering the population thereby improving the results. The quickness of the WUs past the 50% mark I believe is phenomenon associated with the code: I believe that the first 50% is the calculation of the integral and the other half of the WU is the calculation of the actual likelihood given the data (the final result of your crunching): the problem with this is that there are about 100,000,000 calculations in the integral now, and... about 100,000 for the data. You can see the speed issue here. Again, this is just my theory given what I've seen and the structure of the code, but it seems to fit. We'll get the delays and deadlines fixed as soon as I'm able to talk to Travis. Probably, tomorrow (Wednesday). Keep up the good work guys! |
|
13)
Message boards :
MilkyWay@home Science :
What's on the agenda?
(Message 4142)
Posted 15 Jul 2008 by Nathan
Post: I'm working my way down through the paper, but please bear in mind that I'm only an amateur astronomer and have spent most of my "serious" research in astrometry of NEOs. I've thusfar left anything stellar to you pros! ;) I'm starting to dabble in stellar photometry, but my understanding is still on the "low" side. Happy to hear your trying to work you way through it and understanding it so well it sounds. I'll answer your questions below. You refer to F turnoff stars as "not a very good standard candle", but I assume you mean in respect to the precise candles like cepheid and rr lyrae variables. Exactly! Is it that F turnoff stars have a relatively flat absolute magnitude or is it just that they are far more abundant in the Sgr dwarf tidal stream? The reason they are not a good "standard candle" can be seen if you look at this picture  . It is an H-R diagram of the galaxy M55, made by NASA, and it gives a relation of the Luminosity or Magnitude (on the y-axis) versus the Temperature or Color (on the x-axis) of stars. This one being that of all the stars we can see in M55. If you look between the green bars you see what would be considered F turnoff stars. You should also easily see that the magnitude range that they populate on the diagram is relatively spread out but seems to follow a somewhat Gaussian shape. This Gaussian is what we attempt to model with the Absolute Magnitude distribution convolution you see defined in the paper. To add to this a commonly used standard candle is Blue Horizontal Branch (BHB) stars, as they have a relatively constant magnitude. You can see these on the image at Mv ~ 1 and in the range of 0 < B-V < .25. You can see how different these distributions are. Is it that you expect them to more clearly define the tidal stream in relation to the galactic spheroid? Using more than one type of star would be very difficult as it would mean that we need to be able to clearly define what type of star each star is in our data set and be able to dynamically determine what its absolute magnitude is while simultaneously fitting it with our algorithm. Quite frankly this is not possible yet. One of the hardest parts to doing this is that you cannot simply look at the color and determine the type of star. Notice on the image that the farther right you go you get two distinct branches: the main sequence stars on the lower and the giants on the upper. Given just the color of the star how do you then determine which branch it is on? There is some work being done to try and characterize the distributions of stars within an H-R diagram, but it is slow going. Eventually, we would like to give it an entire data set of all stars and fit that way, but not yet. Why then, you may ask, do we use F turnoff stars instead of the better candle BHB stars? This is quite simply numbers. There are a LOT of F turnoff stars compared to the relatively few BHB stars, and given we are doing probabilistic searches over these stars, the more stars we have the better (and more accurate) our results. Therefore, we use the F turnoff stars. However, BHB stars are visible at a much larger distance than F turnoff stars, so in the future we may need to use the BHB stars to fit the more distance pieces of the Sgr stream, as well as other debris. One final reason we use F-turnoff stars is that the spheroid of the Milkyway is comprised of an older metal-poorer population than the disks. This means that the stars in the halo are bluer (shifted to the right of an H-R diagram) than the Milkyway. Thus, by being clever with the color range of stars we fit, we are able to create a sample that has very little contamination (if any) from the disks of the Milkyway and therefore are able to avoid modeling them. This is because the spheroid's F turnoff stars are shifted further to the left than the rest of the Milkyway's F turnoff stars. I hope I answered all your questions, and please don't be afraid to keep asking more! |
|
14)
Message boards :
Number crunching :
Hopefully...
(Message 4088)
Posted 14 Jul 2008 by Nathan
Post: Thanks for all the great feedback guys. Unfortunately, I don't know how to increase deadlines and whatnot, and Travis is at a conference until Tuesday. But we'll get this taken care of then. So the thoughts are a little split, do you guys like this length of WU or do you want to see the time dropped back a little bit? It seems like it's a really good time on the good machines, but the slower machines seem to be suffering. So my question to you is: if I reduce the time how much do you want it reduced? Until then, I have created a new search 3730382 with slightly shorter WUs should be a little less than half the time. Let me know if this works out better. |
|
15)
Message boards :
Number crunching :
No new WU
(Message 4087)
Posted 14 Jul 2008 by Nathan
Post: Lomger? I'll say. 17,000+ seconds on my Q9450 at 3.0Ghz!!!! Thanks for the quick feedback! I'll cut this down, I had a feeling it'd be a little overkill after I had started it. Travis is a conference for the next couple days, so I can't change the deadlines or anything until I talk to him, as I'm not sure how to do this. |
|
16)
Message boards :
Number crunching :
Hopefully...
(Message 4064)
Posted 14 Jul 2008 by Nathan
Post: I've started 2 new searches, 3720282 and 3721282, in response to there being no new work; I don't quite understand yet, ho we ran out of work, but c'est la vie. I've increased the length of the WUs in an effort to ease the server traffic. I hope that this fixes a lot of the assimilator and traffic issues. Be sure to let me know if it works! As a heads up, I increased the time by increasing the accuracy with which we do the integral calculation over the wedge volume. It's possible that I made it too dramatic and increased the runtime by too much, so please let me know what completion times you are getting so I can adjust the next searches accordingly. I will also bring up the potential for needing a hardware upgrade to deal with the increase in traffic. I find this a good thing though as it means that we have all of you guys out there working with us. So I thank you for making this project such a success and I hope that we can get these few issues smoothed out for you as soon as possible. |
|
17)
Message boards :
Number crunching :
Since MW keeps acting funny lets all take a guess at why...
(Message 4063)
Posted 14 Jul 2008 by Nathan
Post: I believe we may be reaching our hardware limit as well and will be bringing it up in the future. Also, I have increased the WU times in hopes this alleviates some of our problems. |
|
18)
Message boards :
Number crunching :
No new WU
(Message 4062)
Posted 14 Jul 2008 by Nathan
Post: I'm not sure what's going on that you guys are saying there's no work at the moment. I have searches running and I don't think that all the WUs went out already, but I guess it's possible. I'm starting two new searches (gs_3720282 and gs_3721282) in hopes it fixes the problem. In response to the longer WU problem, I give you WUs much longer than what you've been experiencing. I'm not quite sure how much longer, so let me know how they are performing; I don't want you crunching so slow it's disheartening now. :) As far as a faster/better/stronger/improved server... I believe we are going to have to talk about this... the last I checked we had 16,000+ machines working for us and we may have reached our capacity with our current setup. No promises, but I will definitely bring this up at our next meeting. Hope this addresses your concerns and the problems. |
|
19)
Message boards :
Cafe MilkyWay :
SETI@home in trouble
(Message 4025)
Posted 9 Jul 2008 by Nathan
Post: I'm sure a lot of you guys have already heard about this, but SETI@home maybe in trouble. The Arecibo Observatory, home of the radio telescope that the SETI data comes from, is probably going to have to make massive budget cuts, and one of the programs cut will be SETI@home. You can find a small article on it here: http://www.g4tv.com/thefeed/blog/post/686804/Help_Save_SETI.html?utm_source=g4tv&utm_medium=rssfeeds&utm_campaign=TheFeed I'm sure you can find a more formal announcement somewhere on the SETI@home or Arecibo observatory site, though. Figured anyone that hadn't heard about this would be interested/concerned and should know what might happen in the near future. |
|
20)
Message boards :
Number crunching :
New Searches
(Message 4024)
Posted 9 Jul 2008 by Nathan
Post: Just added 3710182 and 3711182. Still trying to find the best settings, to use; but I've got it down to at least reasonable answers right now. Sorry for the downtime guys, I got sick and wasn't able to start more searches after the weekend. |
Next 20
©2024 Astroinformatics Group