Welcome to MilkyWay@home

Posts by Tom Donlon

1) Message boards : Number crunching : Delay in getting new work units untill all work units have cleared (Message 69147)
Posted 1 Oct 2019 by ProfileTom Donlon
Post:
Apologies, I thought the problem had been resolved. I'm looking into the problem and will hopefully have a solution soon (or at least an explanation).

- Tom
2) Message boards : Number crunching : Delay in getting new work units untill all work units have cleared (Message 69131)
Posted 27 Sep 2019 by ProfileTom Donlon
Post:
We have been monitoring the situation, and it seems like the community has found fixes to some of the problems you are experiencing.

Jake said that the problem appeared to be some obscure BOINC setting somewhere, and had asked BOINC forums about it. It looks like this issue disappears in the new beta of a BOINC client, so they must have patched whatever was causing problems. When that is released, hopefully the problem will be resolved.

- Tom
3) Message boards : News : New Separation Testing Runs (Message 69118)
Posted 24 Sep 2019 by ProfileTom Donlon
Post:
This may be a good thing. Our test runs are known to be searching for the incorrect number of parameters. If our test runs look different than our real runs, then it could be a sign that our actual runs are searching the correct number of parameters. This is just a guess, though, it could be a number of other things.

Thanks,
Tom
4) Message boards : Number crunching : Resent Invalid and Validate errors de_modfit_85_bundle4_4s_south4s_1_ (Message 69021)
Posted 10 Sep 2019 by ProfileTom Donlon
Post:
Stripes 84 and 85 were taken down. These are validations of older tasks, not new tasks. Workunits generally take up to a week or two to finally stop trickling in validations.

Tom
5) Message boards : News : UPDATE: NO Scheduled Downtime Thursday 9/12/19 Evening (Message 69017)
Posted 9 Sep 2019 by ProfileTom Donlon
Post:
CORRECTION:

There will not be a scheduled downtime for the main server. Sorry for the false notice.

Tom
6) Message boards : News : UPDATE: NO Scheduled Downtime Thursday 9/12/19 Evening (Message 69015)
Posted 9 Sep 2019 by ProfileTom Donlon
Post:
Hi everyone,

The Milkyway server will be down this Thursday, 9/12/2019 starting at 4PM. The server should be back up by midnight. You will not be able to download or return workunits during this time, and the website will also be down. Any issues/questions can be directed at our email milkyway@cs.rpi.edu while the server is down.

- Tom
7) Message boards : News : New Separation Runs [UPDATE] (Message 69005)
Posted 4 Sep 2019 by ProfileTom Donlon
Post:
Stripes 84 and 85 have been taken down. Sorry for the delay, I was trying to get as much optimization out of the runs as possible before stopping them, as they had not yet converged.

Tom
8) Message boards : News : New Separation Runs [UPDATE] (Message 68991)
Posted 27 Aug 2019 by ProfileTom Donlon
Post:
Hi everyone,

Stripes 84 and 85 are beginning to return invalids, as was expected. I will pull them down in a couple days when I am back in the lab.

Best,
Tom
9) Message boards : MilkyWay@home Science : How do you consider this article with your science project (Message 68990)
Posted 27 Aug 2019 by ProfileTom Donlon
Post:
Hi Diego,

Our research mostly involves galactic structure (specifically, Milky Way substructure) using large-scale optimization methods. Machine learning is becoming more and more prevalent in physics and science as a whole, and while we are not planning on incorporating machine learning in MW@h anytime soon, people are already using machine learning algorithms to locate and classify Milky Way substructure.

I think the astronomy community is still figuring out what machine learning can do, and working out how to implement that into important projects often takes a significant amount of time.

This is an interesting article though, thanks for sharing!

Tom
10) Message boards : Number crunching : Computation errors (Message 68938)
Posted 30 Jul 2019 by ProfileTom Donlon
Post:
robertmiles,

Did the workunits actually abort computation, or did they just output that error message? That's a typical output from the separation application that basically just says "you aren't using a lua file so I'm defaulting back to this parameter file". When I test these runs on my client I get that "error" every time because we don't use separation that way anymore.

- Tom
11) Message boards : News : New Separation Testing Runs (Message 68937)
Posted 29 Jul 2019 by ProfileTom Donlon
Post:
If you are not interested in the specifics behind the reasoning for these tests or the project specifics, please feel free to skip this comment. This comment is not a news post, so you aren't missing anything crucial to the project.

I've learned that many of the volunteers for our project are not familiar with what the project actually does behind the scenes, so I'll take a brief moment to describe what the Separation Application is, and what it does.

Milkyway@home has two projects: The first, original project is the Separation Application. The second project is N-body (which I will not be discussing here, but feel free to ask Eric about specifics if you are interested). The Separation Application is designed to take in large amounts of Main Sequence Turnoff Stars (stars that are transitioning out of the stage of "life" our Sun is in to become giant stars) and use this data to find substructure within our Milky Way. This is done using Sloan Digital Sky Survey and PanStarrs data.

The idea is that the Milky Way has many stellar streams in it. These streams form from dwarf galaxies falling into the Milky Way, and getting stretched out due to the same types of forces that cause tides on Earth. These streams are very sensitive indicators of the gravitational fields that our Galaxy produces. By locating and analyzing these streams, it is possible to extract information about the gravitational forces in different regions of the Milky Way. This information can then be used to determine the where dark matter is located within the Milky Way, since dark matter rich regions should produce much stronger gravitational forces than we expect just from looking at how much light matter is there. By finding the dark matter, we are that much closer to figuring out what dark matter is.

The Separation Application attempts to fit a number of stream models (cylinders, basically) to a background Milky Way model in a small stripe of the sky. It then builds a model of where we expect stars to be in the region based on this input, and compares that to the observed data we get from large-scale surveys. This is done through hundreds of thousands of trial and error attempts, where better attempts are used to generate the next generation of attempts (see differential evolution, if you're interested in this process). When you run a workunit for the Separation Application, you are performing the calculations for one of these trials.

The results that we get from a run of the Separation Application should tell us where it is most likely that these streams are located in the data stripe. By combining data from many stripes, you are able to follow streams across larger portions of the sky. For the most part, these results appear to look like streams that we see in the sky. In some cases, however, some of the stream results from a stripe will not look like a stream due to the way that the optimizer fit it. Instead of throwing out this data, I intend to see whether or not the optimizer is trying to "tell" us something -- that is, maybe there isn't a stream in that location, but there is something else, and we just don't understand the results well enough to say what that is yet. Maybe the optimizer is doing its best to fit streams that exist in real life, but are outside the range of the data that we are using. Maybe the optimizer returns strange results when there are only 3 streams in the sky but you try to fit 4 model streams, or vice versa. The goal of these tests is to figure out what inputs return what outputs from the optimizer. This will allow us to more confidently interpret the results of the optimizer, and maybe make even more discoveries than we would have otherwise!

I'm more than happy to answer any questions that you all have. If you want me to explain anything in greater detail, let me know and I can absolutely do that. [/url]
12) Message boards : News : New Separation Testing Runs (Message 68936)
Posted 29 Jul 2019 by ProfileTom Donlon
Post:
Hi Everyone,

Looking at all of the previous runs (including those from previous projects), there are patterns of interesting and unexpected behavior from the optimization results. In order to better understand these results, I have planned a series of extensive testing runs. These tests will allow us to learn how the optimization routine's responds to specific circumstances and inputs. I will make a comment attached to this post describing the tests & reasons for the tests in greater detail if anyone is interested.

EVERYTHING IS WORKING AS INTENDED. These results are not entirely understood, not incorrect. I don't expect to invalidate any results from this project, but I do plan on actually producing more, better results as I learn why the optimizer outputs what it does.

I am rolling out the first 2 series of tests, and have released the runs on the server.

The names of these runs are:
de_modfit_14_bundle5_testing_4s3f_1
de_modfit_14_bundle5_testing_4s3f_2
de_modfit_14_bundle5_testing_4s3f_3
de_modfit_14_bundle4_testing_3s4f_1
de_modfit_14_bundle4_testing_3s4f_2
de_modfit_14_bundle4_testing_3s4f_3


Please let me know if you experience any problems with these runs. Thank you all for your help with this project!

Best,
Tom
13) Message boards : News : New Separation Runs [UPDATE] (Message 68928)
Posted 26 Jul 2019 by ProfileTom Donlon
Post:
This looks like it is still from the old bugged runs. It is a "...bgset" run, and the new runs will have 26 parameters, not 25. Can you clear your queue and see if that helps?

Tom
14) Message boards : News : New Separation Runs [UPDATE] (Message 68923)
Posted 25 Jul 2019 by ProfileTom Donlon
Post:
Hi Marsinph,

Can you describe the error, or post an error log here so I can be more helpful?

Tom
15) Message boards : Number crunching : Computation errors (Message 68922)
Posted 25 Jul 2019 by ProfileTom Donlon
Post:
Hey all,

There was a problem with a flag in the parameters for the runs last night. I fixed it around 10 PM. You may still get some bad runs as your queues purge, but after a day or two at most things should continue to run smoothly.

Tom
16) Message boards : News : New Separation Runs [UPDATE] (Message 68911)
Posted 25 Jul 2019 by ProfileTom Donlon
Post:
Is this from a "...bgset" run, or a "...bgset_2" run? The new "...bgset_2" runs ran fine on my client. This looks identical to the last error for the previous bugged runs. Thank you for the input, though!

Tom
17) Message boards : News : New Separation Runs [UPDATE] (Message 68908)
Posted 25 Jul 2019 by ProfileTom Donlon
Post:
I have updated runs and taken down the bugged ones, waiting to make sure that they are returning results. I'll update the main post once I make sure that these runs are functioning correctly.

If you have errors with runs ending in "...bgset_2", then I would appreciate seeing your error messages.

Sorry for the inconvenience!
18) Message boards : News : New Separation Runs [UPDATE] (Message 68906)
Posted 24 Jul 2019 by ProfileTom Donlon
Post:
Thanks for bringing that to my attention, none of the runs are returning data. I'll try to fix that as soon as I can.

Tom
19) Message boards : News : New Separation Runs [UPDATE] (Message 68904)
Posted 24 Jul 2019 by ProfileTom Donlon
Post:
Hi Everyone,

I just put some new Separation runs up on the server and took down the old ones. You may still see new workunits from old runs in your queues for a few days as those runs finish validating.

The names of the new runs are:
de_modfit_80_bundle4_4s_south4s_bgset
de_modfit_81_bundle4_4s_south4s_bgset
de_modfit_82_bundle4_4s_south4s_bgset
de_modfit_83_bundle4_4s_south4s_bgset
de_modfit_84_bundle4_4s_south4s_bgset
de_modfit_85_bundle4_4s_south4s_bgset
de_modfit_86_bundle4_4s_south4s_bgset


An error processing a flag in the parameter files has been fixed and updated runs have been released. These runs are confirmed to be returning results as of 10 PM on 7/24. I apologize for the inconvenience.

The names of the new runs are:
de_modfit_80_bundle4_4s_south4s_bgset_2
de_modfit_81_bundle4_4s_south4s_bgset_2
de_modfit_82_bundle4_4s_south4s_bgset_2
de_modfit_83_bundle4_4s_south4s_bgset_2
de_modfit_84_bundle4_4s_south4s_bgset_2
de_modfit_85_bundle4_4s_south4s_bgset_2
de_modfit_86_bundle4_4s_south4s_bgset_2


As these runs optimize we may see increased invalidated returns with the stripe 84 and 85 runs. This is a known issue that I believe has something to do with a data cut made in those stripes. If/when this starts happening and the stripes are sufficiently optimized, I will take them down so you all don't have to worry about crunching invalidated WUs.

If you have any questions/comments/concerns, please feel free to post them here. Thanks for all your support!

Best,
Tom
20) Questions and Answers : Web site : create or edit personal profile wont work (Message 68900)
Posted 16 Jul 2019 by ProfileTom Donlon
Post:
Hi all,

Profile editing and creation is currently down. When we migrated the server over to the new machine in May we updated Boinc to a 1.x version, which disabled most of the things we had changed in the website code. We are aware of the problems and are working on fixing them.

- Tom


Next 20

©2020 Astroinformatics Group