Welcome to MilkyWay@home

N-Body 1.18

Message boards : News : N-Body 1.18
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58779 - Posted: 12 Jun 2013, 11:04:26 UTC

I've now watched an N-Body task through being pre-empted by BOINC to allow other projects' tasks to run.

During suspension (with the app left in memory), one thread - perhaps a watchdog or timer thread - continued to show a Cycles Delta ~300K (trivial), but the worker threads were completely dormant.

When BOINC gave the task the resume instruction, all four (in my case) worker threads resumed, with the normal 2 billion Cycles Delta.

IMHO, the application's internal multi-threading behaviour is working properly (though I'll try to watch the final wrap-up at the end of this 100k_chisq_alt task, because I suspect the threads may finish out of synch, and wind down CPU usage to a single 'housekeeper' thread in the final stages.)

I think the remaining issues are the initial runtime estimates for the tasks, and some rough edges in BOINC's scheduling of MT tasks: that hasn't been given much operational testing since BOINC v7 was launched, which is the primary reason why I'm here. Let's discuss that when Travis has got the server back on an even keel with respect to the GPU applications.

BTW, I was recently reminded of a thread I started, and Travis contributed to, with regard to the difficulties of predicting runtimes in advance. I don't think much has happened on this issue in the intervening 18 months.

http://www.setiusa.us/showthread.php?2415-BOINC-DA-admits-CreditNew-is-really-a-random-number-generator
ID: 58779 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58797 - Posted: 12 Jun 2013, 18:56:14 UTC - in response to Message 58688.  

yes, and but on my computer for mt dark tasks estimated at less then 10 minutes, 6 of 12 CPU's are parked for the entire duration of the task. Also, look at run time verses CPU time +/- equal in the results file. Why does the above group of tasks always have this condition! Bottom line the above referenced group of tasks are executing very ineffectively, perhaps with correct results, and are reserving 1100% more resources meaning only 1 CPU is required, and 12 CPUs are reserved for the entire run time!

also note that actual run time is most often 5 to 8 times (*) > original estimated run time. In other words, if MT was really working on the above group, the original estimate is ok.

I think I may have a better idea of this 'dark' task problem now.

I was running ps_nbody_06_06_dark_1371001451_18919 when I took this screenshot.


(direct link)

BOINC Manager (in the background) shows that the task has been running for 3 minutes already, but it's still showing 0%age progress - and it's only using one thread (left-hand Process Explorer window).

A couple of minutes later, it went into full multi-threaded mode, and progress jumped from 0% to 100% in a couple of seconds. Finally, there was a further 5 minutes or so more running after 100% had been reached, during which again only one thread was active.

Other types of task seem to spend far less time (both absolutely and relatively) in the single-threaded plateaux at 0% and 100%, and far longer in the multi-threaded stage where the progress %age counter is incrementing steadily. Hope that help the programmers track down what's going on.
ID: 58797 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Paul Segal

Send message
Joined: 4 Apr 13
Posts: 3
Credit: 572,285
RAC: 0
Message 58801 - Posted: 12 Jun 2013, 21:52:02 UTC

This morning after loading with new work a new mt job was stuck at 62.7% so I suspended all the 40 mwah gpu jobs and the 3 seti jobs (1 gpu), mt job now running to completion.
ID: 58801 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Paul Segal

Send message
Joined: 4 Apr 13
Posts: 3
Credit: 572,285
RAC: 0
Message 58803 - Posted: 12 Jun 2013, 23:20:43 UTC - in response to Message 58711.  

And another mt stuck, this time at 46.415% after one minute precisely. Every test batch has one of these problem mt jobs wanting all the resources to enable completion.
ID: 58803 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58804 - Posted: 12 Jun 2013, 23:27:31 UTC - in response to Message 58803.  

And another mt stuck, this time at 46.415% after one minute precisely. Every test batch has one of these problem mt jobs wanting all the resources to enable completion.

It's not the batch or the WUs wanting anything, it's the bug in the BOINC client scheduler - it's being too assertive when trying to avoid over-committing the CPUs.
ID: 58804 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 22 Jun 11
Posts: 32
Credit: 41,852,496
RAC: 0
Message 58844 - Posted: 14 Jun 2013, 3:27:34 UTC
Last modified: 14 Jun 2013, 4:06:25 UTC

Richard,

I'm not trying to hijack a thread here, but... I wanted to point out that the scheduling policy may not be bugged here.

There's a basic "Job Scheduling" section within the BOINC documentation, found here: http://boinc.berkeley.edu/trac/wiki/ClientSched
Those 4 bullet points drive the main scheduling.

But I think, additionally (per the emails below from April, where I asked David Anderson about it a bit)... the scheduler makes sure never to be commit more than #ncpus, when running a multi-thread (mt) task. Conversely, it won't schedule an mt task if the resulting cpu usage would be more than #ncpus.

What's happening, in your case (I think), is that the mt task runs high-priority for a bit (as "Job Scheduling" bullet 2), then at some point BOINC thinks it can finish by deadline, so... it re-evaluates the task list to do scheduling, and schedules a GPU task (as "Job Scheduling" bullet 3), and then when deciding to schedule the mt task (as "Job Scheduling" bullet 4), it does not schedule it, since doing so would result in cpu usage more than #ncpus. If you have enough GPU tasks to always be running at least 1 GPU task, it will never re-schedule the mt task until BOINC puts it in high-priority mode again.

Believe it or not... I think it may be working correctly, and I cannot think of a better design. If you can, you might dig up the boinc_alpha thread, and reply to it. Maybe a design that treats mt tasks and gpu tasks as equals, above regular-cpu tasks?

I don't run mt tasks, so I don't have behavioral proof, but I'm betting your behavior matches the policies described here; ie: no bug.

Regards,
Jacob Klein

------------------------------------------------------------------------------------
Date: Tue, 2 Apr 2013 14:49:34 -0700
From: davea@ssl.berkeley.edu
To: boinc_alpha@ssl.berkeley.edu
Subject: Re: [boinc_alpha] Using app_config.xml <cpu_usage>2</cpu_usageresults in underloading/overloading CPU

These are both consistent with the current job-scheduling policy:

1) If a multi-thread job is running, the scheduler won't run a CPU job
if doing so would exceed #CPUs.

2) the scheduler will run GPU jobs until the CPU load is #CPUs+1
(but not beyond that).

There is a rationale for both of these, thought they are both
open to debate.

-- David

------------------------------------------------------------------------------------
Date: Thu, 4 Apr 2013 13:48:52 -0700
From: davea@ssl.berkeley.edu
To: jacob_w_klein@msn.com
CC: boinc_alpha@ssl.berkeley.edu
Subject: Re: [boinc_alpha] Using app_config.xml <cpu_usage>2</cpu_usageresults in underloading/overloading CPU

Jacob:

There are situations where overloading the CPUs, even slightly,
can greatly reduce overall throughput.

1) multithread apps. Typically these are tightly coupled,
meaning that each thread uses results recently computed by other threads,
and if these are not available it has to wait.
Multithread apps perform best if all the threads are running all the time.
If 1 thread is descheduled (e.g. because there are 8.5 runnable
threads on an 8-CPU system) this causes the other threads to
sleep and wake up, which has an overhead.
If this happens frequently it has a cascading effect
and this can reduce the throughput of the app by a large factor (like 25%).

2) GPU apps. If the CPU part of a GPU app is descheduled,
the GPU is idle until it's scheduled again.
This can be on the order of a second, esp. on Windows.

In such cases, throughput can be much higher e.g. with
7.5 threads than with 8.5 threads.

So I'm reluctant to make any changes based on our collective intuition,
because that intuition may not be correct.

Ideally, some computer science graduate student would do a research
project to study this issue with a range of real-world applications,
and we could use the findings to improve our policies.
However, the academic computer science world pretends that
volunteer computing doesn't exist, so no such study has been done
or is likely to get done.

-- David
------------------------------------------------------------------------------------
ID: 58844 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Jeffery M. Thompson
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 23 Sep 12
Posts: 159
Credit: 16,977,106
RAC: 0
Message 58847 - Posted: 14 Jun 2013, 5:13:16 UTC

We are looking to get the wu.rsc_fpops_est to mitigate this issue.

We are using the work units reported here and work units with large run times returned to the servers to see if we can get a better estimate to help with priority scheduling issues at the least.

We may need more data to make this work and it may be that due to the nature of chaos in some physical systems modeled that there will always be estimates that are larger than the real run time. But on the average it should balance out.

Also we do have valid work units that will have long run times because they have to calculate more. There are valid models where the particles spread out requiring more individual calculations.

Knowing these cases ahead of the calculation would mean that we would not need to run the calculations we could throw them out before they were run.

So there is a middle somewhere. And that is what I am working on. I believe getting a better estimate will help with the scheduler. And mitigate the issues.

The other thing to look at is the reporting deadline. As our project is asynchronous, new work units are generated based on the early results from the earlier work units, thus timing is important. So I am trying to find a balance in this. The current form may be causing issues that need resolved.

The calculations are done on the server side so there will not be a new binary released at this time.

On the positive side it is returning good science, the results make sense for the problem being addressed.



.Jeff

ID: 58847 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58849 - Posted: 14 Jun 2013, 8:48:36 UTC - in response to Message 58844.  

Richard,

I'm not trying to hijack a thread here, but... I wanted to point out that the scheduling policy may not be bugged here.

There's a basic "Job Scheduling" section within the BOINC documentation, found here: http://boinc.berkeley.edu/trac/wiki/ClientSched
Those 4 bullet points drive the main scheduling.

But I think, additionally (per the emails below from April, where I asked David Anderson about it a bit)... the scheduler makes sure never to be commit more than #ncpus, when running a multi-thread (mt) task. Conversely, it won't schedule an mt task if the resulting cpu usage would be more than #ncpus.
------------------------------------------------------------------------------------
Date: Tue, 2 Apr 2013 14:49:34 -0700
From: davea@ssl.berkeley.edu
To: boinc_alpha@ssl.berkeley.edu
Subject: Re: [boinc_alpha] Using app_config.xml <cpu_usage>2</cpu_usageresults in underloading/overloading CPU

These are both consistent with the current job-scheduling policy:

1) If a multi-thread job is running, the scheduler won't run a CPU job
if doing so would exceed #CPUs.

2) the scheduler will run GPU jobs until the CPU load is #CPUs+1
(but not beyond that).

There is a rationale for both of these, thought they are both
open to debate.

-- David
------------------------------------------------------------------------------------

I have no query with policy (1) in that email - that seems to be working as designed and intended.

But I think the current v7.0.64 scheduler is failing to obey policy (2) - the relationship between MT and GPU tasks.

I've been following this one for quite a while: very few projects supply MT apps, so it doesn't get much attention in alpha testing - the ClientSched document you linked doesn't even mention MT jobs, let alone co-scheduling with GPUs, and the change from SVN to GIT has made old code changes unsearchable. But I'm pretty certain I had a hand in:
------------------------------------------------------------------------------------
Revision: 97ee3a38f265653d6b16bd5611df3ece4b2eef91
Author: David Anderson <davea@ssl.berkeley.edu>
Date: 22/09/2009 00:23:40
Message:
- client: tweak CPU scheduling policy to avoid running multithread apps overcommitted.
Actually: allow overcommitment but only a fractional CPU
(so that, e.g., we can run a GPU app and a 4-CPU app on a 4-CPU host)


svn path=/trunk/boinc/; revision=19126
----
Modified: checkin_notes
Modified: client/cpu_sched.cpp
------------------------------------------------------------------------------------
At the moment, we can only do the highlighted bit some of the time.
ID: 58849 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 22 Jun 11
Posts: 32
Credit: 41,852,496
RAC: 0
Message 58853 - Posted: 14 Jun 2013, 12:45:00 UTC - in response to Message 58849.  
Last modified: 14 Jun 2013, 13:02:40 UTC

But I think the current v7.0.64 scheduler is failing to obey policy (2) - the relationship between MT and GPU tasks.

Revision: 97ee3a38f265653d6b16bd5611df3ece4b2eef91
Author: David Anderson <davea@ssl.berkeley.edu>
Date: 22/09/2009 00:23:40
Message:
- client: tweak CPU scheduling policy to avoid running multithread apps overcommitted.
Actually: allow overcommitment but only a fractional CPU
(so that, e.g., we can run a GPU app and a 4-CPU app on a 4-CPU host)


Interesting. Sorry, I was not aware of that 2009 checkin where the concession was made. Perhaps this will inspire me to take on an MT task to test exactly how it works.

Are you saying that, if the MT task is scheduled first, and then GPU tasks are allowed to also be scheduled, then BOINC will schedule both [correctly]?

And are you also saying that, if the GPU task is scheduled first, and then the MT task is allowed to also be scheduled, then BOINC will only schedule the GPU task [incorrectly]?

If those statements are how it is currently behaving, and that inconsistency exists, then it does indeed sound like a BOINC scheduling bug.

[Edit] I just tested with 7.1.15 alpha, and it appears that BOINC is correctly scheduling for me. If the 8-CPU MT task is started first, BOINC will allow 2 GPU tasks that each use 0.5 CPU. If the 2 GPU tasks are started first, BOINC will allow the 8-CPU MT task to also be scheduled. Am I trying to reproduce the bug incorrectly?
ID: 58853 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58855 - Posted: 14 Jun 2013, 13:41:07 UTC
Last modified: 14 Jun 2013, 14:07:35 UTC

My observations, it appears to me that the more CPU cores you have the more "ok" it is to overcommit. For reference, I have 12 cores, and the MT tasks can run 2 GPU tasks quite well with total machine CPU loads at +/- 85% total CPU percent used. The scheduler does allow GPU tasks to load / execute while MT tasks are running, with the apparent exception of when a High Priority MT task changes to Normal priority. WHEN THIS HAPPENS, then the SCHEDULER stops the MT tasks puts the task in "wait" status and will not allow any other CPU based tasks to execute until ALL other work - INCLUDIONG ALL GPU tasks are complete, Including GPU tasks in QUE, which can mean if Boinc scheduler is obtaining more GPU tasks, CPU task will never commence. The Scheduler will not allow other any CPU work to start until the MT task is complete. Remember it is in "wait" state, waiting for 100% CPU availability. Essentially all 12 cores are available and only are supporting GPU work.

Bottom line, my opinion is that the more cores you have, it is GOOD to allow overcommitting. This allows cores that are underperforming because they have to wait to contribute to GPU activity. I also agree that the Sum GPU CPU core requirements needs to be <= 1 CPU core OR perhaps <= .1 * # of cores (.1*12 cores = 1.2 provided that no GPU task requires a 1, a whole core).

I would be glad to record or snapshot total CPU use on this computer with 12 cores. Please advise...

There is a number of issues here, the most serious being that the MT task when it changes status from High Priority to normal, then goes into "Wait" and holds all remaining CPU tasks hostage.
ID: 58855 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58859 - Posted: 14 Jun 2013, 18:30:10 UTC - in response to Message 58855.  

There is a number of issues here, the most serious being that the MT task when it changes status from High Priority to normal, then goes into "Wait" and holds all remaining CPU tasks hostage.

My observation too, and I think it ties in with my conversation with Jacob Klein.

Back in 2009, when I was doing similar tests and having similar conversations with the 'AQUA' project, we were still using BOINC v6 versions with separate CPU and GPU scheduling, each resource maintaining a separate 'debt' level with respect to other projects.

Now, I'm testing with BOINC v7.0.64, which maintains a single common 'REC' for each project, which is used to schedule both CPU and GPU tasks.

I'm seeing this failure to run MT and GPU tasks together after a long - many hours - exclusive occupation of all CPUs by the MT task running in high priority. This means the REC for MW (as a whole) will be high, and runtime prio will be low: BOINC will be reluctant to schedule MW for a while to come. This may result in a different set of scheduling decisions from those seen by Jacob, who - with very little MW running under his belt - will be in the opposite 'low REC, high prio' state. I've seen MT + GPU running properly in that state, too.

In order to help the bug-hunt here, here's the message log for the event I posted to boinc_alpha:

10/06/2013 13:53:11 | | [cpu_sched_debug] Request CPU reschedule: periodic CPU scheduling
10/06/2013 13:53:11 | | [cpu_sched_debug] schedule_cpus(): start
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] thrashing prevention: mark de_nbody_100k_chisq_alt_40913_1366886102_708586_1 as deadline miss
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] scheduling 10ja12ab.22792.19492.13.12.3_0 (coprocessor job, FIFO) (prio -1.377456)
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] scheduling de_nbody_100k_chisq_alt_40913_1366886102_708586_1 (CPU job, priority order) (prio -0.245088)
10/06/2013 13:53:11 | | [cpu_sched_debug] enforce_schedule(): start
10/06/2013 13:53:11 | | [cpu_sched_debug] preliminary job list:
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] 0: 10ja12ab.22792.19492.13.12.3_0 (MD: no; UTS: yes)
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] 1: de_nbody_100k_chisq_alt_40913_1366886102_708586_1 (MD: no; UTS: no)
10/06/2013 13:53:11 | | [cpu_sched_debug] final job list:
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] 0: 10ja12ab.22792.19492.13.12.3_0 (MD: no; UTS: yes)
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] 1: de_nbody_100k_chisq_alt_40913_1366886102_708586_1 (MD: no; UTS: no)
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] scheduling 10ja12ab.22792.19492.13.12.3_0
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] avoid MT overcommit: skipping de_nbody_100k_chisq_alt_40913_1366886102_708586_1
10/06/2013 13:53:11 | | [cpu_sched_debug] using 0.04 out of 4 CPUs
10/06/2013 13:53:11 | NumberFields@home | [cpu_sched_debug] wu_DS-16x121_Grp514181of819200_0 sched state 1 next 1 task state 9
10/06/2013 13:53:11 | NumberFields@home | [cpu_sched_debug] wu_DS-16x121_Grp523637of819200_0 sched state 1 next 1 task state 9
10/06/2013 13:53:11 | Milkyway@Home | [cpu_sched_debug] de_nbody_100k_chisq_alt_40913_1366886102_708586_1 sched state 1 next 1 task state 9
10/06/2013 13:53:11 | NumberFields@home | [cpu_sched_debug] wu_DS-16x121_Grp526885of819200_0 sched state 1 next 1 task state 9
10/06/2013 13:53:11 | SETI@home | [cpu_sched_debug] 10ja12ab.22792.19492.13.12.3_0 sched state 2 next 2 task state 1
10/06/2013 13:53:11 | | [cpu_sched_debug] enforce_schedule: end

I'm assuming that, somewhere in the switchover from debt-based to REC-based scheduling, a code path was introduced which overlooks the 'allow minimal overcommits for GPUs' rule. It will be interesting to see if this has persisted into v 7.1.15 and later. I haven't seen any alterations in that area, but Jacob has been watching that area of code more closely than I have over the last few weeks.
ID: 58859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 22 Jun 11
Posts: 32
Credit: 41,852,496
RAC: 0
Message 58860 - Posted: 14 Jun 2013, 18:38:52 UTC - in response to Message 58859.  
Last modified: 14 Jun 2013, 18:41:22 UTC

Now, THAT is interesting. Look closer.
The code has logic to keep a task set as "deadline miss - high priority"... if it has PREVIOUSLY been marked "high priority". I think that's the "thrashing prevention" message you see there.
That may be key to getting into the scenario that has the problem.
"Previously high priority, to the point where BOINC would keep it high priority to prevent thrashing... and now in a situation where it's no longer a deadline miss."

So, serious question, what happens when this occurs on your system? Does it just run the GPU task, and leave the other 3.96 CPUs completely idle?
ID: 58860 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58861 - Posted: 14 Jun 2013, 18:47:42 UTC

Richard, great information. Thank you. Now what happens?

Do we have to wait for the next version of Boinc? Would a temporary band aid fix be to modify the MT task to be Number of available core processors - 1 (My case 12-1=11)?
ID: 58861 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58862 - Posted: 14 Jun 2013, 18:50:47 UTC - in response to Message 58860.  

So, serious question, what happens when this occurs on your system? Does it just run the GPU task, and leave the other 3.96 CPUs completely idle?

On my system, all CPU cores other than those that support GPU's go idle, EVEN if there are other CPU tasks available to run.
ID: 58862 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58863 - Posted: 14 Jun 2013, 18:57:33 UTC

FYI, I have the conditions currently executing so that in the next 30 minutes +/- I believe I will be in the "hung" position as being discussed... Advise if there is anyway I can assist....
ID: 58863 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 4 Sep 12
Posts: 219
Credit: 456,474
RAC: 0
Message 58864 - Posted: 14 Jun 2013, 19:03:10 UTC - in response to Message 58860.  

So, serious question, what happens when this occurs on your system? Does it just run the GPU task, and leave the other 3.96 CPUs completely idle?

Yes. That was part of my first "extended run" MT task, and I wanted to finish it and report it to see what happened next - so (and since there only appeared to this human to be a few more minutes to run), I suspended all other CPU work hoping that the MT task would run. It didn't, until I suspend the GPU task as well.

I'll have to run some more tasks - may be difficult this weekend - and get it into the same state with more debug log flags active - I think I'm going to need both priority and work_fetch, in addition to that cpu_sched.

@ jdzukley, extended logging as I've just suggested would be helpful, if you're familiar with setting that up in time - but unfortunately, I've got to go out now, so won't be around to guide you.
ID: 58864 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58865 - Posted: 14 Jun 2013, 19:27:34 UTC
Last modified: 14 Jun 2013, 19:28:41 UTC

Are these the log flags wanted:

<log_flags>
<coproc_debug>1</coproc_debug>
<cpu_sched>1</cpu_sched>
<cpu_sched_debug>1</cpu_sched_debug>
<priority_debug>1</priority_debug>
</log_flags>


I have these running at the moment, advise if I should have others...
ID: 58865 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58866 - Posted: 14 Jun 2013, 20:18:29 UTC

A snapshots of the debugs, first here is the job in execution:

6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_21_2s_sscon_1_1371083750_657599_0 (coprocessor job, FIFO) (prio -1.000000)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_79_DR8_rev_3_1371083750_657610_0 (coprocessor job, FIFO) (prio -1.020216)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] scheduling ps_nbody_06_11_dark_4_1371083750_62813_0 (CPU job, priority order) (prio -1.040431)
6/14/2013 3:44:39 PM | | [cpu_sched_debug] enforce_schedule(): start
6/14/2013 3:44:39 PM | | [cpu_sched_debug] preliminary job list:
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 0: de_separation_21_2s_sscon_1_1371083750_657599_0 (MD: no; UTS: yes)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 1: de_separation_79_DR8_rev_3_1371083750_657610_0 (MD: no; UTS: no)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 2: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:44:39 PM | | [cpu_sched_debug] final job list:
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 0: de_separation_21_2s_sscon_1_1371083750_657599_0 (MD: no; UTS: yes)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 1: de_separation_79_DR8_rev_3_1371083750_657610_0 (MD: no; UTS: no)
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] 2: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:44:39 PM | Milkyway@Home | [coproc] NVIDIA instance 1: confirming for de_separation_21_2s_sscon_1_1371083750_657599_0
6/14/2013 3:44:39 PM | Milkyway@Home | [coproc] Assigning NVIDIA instance 0 to de_separation_79_DR8_rev_3_1371083750_657610_0
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_21_2s_sscon_1_1371083750_657599_0
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_79_DR8_rev_3_1371083750_657610_0
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] avoid MT overcommit: skipping ps_nbody_06_11_dark_4_1371083750_62813_0
6/14/2013 3:44:39 PM | | [cpu_sched_debug] using 0.79 out of 12 CPUs
6/14/2013 3:44:39 PM | climateprediction.net | [cpu_sched_debug] hadcm3n_o3e8_1940_40_008382078_1 sched state 1 next 1 task state 9
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] ps_nbody_06_11_dark_4_1371083750_62813_0 sched state 1 next 1 task state 9
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] de_separation_21_2s_sscon_1_1371083750_657599_0 sched state 2 next 2 task state 1
6/14/2013 3:44:39 PM | Milkyway@Home | [cpu_sched_debug] de_separation_79_DR8_rev_3_1371083750_657610_0 sched state 0 next 2 task state 0
6/14/2013 3:44:39 PM | Milkyway@Home | Starting task de_separation_79_DR8_rev_3_1371083750_657610_0 using milkyway version 102 (opencl_nvidia) in slot 0
6/14/2013 3:44:39 PM | | [cpu_sched_debug] enforce_schedule: end



Now is the shot where the priority changed and the job was suspended:
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_21_2s_sscon_1_1371083750_657599_0 (coprocessor job, FIFO) (prio -1.000000)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_79_DR8_rev_3_1371083750_657610_0 (coprocessor job, FIFO) (prio -1.020216)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] reserving 1.000000 of coproc NVIDIA
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] scheduling ps_nbody_06_11_dark_4_1371083750_62813_0 (CPU job, priority order) (prio -1.040431)
6/14/2013 3:44:55 PM | | [cpu_sched_debug] enforce_schedule(): start
6/14/2013 3:44:55 PM | | [cpu_sched_debug] preliminary job list:
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 0: de_separation_21_2s_sscon_1_1371083750_657599_0 (MD: no; UTS: yes)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 1: de_separation_79_DR8_rev_3_1371083750_657610_0 (MD: no; UTS: no)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 2: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:44:55 PM | | [cpu_sched_debug] final job list:
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 0: de_separation_21_2s_sscon_1_1371083750_657599_0 (MD: no; UTS: yes)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 1: de_separation_79_DR8_rev_3_1371083750_657610_0 (MD: no; UTS: yes)
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] 2: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:44:55 PM | Milkyway@Home | [coproc] NVIDIA instance 1: confirming for de_separation_21_2s_sscon_1_1371083750_657599_0
6/14/2013 3:44:55 PM | Milkyway@Home | [coproc] NVIDIA instance 0: confirming for de_separation_79_DR8_rev_3_1371083750_657610_0
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_21_2s_sscon_1_1371083750_657599_0
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] scheduling de_separation_79_DR8_rev_3_1371083750_657610_0
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] avoid MT overcommit: skipping ps_nbody_06_11_dark_4_1371083750_62813_0
6/14/2013 3:44:55 PM | | [cpu_sched_debug] using 0.79 out of 12 CPUs
6/14/2013 3:44:55 PM | climateprediction.net | [cpu_sched_debug] hadcm3n_o3e8_1940_40_008382078_1 sched state 1 next 1 task state 9
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] ps_nbody_06_11_dark_4_1371083750_62813_0 sched state 1 next 1 task state 9
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] de_separation_21_2s_sscon_1_1371083750_657599_0 sched state 2 next 2 task state 1
6/14/2013 3:44:55 PM | Milkyway@Home | [cpu_sched_debug] de_separation_79_DR8_rev_3_1371083750_657610_0 sched state 2 next 2 task state 1
6/14/2013 3:44:55 PM | | [cpu_sched_debug] enforce_schedule: end


Unfortunately for this log capture, I did not have other CPU tasks ready for execution (the one job was suspended.


Next, I allowed the 2 GPU tasks in execution to finish, and suspended all remaining GPU tasks que.. this is the log right after the last GPU task finished showing the start of the MT tasks:

6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched_debug] scheduling ps_nbody_06_11_dark_4_1371083750_62813_0 (CPU job, priority order) (prio -1.000000)
6/14/2013 3:51:49 PM | | [cpu_sched_debug] enforce_schedule(): start
6/14/2013 3:51:49 PM | | [cpu_sched_debug] preliminary job list:
6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched_debug] 0: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:51:49 PM | | [cpu_sched_debug] final job list:
6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched_debug] 0: ps_nbody_06_11_dark_4_1371083750_62813_0 (MD: no; UTS: no)
6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched_debug] scheduling ps_nbody_06_11_dark_4_1371083750_62813_0
6/14/2013 3:51:49 PM | climateprediction.net | [cpu_sched_debug] hadcm3n_o3e8_1940_40_008382078_1 sched state 1 next 1 task state 9
6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched_debug] ps_nbody_06_11_dark_4_1371083750_62813_0 sched state 1 next 2 task state 9
6/14/2013 3:51:49 PM | Milkyway@Home | [cpu_sched] Resuming ps_nbody_06_11_dark_4_1371083750_62813_0
6/14/2013 3:51:49 PM | Milkyway@Home | Resuming task ps_nbody_06_11_dark_4_1371083750_62813_0 using milkyway_nbody version 118 (mt) in slot 3
6/14/2013 3:51:49 PM | | [cpu_sched_debug] enforce_schedule: end


Advise if you folks want a copy of the full log file for this time frame.
ID: 58866 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jdzukley

Send message
Joined: 26 May 11
Posts: 32
Credit: 43,959,896
RAC: 794
Message 58867 - Posted: 14 Jun 2013, 20:43:40 UTC
Last modified: 14 Jun 2013, 21:17:46 UTC

To help avoid High Priority MT Tasks and the slew of problems presented below, consider a quick temporary fix on the task generator server to update the estimated run time for the task to be no greater than (required completion date & time - (now() + say 24hours)). This way the task most often should not become High Priority.

estimated task hours = min (system generated estimated task hours, (required completion date - (now() + 24 hours)))

And but, I do not understand why a task that start out as High Priority, after it become Normal Priority, at no time can a GPU task can run until the specific MT task has completed... Any other MT task will allow GPU's to come and go...

Alternately, MT tasks can be set / sent to be # of processors - 1 and then this whole discussion thread is mute...

The CC config file for Boinc probably needs an optional setting for how many cores can be made available for MT processing.

My third long run High Priority MT task has just started (consecutively, no other short runs this afternoon after the first long started). I will be setting No New Tasks on Milkyway as I have better things to do on a Friday night than baby sit long run MT tasks that suspend most work on my system...
ID: 58867 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Overtonesinger
Avatar

Send message
Joined: 15 Feb 10
Posts: 63
Credit: 1,836,010
RAC: 0
Message 58880 - Posted: 15 Jun 2013, 7:47:35 UTC

Interesting WU, which crashed the 1.18 (mt) Win-x64 application on my very very stable mobile Core i7 CPU "720QM":

-1073741571 (0xffffffffc00000fd) Unknown error number

link to WU ID 382354923 is:
http://milkyway.cs.rpi.edu/milkyway/workunit.php?wuid=382354923

Err:
[code]<core_client_version>7.0.64</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -1073741571 (0xc00000fd)
</message>
<stderr_txt>
x0
002147f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002148c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214990 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214a60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214b30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214c00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214cd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214da0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214e70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00214f40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215010 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002150e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002151b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215280 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215350 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215420 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002154f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002155c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215690 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215760 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215830 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215900 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002159d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215aa0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215b70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215c40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215d10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215de0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215eb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00215f80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216050 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216120 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002161f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002162c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216390 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216460 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216530 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216600 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002166d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002167a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216870 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216940 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216a10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216ae0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216bb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216c80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216d50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216e20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216ef0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00216fc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217090 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217160 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217230 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217300 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002173d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002174a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217570 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217640 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217710 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002177e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002178b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217980 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217a50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217b20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217bf0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217cc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217d90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217e60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00217f30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218000 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002180d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002181a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218270 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218340 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218410 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002184e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002185b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218680 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218750 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218820 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002188f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002189c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218a90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218b60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218c30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218d00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218dd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218ea0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00218f70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219040 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219110 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002191e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002192b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219380 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219450 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219520 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002195f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002196c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219790 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219860 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219930 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219a00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219ad0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219ba0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219c70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219d40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219e10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219ee0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00219fb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a080 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a150 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a220 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a2f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a3c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a490 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a560 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a630 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a700 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a7d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a8a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021a970 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021aa40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ab10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021abe0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021acb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ad80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ae50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021af20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021aff0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b0c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b190 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b260 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b330 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b400 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b4d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b5a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b670 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b740 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b810 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b8e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021b9b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ba80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021bb50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021bc20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021bcf0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021bdc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021be90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021bf60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c030 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c100 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c1d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c2a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c370 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c440 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c510 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c5e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c6b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c780 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c850 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c920 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021c9f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021cac0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021cb90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021cc60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021cd30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ce00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ced0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021cfa0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d070 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d140 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d210 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d2e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d3b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d480 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d550 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d620 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d6f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d7c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d890 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021d960 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021da30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021db00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021dbd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021dca0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021dd70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021de40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021df10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021dfe0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e0b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e180 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e250 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e320 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e3f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e4c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e590 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e660 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e730 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e800 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e8d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021e9a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ea70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021eb40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ec10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ece0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021edb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ee80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ef50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f020 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f0f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f1c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f290 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f360 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f430 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f500 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f5d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f6a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f770 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f840 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f910 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021f9e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fab0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fb80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fc50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fd20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fdf0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021fec0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0021ff90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220060 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220130 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220200 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002202d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002203a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220470 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220540 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220610 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002206e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002207b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220880 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220950 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220a20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220af0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220bc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220c90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220d60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220e30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220f00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00220fd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002210a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221170 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221240 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221310 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002213e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002214b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221580 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221650 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221720 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002217f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002218c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221990 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221a60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221b30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221c00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221cd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221da0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221e70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00221f40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222010 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002220e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002221b0 00406485 00000000 b0f50000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222280 00406485 00000000 00000000 02980080 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222350 00406485 00000000 00000000 00000000 002c0000 milkyway_nbody_1.18_windows_x86!+0x0
00222420 00406485 00000000 00000000 00000000 003c0230 milkyway_nbody_1.18_windows_x86!+0x0
002224f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002225c0 00406485 00000000 003c0230 00000000 ffffffff milkyway_nbody_1.18_windows_x86!+0x0
00222690 00406485 003c0230 00000000 003cc3a0 003c0230 milkyway_nbody_1.18_windows_x86!+0x0
00222760 00406485 00000000 62000062 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222830 00406485 003c0000 00000001 00000000 00338b70 milkyway_nbody_1.18_windows_x86!+0x0
00222900 00406485 00494128 00458c86 00000000 0051ae00 milkyway_nbody_1.18_windows_x86!+0x0
002229d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222aa0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222b70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222c40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222d10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222de0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00222eb0 00406485 00000000 00000000 00000000 434e494f milkyway_nbody_1.18_windows_x86!+0x0
00222f80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223050 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223120 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002231f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002232c0 00406485 00000000 37313039 34643331 34373038 milkyway_nbody_1.18_windows_x86!+0x0
00223390 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223460 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223530 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223600 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002236d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002237a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223870 00406485 6f4d2036 53203033 20676e69 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223940 00406485 70206564 6d206373 6d206561 61203878 milkyway_nbody_1.18_windows_x86!+0x0
00223a10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223ae0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223bb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223c80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223d50 00406485 4dc8e636 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223e20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223ef0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00223fc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224090 00406485 00555043 00000000 00000001 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224160 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224230 00406485 0022d940 00000000 773e926a 00000002 milkyway_nbody_1.18_windows_x86!+0x0
00224300 00406485 773e8dde 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002243d0 00406485 00000000 00000000 00000000 003a0043 milkyway_nbody_1.18_windows_x86!+0x0
002244a0 00406485 0022e9b8 00000000 773e9308 00000001 milkyway_nbody_1.18_windows_x86!+0x0
00224570 00406485 000000c9 00000000 00220000 003ca200 milkyway_nbody_1.18_windows_x86!+0x0
00224640 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224710 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002247e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002248b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224980 00406485 00000000 003c0388 003c1330 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224a50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224b20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224bf0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224cc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224d90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224e60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00224f30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225000 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002250d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002251a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225270 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225340 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225410 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002254e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002255b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225680 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225750 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225820 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002258f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002259c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225a90 00406485 00000000 00000000 01010101 01010101 milkyway_nbody_1.18_windows_x86!+0x0
00225b60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225c30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225d00 00406485 0000003e 00000000 00000001 00000001 milkyway_nbody_1.18_windows_x86!+0x0
00225dd0 00406485 73736572 6f74735f 686b5f6c 6168735f milkyway_nbody_1.18_windows_x86!+0x0
00225ea0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00225f70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226040 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226110 00406485 00000000 00000000 40000000 00008000 milkyway_nbody_1.18_windows_x86!+0x0
002261e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002262b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226380 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226450 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226520 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002265f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002266c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226790 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226860 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226930 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226a00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226ad0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226ba0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226c70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226d40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226e10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226ee0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00226fb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227080 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227150 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227220 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002272f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002273c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227490 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227560 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227630 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227700 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002277d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002278a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227970 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227a40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227b10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227be0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227cb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227d80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227e50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227f20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00227ff0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002280c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228190 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228260 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228330 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228400 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002284d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002285a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228670 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228740 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228810 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002288e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002289b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228a80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228b50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228c20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228cf0 00406485 01010101 01010101 01010101 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228dc0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228e90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00228f60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229030 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229100 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002291d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002292a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229370 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229440 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229510 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002295e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002296b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229780 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229850 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229920 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
002299f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229ac0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229b90 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229c60 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229d30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229e00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229ed0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
00229fa0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a070 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a140 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a210 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a2e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a3b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a480 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a550 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a620 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a6f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a7c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a890 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022a960 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022aa30 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ab00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022abd0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022aca0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ad70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ae40 00406485 01010101 01010101 01010101 01010101 milkyway_nbody_1.18_windows_x86!+0x0
0022af10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022afe0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b0b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b180 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b250 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b320 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b3f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b4c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b590 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b660 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b730 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b800 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b8d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022b9a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ba70 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022bb40 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022bc10 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022bce0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022bdb0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022be80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022bf50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c020 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c0f0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c1c0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c290 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c360 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c430 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c500 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c5d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c6a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c770 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c840 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c910 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022c9e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cab0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cb80 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cc50 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cd20 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cdf0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cec0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022cf90 00406485 00000000 00000000 00000000 01010101 milkyway_nbody_1.18_windows_x86!+0x0
0022d060 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d130 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d200 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d2d0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d3a0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d470 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d540 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d610 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d6e0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d7b0 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d880 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022d950 00406485 00000000 02b6e070 00000000 02b6e080 milkyway_nbody_1.18_windows_x86!+0x0
0022da20 00406485 000004be 00000000 003bea20 003c12f0 milkyway_nbody_1.18_windows_x86!+0x0
0022daf0 00406485 00000080 000003ff 00000000 0000002c milkyway_nbody_1.18_windows_x86!+0x0
0022dbc0 00406485 00640230 006bdae0 00000075 00000016 milkyway_nbody_1.18_windows_x86!+0x0
0022dc90 00406485 06000600 003c12f0 774a5400 00000022 milkyway_nbody_1.18_windows_x86!+0x0
0022dd60 00406485 006bdad0 006bdae0 00000023 003c12f0 milkyway_nbody_1.18_windows_x86!+0x0
0022de30 00406485 003c1fb8 00000049 003c0000 00000002 milkyway_nbody_1.18_windows_x86!+0x0
0022df00 00406485 02e80370 00640158 00000000 00640158 milkyway_nbody_1.18_windows_x86!+0x0
0022dfd0 00406485 0022e178 00450044 0208002a 00643358 milkyway_nbody_1.18_windows_x86!+0x0
0022e0a0 00406485 0022e460 ffffffff 00640158 0022e196 milkyway_nbody_1.18_windows_x86!+0x0
0022e170 00406485 006f0074 006d0061 00640158 00640000 milkyway_nbody_1.18_windows_x86!+0x0
0022e240 00406485 00643330 0022e406 0022e3a0 00640230 milkyway_nbody_1.18_windows_x86!+0x0
0022e310 00406485 00000000 00000003 000000a0 00000045 milkyway_nbody_1.18_windows_x86!+0x0
0022e3e0 00406485 006404a8 006bdb70 00790061 006bdb80 milkyway_nbody_1.18_windows_x86!+0x0
0022e4b0 00406485 00000022 00000000 774a5410 00640230 milkyway_nbody_1.18_windows_x86!+0x0
0022e580 00406485 00000000 00000000 006b88a0 773b81c4 milkyway_nbody_1.18_windows_x86!+0x0
0022e650 00406485 0022ea70 00640000 0022e718 0022e760 milkyway_nbody_1.18_windows_x86!+0x0
0022e720 00406485 00000000 00000003 00000040 0000000a milkyway_nbody_1.18_windows_x86!+0x0
0022e7f0 00406485 00643330 774a7540 003a0044 006bdad0 milkyway_nbody_1.18_windows_x86!+0x0
0022e8c0 00406485 00000000 00643330 00000003 00000092 milkyway_nbody_1.18_windows_x86!+0x0
0022e990 00406485 0022ea00 00000045 00000080 0022edc0 milkyway_nbody_1.18_windows_x86!+0x0
0022ea60 00406485 00410044 0070005c 0065006a 005c0073 milkyway_nbody_1.18_windows_x86!+0x0
0022eb30 00406485 00000000 00000101 00000000 0000004f milkyway_nbody_1.18_windows_x86!+0x0
0022ec00 00406485 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ecd0 00406485 00220000 00000000 00000003 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022eda0 00406485 00000000 0000002c 00000000 00000009 milkyway_nbody_1.18_windows_x86!+0x0
0022ee70 00406485 00000000 0000005d 03d40070 03d40080 milkyway_nbody_1.18_windows_x86!+0x0
0022ef40 00406485 00000001 000003ff 0022f180 0000005f milkyway_nbody_1.18_windows_x86!+0x0
0022f010 00406485 dbd47c96 02e80360 00000001 00000007 milkyway_nbody_1.18_windows_x86!+0x0
0022f0e0 00406485 ffffffff fd1f2bd6 003b9aa0 00000029 milkyway_nbody_1.18_windows_x86!+0x0
0022f1b0 00406485 00000080 02b00080 0022f220 003c12f0 milkyway_nbody_1.18_windows_x86!+0x0
0022f280 00406485 06000600 003c12f0 06000600 000001ce milkyway_nbody_1.18_windows_x86!+0x0
0022f350 00406485 00000000 00000000 003c12f0 2e00002e milkyway_nbody_1.18_windows_x86!+0x0
0022f420 00406485 02000002 773e413d 003c0000 00340301 milkyway_nbody_1.18_windows_x86!+0x0
0022f4f0 00406485 00000000 03d40080 02000002 fd1e8101 milkyway_nbody_1.18_windows_x86!+0x0
0022f5c0 00406485 00004320 003c0000 0022f790 00004309 milkyway_nbody_1.18_windows_x86!+0x0
0022f690 00406485 079cf041 33f4b57a 00000000 ffffffff milkyway_nbody_1.18_windows_x86!+0x0
0022f760 00406485 00000021 00000010 02b70078 fd1e10c8 milkyway_nbody_1.18_windows_x86!+0x0
0022f830 00406485 02b6f7f8 02b70078 02b70058 02b719f8 milkyway_nbody_1.18_windows_x86!+0x0
0022f900 00406589 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022f9a0 00402964 294af1b1 7f6d7119 7d243c3a 18573a3e milkyway_nbody_1.18_windows_x86!+0x0
0022fa50 00403264 00000000 00000028 003ca8a0 003ca8a0 milkyway_nbody_1.18_windows_x86!+0x0
0022fc50 004f7a07 fd272ae0 0050af08 00000008 00000008 milkyway_nbody_1.18_windows_x86!+0x0
0022fe60 004013ce 00000011 00327380 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ff20 004014e8 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ff50 7728652d 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
0022ff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
0022ffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 8512 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000000000000 rbx=0000000000000000 rcx=00000000fd25a7a0 rdx=00000000027eeab8 rsi=0000000000000064 rdi=0000000000000000
r8=00000000027eea84 r9=0000000000000000 r10=0000000000000200 r11=0000000000000063 r12=00000000027efeb0 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000 rip=00000000773e165a rsp=00000000027efe88 rbp=0000000000000000
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
027efe80 fc6f1203 027eff48 00000000 00000000 00000000 ntdll!ZwDelayExecution+0x0
027eff20 0045b341 77292b20 00000000 00000000 00000000 KERNELBASE!SleepEx+0x0
027eff50 7728652d 00000000 00000000 00000000 00000000 milkyway_nbody_1.18_windows_x86!+0x0
027eff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
027effd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 6152 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000005c6f520 rbx=00000000069ffc80 rcx=000000000000e28d rdx=0000000006215200 rsi=0000000000000000 rdi=0000000000000002
r8=000000000000e300 r9=0000000000000001 r10=000000000000e280 r11=0000000000000246 r12=0000000000000000 r13=00000000069ffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=00000000069ffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
069ffb40 fc6f1430 00000000 00018680 0030d000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
069ffc40 77281220 069ffc90 069ffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
069ffcd0 672c67d3 ffffffff 046809b8 00335a30 003c77b8 kernel32!WaitForMultipleObjects+0x0
069ffd30 672c6c61 0022f9a0 00407c30 003c79b8 003c79b8 pthreadGC2_64!pthreadCancelableWait+0x0
069ffda0 6ab85ae5 00000000 fd1e10c8 003ca1c0 003c79b0 pthreadGC2_64!sem_wait+0x0
069ffde0 6ab85c1f 00334a78 672c4940 00336d50 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
069ffe30 6ab84d05 003c5bf0 00001808 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
069ffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
069ffef0 fd1e415f 04680cd0 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
069fff20 fd1e6ebd fd271ea0 04680cd0 00000000 00000000 msvcrt!srand+0x0
069fff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
069fff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
069fffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 5184 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000005fe00e0 rbx=0000000006bffc80 rcx=0000000000017557 rdx=000000000633ab40 rsi=0000000000000000 rdi=0000000000000002
r8=0000000000017580 r9=0000000000000001 r10=0000000000017500 r11=0000000000000246 r12=0000000000000000 r13=0000000006bffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=0000000006bffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
06bffb40 fc6f1430 00000000 00018680 0030d000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
06bffc40 77281220 06bffc90 06bffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
06bffcd0 672c67d3 fffffffd 04680a18 0032bfd0 003c77b8 kernel32!WaitForMultipleObjects+0x0
06bffd30 672c6c61 00000000 00000000 00000000 fc6f133c pthreadGC2_64!pthreadCancelableWait+0x0
06bffda0 6ab85ae5 00000000 fd1e10c8 003c9ce0 003c7790 pthreadGC2_64!sem_wait+0x0
06bffde0 6ab85c1f 00334a78 672c4940 00336dc0 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
06bffe30 6ab84d05 003c5bf0 00001440 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
06bffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
06bffef0 fd1e415f 04740010 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
06bfff20 fd1e6ebd fd271ea0 04740010 00000000 00000000 msvcrt!srand+0x0
06bfff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
06bfff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
06bfffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 1572 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000005c53200 rbx=0000000006dffc80 rcx=000000000000ddda rdx=000000000620bba0 rsi=0000000000000000 rdi=0000000000000002
r8=000000000000de00 r9=0000000000000001 r10=000000000000dd80 r11=0000000000000246 r12=0000000000000000 r13=0000000006dffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=0000000006dffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
06dffb40 fc6f1430 00000000 00018600 0030c000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
06dffc40 77281220 06dffc90 06dffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
06dffcd0 672c67d3 fffffff9 04680a78 003cb130 003c77b8 kernel32!WaitForMultipleObjects+0x0
06dffd30 672c6c61 0022f9a0 00407c30 003c7898 003c7898 pthreadGC2_64!pthreadCancelableWait+0x0
06dffda0 6ab85ae5 00000000 fd1e10c8 003c9d10 003c7890 pthreadGC2_64!sem_wait+0x0
06dffde0 6ab85c1f 00334a78 672c4940 00336e30 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
06dffe30 6ab84d05 003c5bf0 00000624 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
06dffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
06dffef0 fd1e415f 04680cd0 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
06dfff20 fd1e6ebd fd271ea0 04680cd0 00000000 00000000 msvcrt!srand+0x0
06dfff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
06dfff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
06dfffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 9668 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000006043280 rbx=0000000006fffc80 rcx=00000000004fa660 rdx=000000000053d140 rsi=0000000000000000 rdi=0000000000000002
r8=00000000044c9300 r9=0000000000000000 r10=0000000000000001 r11=0000000000000206 r12=0000000000000000 r13=0000000006fffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=0000000006fffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
06fffb40 fc6f1430 00000000 00018600 0030c000 00924000 ntdll!ZwWaitForMultipleObjects+0x0
06fffc40 77281220 06fffc90 06fffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
06fffcd0 672c67d3 fffffffb 04680ad8 04740010 003c77b8 kernel32!WaitForMultipleObjects+0x0
06fffd30 672c6c61 0022f9a0 00407c30 003c7818 003c7818 pthreadGC2_64!pthreadCancelableWait+0x0
06fffda0 6ab85ae5 00000000 fd1e10c8 003c9b60 003c7810 pthreadGC2_64!sem_wait+0x0
06fffde0 6ab85c1f 00334a78 672c4940 00336ea0 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
06fffe30 6ab84d05 003c5bf0 000025c4 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
06fffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
06fffef0 fd1e415f 003cfca0 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
06ffff20 fd1e6ebd fd271ea0 003cfca0 00000000 00000000 msvcrt!srand+0x0
06ffff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
06ffff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
06ffffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 8696 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000005f98920 rbx=00000000071ffc80 rcx=000000000001696d rdx=0000000006322e00 rsi=0000000000000000 rdi=0000000000000002
r8=0000000000016980 r9=0000000000000001 r10=0000000000016900 r11=0000000000000206 r12=0000000000000000 r13=00000000071ffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=00000000071ffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
071ffb40 fc6f1430 00000000 00018680 0030d000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
071ffc40 77281220 071ffc90 071ffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
071ffcd0 672c67d3 fffffffe 04680b38 04680cd0 003c77b8 kernel32!WaitForMultipleObjects+0x0
071ffd30 672c6c61 0022f9a0 00407c30 003c79d8 fc6f133c pthreadGC2_64!pthreadCancelableWait+0x0
071ffda0 6ab85ae5 00000000 fd1e10c8 003c9f20 003c79d0 pthreadGC2_64!sem_wait+0x0
071ffde0 6ab85c1f 00334a78 672c4940 00336f10 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
071ffe30 6ab84d05 003c5bf0 000021f8 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
071ffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
071ffef0 fd1e415f 00337fb0 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
071fff20 fd1e6ebd fd271ea0 00337fb0 00000000 00000000 msvcrt!srand+0x0
071fff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
071fff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
071fffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 8900 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000006004800 rbx=00000000073ffc80 rcx=0000000000017b6a rdx=0000000006346da0 rsi=0000000000000000 rdi=0000000000000002
r8=0000000000017b80 r9=0000000000000001 r10=0000000000017b00 r11=0000000000000246 r12=0000000000000000 r13=00000000073ffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=00000000073ffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
073ffb40 fc6f1430 00000000 00018680 0030d000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
073ffc40 77281220 073ffc90 073ffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
073ffcd0 672c67d3 fffffffc 04680b98 00338570 003c77b8 kernel32!WaitForMultipleObjects+0x0
073ffd30 672c6c61 0022f9a0 00407c30 003c7958 003c7958 pthreadGC2_64!pthreadCancelableWait+0x0
073ffda0 6ab85ae5 00000000 fd1e10c8 003ca130 003c7950 pthreadGC2_64!sem_wait+0x0
073ffde0 6ab85c1f 00334a78 672c4940 00336f80 6ab85c6e libgomp_64-1!omp_get_num_procs+0x0
073ffe30 6ab84d05 003c5bf0 000022c4 00000000 00000000 libgomp_64-1!omp_get_num_procs+0x0
073ffea0 672c628b fd271a80 00000000 00000000 fd1e1407 libgomp_64-1!GOMP_taskwait+0x0
073ffef0 fd1e415f 003cfca0 00000000 00000000 00000000 pthreadGC2_64!pthread_create+0x0
073fff20 fd1e6ebd fd271ea0 003cfca0 00000000 00000000 msvcrt!srand+0x0
073fff50 7728652d 00000000 00000000 00000000 00000000 msvcrt!_ftime64_s+0x0
073fff80 773bc521 00000000 00000000 00000000 00000000 kernel32!BaseThreadInitThunk+0x0
073fffd0 00000000 00000000 00000000 00000000 00000000 ntdll!RtlUserThreadStart+0x0

*** Dump of thread ID 10120 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
rax=0000000005de3f40 rbx=00000000075ffc80 rcx=00000000000120a8 rdx=0000000006291560 rsi=0000000000000000 rdi=0000000000000002
r8=0000000000012100 r9=0000000000000001 r10=0000000000012080 r11=0000000000000246 r12=0000000000000000 r13=00000000075ffbf0
r14=0000000000000000 r15=0000000004680480 rip=00000000773e18ca rsp=00000000075ffb48 rbp=0000000000000002
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
075ffb40 fc6f1430 00000000 fc6f133c 0030c000 00000000 ntdll!ZwWaitForMultipleObjects+0x0
075ffc40 77281220 075ffc90 075ffc80 00000000 00000000 KERNELBASE!GetCurrentProcess+0x0
075ffcd0 672c67d3 fffffffa 04680bf8 00337fb0 003c77b8 kernel32!WaitForMultipleObjects+0x0
075ffd30 672c6c61 0022f9a0 00407c30 003c7738 fc6f133c pthreadGC2_64!pthrea
ID: 58880 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : News : N-Body 1.18

©2024 Astroinformatics Group