Message boards :
Application Code Discussion :
source v0.18
Message board moderation
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
meh :( i cut like... 10 multiplies and a divide out of the calculate probabilities :( |
Send message Joined: 22 Feb 08 Posts: 260 Credit: 57,387,048 RAC: 0 |
meh :( i cut like... 10 multiplies and a divide out of the calculate probabilities :( well, the improvement for the stock gcc-compiled app should be more than 30sec... BTW, how do you like the new stderr? mic. |
Send message Joined: 22 Feb 08 Posts: 260 Credit: 57,387,048 RAC: 0 |
If someone feels like testing, here are the new v18(beta) apps: Linux32 Pack Linux64 Pack LinuxAMD Pack Report problems & errors here... mic. |
Send message Joined: 16 Nov 07 Posts: 23 Credit: 4,774,710 RAC: 0 |
Guess i'll have to install some *nix real soon now on my new quad... It runs XP64 after a few tries with drivers and stuff, but tx to a crappy GPU bios (reports itself as an engineering version, would you believe that?) I have no GPU crunching atm. Now I only need a few 'recreational' hours (plus a few, plus a few, hey it's IT...) to install *nix. Any recommendations for the flavour? I'm not a total newbie, did some AIX sysadm in my early days (and not only smitty). But I would also like to use some regular programs like a spreadsheet, mail, photoeditor and stuff. |
Send message Joined: 22 Feb 08 Posts: 260 Credit: 57,387,048 RAC: 0 |
Guess i'll have to install some *nix real soon now on my new quad... I usually use ubuntu for desktops and notebooks, debian for servers. mic. |
Send message Joined: 13 Feb 08 Posts: 1124 Credit: 46,740 RAC: 0 |
If someone feels like testing, here are the new v18(beta) apps: No problems, ran test units and new ps work fine ... checkpointing OK. |
Send message Joined: 4 Oct 08 Posts: 1734 Credit: 64,228,409 RAC: 0 |
Are Windows opti compiled applications coming soon from the V0.18? |
Send message Joined: 9 Sep 07 Posts: 22 Credit: 320,035 RAC: 0 |
Now I only need a few 'recreational' hours (plus a few, plus a few, hey it's IT...) to install *nix. Any recommendations for the flavour? I'm not a total newbie, did some AIX sysadm in my early days (and not only smitty). But I would also like to use some regular programs like a spreadsheet, mail, photoeditor and stuff. I like Fedora. But Ubuntu is OK too. I use Open Office for word processing and spread sheet type stuff. GIMP is good for high power photo stuff. For simple resizing and cropping I use KolourPaint (included with KDE), which is more powerful than MS Paint. And back on topic, I'm going to install the new beta app (32 bit) in a few minutes. Kathryn :o) The BOINC FAQ Service The Unofficial BOINC Wiki The Trac System More BOINC information than you can shake a stick of RAM at. |
Send message Joined: 26 Jul 08 Posts: 627 Credit: 94,940,203 RAC: 0 |
Are Windows opti compiled applications coming soon from the V0.18? No, 0.18 is still too slow ;) The current 0.16 apps (they are also not compiled from the 0.16 sources) will just be relabeled as 0.18. |
Send message Joined: 26 Jul 08 Posts: 627 Credit: 94,940,203 RAC: 0 |
meh :( i cut like... 10 multiplies and a divide out of the calculate probabilities :( @speedimic: If you want to get closer to the performance of the optimized Windows apps, prepare the compilation of the 0.18d sources, hopefully coming tomorrow. They will save a bit more than 30s ;) |
Send message Joined: 2 Jan 08 Posts: 123 Credit: 69,522,293 RAC: 1,631 |
Installed the AMD pack and used the 32 bit source. 0.18c is running well and validating. It did do a strange thing when I changed over to the new application. The currently running jobs all had the percentage done reset to zero and start again, the time taken did not change but kept ticking over. This meant that the first 3 or 4 jobs I processed with the new app had much longer process times than the ones that have been done with the new app alone. Other than that all running smoothly and about 1 to 2 minutes quicker than the 0.16 version (on my AMD Opteron this means it drops from 33 to 34 minutes to 31 to 32 minutes with Linux Fedora Core 3 and 6, compare this to the Windows app that takes just 8 minutes with the same hardware, but still much better than an hour on the original stock app). Thanks to both Speedimic and Cluster Physik for the much improved optimized applications and to Travis and Dave for providing the app in the first place. |
Send message Joined: 22 Feb 08 Posts: 260 Credit: 57,387,048 RAC: 0 |
meh :( i cut like... 10 multiplies and a divide out of the calculate probabilities :( Looking forward to that. My compiler is ready and waiting... mic. |
Send message Joined: 22 Feb 08 Posts: 260 Credit: 57,387,048 RAC: 0 |
... must be the new checkpointing... mic. |
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
... Yeah that was probably the new checkpointing. I've updated the v0.18 code for everyone. It's pretty fast. It gave me around a 30% speedup on my machine with the current WU sizes, going from 25m12s down to 18m38s on stripe 86. |
Send message Joined: 30 Aug 07 Posts: 2046 Credit: 26,480 RAC: 0 |
The major change from 0.18c to 0.18d is the following: In calculating bg_prob (from calculate probabilities) we're now doing: <pre style="white-space:pre-wrap; "> (*bg_prob) = 0; if (alpha == delta == 1) { for (i = 0; i < ap->convolve; i++) { xyz[i][2] = r_point[i] * sinb; zp = r_point[i] * cosb; xyz[i][0] = zp * cosl - lbr_r; xyz[i][1] = zp * sinl; rg = sqrt(xyz[i][0]*xyz[i][0] + xyz[i][1]*xyz[i][1] + (xyz[i][2]*xyz[i][2])/(q*q)); rs = rg + r0; (*bg_prob) += qw_r3_N[i] / (rg * rs * rs * rs); } } else { for (i = 0; i < ap->convolve; i++) { xyz[i][2] = r_point[i] * sinb; zp = r_point[i] * cosb; xyz[i][0] = zp * cosl - lbr_r; xyz[i][1] = zp * sinl; rg = sqrt(xyz[i][0]*xyz[i][0] + xyz[i][1]*xyz[i][1] + (xyz[i][2]*xyz[i][2])/(q*q)); (*bg_prob) += qw_r3_N[i] / (pow(rg, alpha) * pow(rg + r0, alpha_delta3)); } } (*bg_prob) *= reff_xr_rp3; </pre> Since the parameter files we've been using have a fixed alpha and delta, which also happens to be 1, we can cut out those two calls to pow. This speeds things up quite a bit :) |
Send message Joined: 8 Nov 08 Posts: 178 Credit: 6,140,854 RAC: 0 |
Just tried profiling my custom build of the MW app earlier today. Seems that almost half of the time in the application, 47.8%, is from the exp function, which as I can tell, calculates e^x. The majority of those calls are from calculate_integral. The other half, 47.4%, is in the calculate_probabilities function proper. After that, with 0.4 and 0.2% are the sin and cos functions, respectively. Hope that helps optimizing the application somewhat. ;) |
Send message Joined: 18 Nov 07 Posts: 280 Credit: 2,442,757 RAC: 0 |
Here's where exp() is called: (it is also called in probability.c, but calculate_integrals() doesn't use functions from that file) function . . . . .. . . . . . frequency . param set_probability_constants() . 1 . . . . . sigmoid_curve_params[1] * (gPrime - sigmoid_curve_params[2]) set_probability_constants() . n . . . . . -((g-gPrime) * (g-gPrime) / (2 * stdev * stdev)) calculate_probabilities() . . n^2 . . . . -xyz_norm / stream_sigma_sq2[i] calculate_likelyhood() . .. . 1 . . . . . ap->background_weight calculate_likelyhood() . .. . n . . . . . ap->stream_weights[i] calculate_likelyhood() . .. . n . . . . . [same as above, presumably optimized out] From those, their frequency in calculate_integrals(): function . . . . .. . . . . . frequency set_probability_constants() . n calculate_probabilities() . . n^3 So calculate_probabilities is called with O(n^5) frequency! PS: sorry about the dots, the code tags are broken and remove extra spaces. |
Send message Joined: 12 Apr 08 Posts: 621 Credit: 161,934,067 RAC: 0 |
Any thoughts on the memory leak mentioned in the other thread? I note with my ATI crunching host I get about 1.6 G memory use in 24 hours or so. Stopping BOINC and killing the left over processes and restarting are required ... otherwise the system continues to slow ... |
Send message Joined: 12 Nov 07 Posts: 2425 Credit: 524,164 RAC: 0 |
Would something like this help to find the error when a wu errors out? Or add any other usefull information. I'm sure you've thought of some of this Cluster. I think your addition of the CPU & wall time is great. Rosetta's stderr out (most): BOINC:: Initializing ... ok. [2009- 3-14 9:12: 1:] :: BOINC :: boinc_init() BOINC:: Setting up shared resources ... ok. BOINC:: Setting up semaphores ... ok. BOINC:: Updating status ... ok. BOINC:: Registering timer callback... ok. BOINC:: Worker initialized successfully. Registering options.. Registered extra options. Initializing core... Initializing options.... ok Loaded options.... ok Processed options.... ok Initializing random generators... ok Initialization complete. Setting WU description ... Firstarg=true; pp=-d./ firstarg: <-d./> End of unzipping. Unpacking WU data ... Firstarg=true; pp=-d./ firstarg: <-d./> End of unzipping. Setting database description ... Setting up checkpointing ... BOINC:: Worker startup. Starting watchdog... Watchdog active. # cpu_run_time_pref: 36000 ====================================================== DONE :: 1 starting structures 34685.1 cpu seconds This process generated 25 decoys from 25 attempts ====================================================== Doesn't expecting the unexpected make the unexpected the expected? If it makes sense, DON'T do it. |
Send message Joined: 26 Jul 08 Posts: 627 Credit: 94,940,203 RAC: 0 |
Any thoughts on the memory leak mentioned in the other thread? That's strange. My apps (CPU and GPU) should not suffer from the leaks mentioned in the other thread. I use a different layout of the arrays than the stock app and have therefore also changed the memory allocation and deallocation. |
©2024 Astroinformatics Group