Welcome to MilkyWay@home

Crunchoff MW vs SETI

Message boards : Number crunching : Crunchoff MW vs SETI
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 12768 - Posted: 24 Feb 2009, 20:50:43 UTC - in response to Message 12745.  
Last modified: 24 Feb 2009, 20:52:00 UTC


MW has always done better on AMD's while SETI has done better on Intel's


If true, which I don't doubt that it is true, is a fundamental reason why one cannot rely on "the chart" to be the ultimate authority on "Cross-Project Parity". All the chart can do is show a composite average. The results of individual systems will vary from the average, thus leading to the same "credit shopping" that those who so strongly support CPP claim to wish to avoid.

CPP is a nice concept, but the current method to get there leaves a lot to be desired... IMO, the results of your testing will likely show that MW is about 1.6X - 1.8X of SETI, yet the chart is showing 0.902 right now...
ID: 12768 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 12 Apr 08
Posts: 621
Credit: 161,934,067
RAC: 0
Message 12773 - Posted: 24 Feb 2009, 21:33:41 UTC

I wonder, in the interests of research if I should reattach all my hosts to the universe and run them for the month so we can see the CPCS numbers ... Not all projects have work, and I don't use the same resource shares ... but it might be something to consider?

I am of two, or three, minds on this ... not at all sure what might fall out ...

I do know that most of my systems are fast enough that when I get work from projects I do tend to do one or more tasks a day for most projects ...

So, should I? Or shouldn't I ... I don't use a hairdresser, so, no one knows for sure ...
ID: 12773 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevint
Avatar

Send message
Joined: 22 Nov 07
Posts: 285
Credit: 1,076,786,368
RAC: 0
Message 12804 - Posted: 25 Feb 2009, 2:19:23 UTC



Had a slight problem with the network, the hub the boxes were plugged into was acting up - I shut down both machines - replaced the hub.

Total downtime - approx 1 hour.


The results of individual systems will vary from the average, thus leading to the same "credit shopping" that those who so strongly support CPP claim to wish to avoid.

CPP is a nice concept, but the current method to get there leaves a lot to be desired... IMO, the results of your testing will likely show that MW is about 1.6X - 1.8X of SETI, yet the chart is showing 0.902 right now...



Well it has been my experience that certain CPU architectures do better on different projects. E.g. SIMAP favors AMD, QMC favors Intel, Prime - Intel, and from what I have experianced over the past year during the growing pains, MW does much better on AMD chips. SETI has favored Intel's for as long as I can remember. One of the biggest reasons that I have a mostly Intel farm.

I ran a old Pent D 3.0 duo core, and a single core AMD 3400+ on MW several months ago before all the credit juggling, and the single core out performed the intel chip by more than double. In fact, it was seeing the same credit as a Intel core 2 duo 6600 (not quad). I mentioned this to Travis at the time - and of course he could not understand why. But, if you like, try it out.

Test it against a AMD quad and an intel quad of equal specs - and the AMD will do better than the Intel on this project.


.
ID: 12804 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevint
Avatar

Send message
Joined: 22 Nov 07
Posts: 285
Credit: 1,076,786,368
RAC: 0
Message 12867 - Posted: 25 Feb 2009, 18:02:46 UTC
Last modified: 25 Feb 2009, 18:07:41 UTC

I made a slight change to the Milkyway host.

During the past day or so, there have been some dry spells for work availability on MW. So far we have been lucky, and this host has not run out of cache, but I also don't want to have it happen during this test.

BOINC goes into an automatic wait mode if no new work is ready to download. If no work is available BOINC resets the request timer to a longer time period, eventually reaching 3 hours. The SETI box is fully cached and can last out these dry spells so no problem there.

So I have created a short script that cycles every 30 minutes and forces an update from the project. Similar to what you do if you just pressed the update button every 30 minutes.

This "should" eliminate the forced wait time that BOINC does if there is no work available.

So every 30 minutes BOINC will force a request for more work.
.
ID: 12867 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Labbie
Avatar

Send message
Joined: 29 Aug 07
Posts: 327
Credit: 116,463,193
RAC: 0
Message 12871 - Posted: 25 Feb 2009, 18:55:27 UTC - in response to Message 12867.  

I made a slight change to the Milkyway host.

During the past day or so, there have been some dry spells for work availability on MW. So far we have been lucky, and this host has not run out of cache, but I also don't want to have it happen during this test.

BOINC goes into an automatic wait mode if no new work is ready to download. If no work is available BOINC resets the request timer to a longer time period, eventually reaching 3 hours. The SETI box is fully cached and can last out these dry spells so no problem there.

So I have created a short script that cycles every 30 minutes and forces an update from the project. Similar to what you do if you just pressed the update button every 30 minutes.

This "should" eliminate the forced wait time that BOINC does if there is no work available.

So every 30 minutes BOINC will force a request for more work.


Sounds fair to me, I've been beating on one of mine all day to get work.


Calm Chaos Forum...Join Calm Chaos Now
ID: 12871 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kalessin
Avatar

Send message
Joined: 10 Nov 07
Posts: 42
Credit: 27,012,695
RAC: 0
Message 12875 - Posted: 25 Feb 2009, 20:01:00 UTC

Momentary standing: 6118 MW against 4551,54 Seti (added all pending)

Ratio decreased from 1.45 to 1.34 but still MW in front.
Dragons can fly because they don't fit into pirate ships!
ID: 12875 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 12882 - Posted: 25 Feb 2009, 21:55:23 UTC - in response to Message 12875.  

Momentary standing: 6118 MW against 4551,54 Seti (added all pending)


Due to what I mentioned about BOINC 3.x and 4.x clients in the mix, this is a flawed approach for SETI as the system is not guaranteed to be granted the full amount claimed.
ID: 12882 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kalessin
Avatar

Send message
Joined: 10 Nov 07
Posts: 42
Credit: 27,012,695
RAC: 0
Message 13410 - Posted: 28 Feb 2009, 22:20:50 UTC - in response to Message 12882.  
Last modified: 28 Feb 2009, 22:22:43 UTC

Due to what I mentioned about BOINC 3.x and 4.x clients in the mix, this is a flawed approach for SETI as the system is not guaranteed to be granted the full amount claimed.

But every cruncher for Seti will experience this, so in the long or medium range this does not involve in the comparism between the two projects.

In the moment: MW 12,703 Seti 9,566 (added all pending) Ratio 1.328 for MW

Not much change since the last value 1.344
Seems the slowing ratio drop is due to the slower reporting of the seti cruncher leveling out over time.
Dragons can fly because they don't fit into pirate ships!
ID: 13410 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brian Silvers

Send message
Joined: 21 Aug 08
Posts: 625
Credit: 558,425
RAC: 0
Message 13445 - Posted: 1 Mar 2009, 3:46:22 UTC - in response to Message 13410.  
Last modified: 1 Mar 2009, 3:48:34 UTC

Due to what I mentioned about BOINC 3.x and 4.x clients in the mix, this is a flawed approach for SETI as the system is not guaranteed to be granted the full amount claimed.

But every cruncher for Seti will experience this, so in the long or medium range this does not involve in the comparism between the two projects.


However for the purpose of this experiment, whether or not other hosts are impacted is completely irrelevant. The only relevance is if his host is impacted.


In the moment: MW 12,703 Seti 9,566 (added all pending) Ratio 1.328 for MW

Not much change since the last value 1.344
Seems the slowing ratio drop is due to the slower reporting of the seti cruncher leveling out over time.


I would be more inclined to think that the host here ran into results being declared invalid and receiving 0 credit combined with the high probability of a work shortage. I still maintain that the ratio should end up around 1.6-1.8, however the recent events here demonstrate why production projects typically should not be compared with alpha/beta projects.
ID: 13445 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13452 - Posted: 1 Mar 2009, 8:25:56 UTC - in response to Message 12745.  

My experience shows that MW opt SSE3 outshines Seti's opt SSE3 astropulse on an AMD FX74, but on the core 2 duo it's the other way round. Although the developers of the opt apps at Seti would deny it, their opt apps are coded (whether by design or accident) using tools (and optimisations?) that prefer Intel CPUs.


You are right in your conclusions, but incorrect in your statement about the transparency of the developer's process. At least if you're referring to the KWSN "chicken" apps. They make it quite clear in their documentation that they are using an Intel compiler which (surprise!) paints all AMD processors with the same brush (as it were) and disables some optimizations because one small run of AMD processors at one time had problems with those instructions. Some people have compiled psuedo-Intel apps to run on AMDs with great success.

Probably more importantly, the chicken apps take advantage of whatever L2 cache you have, and the processing speeds up accordingly. It seems to me that some C2Ds have as much as 4MB. I think C2Qs have something like 16MB. My Opty has 2MB (and that's a major reason why I got that particular CPU to start with). Many AMD X2s have as little as 1MB (total).

Also, I think that C2D and later processors have a set of optimized instructions that even my not-that-old Opty and the like do NOT. The chicken apps are very granular in that there are versions available for small subsets of optimized instructions, and take full advantage of the newer sets.

ID: 13452 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13454 - Posted: 1 Mar 2009, 9:18:32 UTC - in response to Message 12695.  
Last modified: 1 Mar 2009, 9:22:12 UTC

But at the end of the day I might want to know where it's best for me to crunch, which project, if I want to maximise my credits.

This is not to see what give you personaly more credits per hour but to achieve credit parity between the stock applications.

To get the data for your own machine you will have to run and watch yourself, especially as the optimisations may vary considerably for different OS/CPU/GPU setups.


1) Pray tell why are you so concerned with "parity", and, for that matter, "what gives you personally more credit per hour"? As far as that goes, are you certain that Ice doesn't have (or isn't planning to get) rigs that are comparable to the test rigs?
2) I have news for you: The performance of ANY BOINC app (stock or optimized) is going to vary with differences in CPU and OS, sometimes to a considerable degree. For example, I've heard that, on one project, otherwise identical rigs crunch a lot faster using linux versus Windows. So where's your baseline? On the linux version of the app, or the Windows version? On C2 or AMD (there are some projects that run better on either, though, frankly probably mostly on C2 nowadays)?

At one time, my "farm" was more like "garden", and consisted of an Opti 170 (AMD X2), an AMD Socket 740 Sempron (which was later upgraded to a 3400+ Venice), an AMD 3700+ San Diego dedicated cruncher (built around the former CPU for my Opty rig), an old quad PIII Netfinity server that I could only keep running a week or two at a time, and a P4 Netvista that I could only keep running a day or two at a time. The PIII actually did pretty well running the SETI KWSN "Chicken" optimized app, because, as has been noted elsewhere, it is inordinately effective on Intel CPUs, compared to AMD.

Then, because of this success, the Powers That Be at SETI decided to cut credit granted by their project to ALL participants. Twice.

So, it became essentially not worth it to bother fixing my quad PIII server yet again (and paying for the electricity to run it), because there wasn't anything I could find that would produce anything like the credits it had previously.

And the AMD CPU rigs - the most powerful rig that I own is still the Opty - were essentially crippled, even with the optimized app, because the credits they could produce, in absolute terms, were even fewer than a comparable Intel processor. Consequently, SETI gets very few cycles from me nowadays.

For my part, I am extremely interested in the results of this comparison, as it ought to provide some actual, solid proof, where we've had no small amount of rhetoric and conjecture on this subject. The only thing I can speak to with anything resembling the authority of personal experience is highly anecdotal (my RAC went down noticeably when my most powerful rig was running SETI instead of MW for a time).
ID: 13454 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13456 - Posted: 1 Mar 2009, 9:40:01 UTC - in response to Message 12804.  
Last modified: 1 Mar 2009, 9:49:48 UTC

I ran a old Pent D 3.0 duo core, and a single core AMD 3400+ on MW several months ago before all the credit juggling, and the single core out performed the intel chip by more than double.


Not that surprising. My experience, pre C2D, was my AMD64 processors, even the socket 740 one, would flat smoke any Pentium, even if the latter had a much higher clock frequency. Well, I guess I'm a little surprised that a single core beat a D, though that Venice was always a sweet chip.


In fact, it was seeing the same credit as a Intel core 2 duo 6600 (not quad). I mentioned this to Travis at the time - and of course he could not understand why. But, if you like, try it out.


Now that's surprising! Though my Opty 170 X2(2.0 sometimes OC to 2.04 or 2.2 gHz) is a bit more productive than the dual Xeon 2.5 gHz servers. I just wrote that off to the Opty being newer.

Test it against a AMD quad and an intel quad of equal specs - and the AMD will do better than the Intel on this project.

That would be really interesting, especially given that TigerDirect sometimes has Phenom X4s for $100 or so. It would be kind of hard to match them up, as the C2s have much bigger L2 caches, though it would be cool if the AMD still beat the C2.

[edit]Given what you say about the relative performance of AMD vs Intel on the respective projects; what might be even more illuminating would be to duplicate the test, but with Phenom X4s instead. If AMDs process this project that much better than SETI, then the multipliers should be pretty much reversed.

So much for "parity". [/edit]
ID: 13456 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13457 - Posted: 1 Mar 2009, 9:45:44 UTC - in response to Message 12768.  

If true, which I don't doubt that it is true, is a fundamental reason why one cannot rely on "the chart" to be the ultimate authority on "Cross-Project Parity". [snip]


I submit that one can't rely on ANYTHING for "Cross-Project Parity", because, as with all similar notions of "equality", it is an utterly unrealizable fiction.

ID: 13457 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Debs

Send message
Joined: 15 Jan 09
Posts: 169
Credit: 6,734,481
RAC: 0
Message 13467 - Posted: 1 Mar 2009, 12:56:14 UTC - in response to Message 13452.  

My experience shows that MW opt SSE3 outshines Seti's opt SSE3 astropulse on an AMD FX74, but on the core 2 duo it's the other way round. Although the developers of the opt apps at Seti would deny it, their opt apps are coded (whether by design or accident) using tools (and optimisations?) that prefer Intel CPUs.


You are right in your conclusions, but incorrect in your statement about the transparency of the developer's process. At least if you're referring to the KWSN "chicken" apps. They make it quite clear in their documentation that they are using an Intel compiler which (surprise!) paints all AMD processors with the same brush (as it were) and disables some optimizations because one small run of AMD processors at one time had problems with those instructions. Some people have compiled psuedo-Intel apps to run on AMDs with great success.


Doesn't KWSN exclusively code MB apps? Meaning the optimised AP apps are done by someone else who publicly states they do NOT use the Intel compiler, yet the executables have information in them that points to the exact opposite. That is the reason for my comment. The KWSN compiles of the MB apps are totally unrelated to what I said.
ID: 13467 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile arkayn
Avatar

Send message
Joined: 14 Feb 09
Posts: 999
Credit: 74,932,619
RAC: 0
Message 13498 - Posted: 1 Mar 2009, 17:29:03 UTC - in response to Message 13467.  

The current KWSN apps are based off of the work that Alex Kan did for OSX optimized apps.

JDWhale took the code apart and made it work on Windows and Crunch3r then compiled the Linux apps.
ID: 13498 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13501 - Posted: 1 Mar 2009, 17:55:17 UTC - in response to Message 13467.  

Doesn't KWSN exclusively code MB apps? Meaning the optimised AP apps are done by someone else who publicly states they do NOT use the Intel compiler, yet the executables have information in them that points to the exact opposite. That is the reason for my comment. The KWSN compiles of the MB apps are totally unrelated to what I said.


Well, it appears to me that they post both MB and AP apps, though I'll be the first one to admit that my recollection could have been faulty, or that my understanding of how this all works could still be faulty. A look at their "Source and Downloads" page seems to indicate two authors of the various versions (one based on source by a third author), including a link to a page by "Crunch3r" for one (linux) "flavor".

It could be that you are referring to the latter as being the person that doesn't use Intel compilers. I didn't find that with a quick glance at his site, and that doesn't mean that it's not there, or that he hasn't talked about it elsewhere.

A quick glance just now at the KWSN "How to make your own optimized Seti@Home client for Windows"" page indicates that he tells how to obtain an Intel compiler to compile the app.

My own "druthers" were to get an AMD compiler and use their libraries to compile a truly AMD-optimized version of the app. As it turns out, since I last checked (which was quite some time ago), they have come out with a newer version for AMD/linux that takes proper advantage of SSE3 on AMD (which Crunch3r states on his site doesn't help much). It's not immediately obvious to me which type of WU this is for.

On a related note, someone has come up with a very cool-looking hack (in the traditional, positive sense, not meaning "vandalization) to run on the CPU and CUDA at the same time.
ID: 13501 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lloyd M.

Send message
Joined: 1 Dec 08
Posts: 139
Credit: 8,721,208
RAC: 0
Message 13502 - Posted: 1 Mar 2009, 17:59:59 UTC - in response to Message 13498.  

The current KWSN apps are based off of the work that Alex Kan did for OSX optimized apps.

JDWhale took the code apart and made it work on Windows and Crunch3r then compiled the Linux apps.


Uh, OK. That isn't what it looked like to me, and I guess I just don't get it.

ID: 13502 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevint
Avatar

Send message
Joined: 22 Nov 07
Posts: 285
Credit: 1,076,786,368
RAC: 0
Message 13505 - Posted: 1 Mar 2009, 18:10:13 UTC - in response to Message 13445.  


I would be more inclined to think that the host here ran into results being declared invalid and receiving 0 credit combined with the high probability of a work shortage. I still maintain that the ratio should end up around 1.6-1.8, however the recent events here demonstrate why production projects typically should not be compared with alpha/beta projects.



As far as I know , neither of these have happened. You can review both hosts, to determine if there have been any zero credit - true MW has quick purge, but I have checked occasionally, and see none of the zero credits on ether box. And even during the work shortage at MW, the host did not run out of work.

I still have not decided on a shut off for the test, this should have been something decided before starting test. But should we say next Sunday 12:00 MST.

I will stop both boxes at the same time, then we will have to wait until all pendings are added up..

.
ID: 13505 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile banditwolf
Avatar

Send message
Joined: 12 Nov 07
Posts: 2425
Credit: 524,164
RAC: 0
Message 13506 - Posted: 1 Mar 2009, 18:16:52 UTC - in response to Message 13505.  

But should we say next Sunday 12:00 MST.


12:03:04. :P
Doesn't expecting the unexpected make the unexpected the expected?
If it makes sense, DON'T do it.
ID: 13506 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kevint
Avatar

Send message
Joined: 22 Nov 07
Posts: 285
Credit: 1,076,786,368
RAC: 0
Message 13507 - Posted: 1 Mar 2009, 18:18:47 UTC - in response to Message 13456.  


That would be really interesting, especially given that TigerDirect sometimes has Phenom X4s for $100 or so. It would be kind of hard to match them up, as the C2s have much bigger L2 caches, though it would be cool if the AMD still beat the C2.



I was going to add in an AMD Phenom x4 into the mix, but for some crazy reason it will not load WIN-64. Only linux I have tried everything to get the damn thing to load WIN-64 - SO, no AMD x4 Testing.

I don't have that many AMD boxes - I think I only have 6 so I am limited to what I can do there.
.
ID: 13507 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : Crunchoff MW vs SETI

©2024 Astroinformatics Group