Has anyone built a 4-card HD4870X2 machine ? If so, what are the pitfalls to watch ?

Are there other projects besides MilkyWay and FAH that specifically target ATI cards ?

I am weighing the pros and cons of building a system like that. (alternate considerations are quad HD5000, to be released coming friday or the upcoming G300) I just need the info on the ATI cards, I already have all the other components.

---

Team Bletchley Park.

Well the biggest issue will be cost. If you are able to obtain the parts for system like this or you are somehow sponsored or can otherwise justify then I dont see why you couldnt try it. But for most people buying four of one of the most expensive cards on the market is too much. Judging by your profile I assume you have $$$ to burn for whatever reason...

The biggest problems will probably be power consumption. A single 4870x2 uses up to 275W while gaming. However, you can underclock the memory to save power (I have not had time to measure the saving from the AC power with my meter but judging by the current draw drops in GPU-Z it should be about 15 to 20% less power consumption. Note that memory speed does not really affect MW so you can actually just leave it on 500mhz. (I have tried it - with a stopwatch, at 900mhz and 500mhz and there is no difference in work unit time.) You can do this easily in CCC just with the sliders OR you could edit the BIOS of all the cards.

Assuming you got it down to 250W each at ~100% load on both cores on all 4 cards you are still left with 1000W power consumption. There are only a few PSU's which exceed this (and are any good) so either you will want to have a very power efficient motherboard/ram and perhaps not too many other things in the system OR you could try setting up a dual-PSU system and have say two 750+W PSU's etc.

Unless you live where it is epic cold you will either need vent the heat outside or run aircon constantly as your computer is, as you say in your profile, a heater. Except it would be worse in this build.

Of course, you are going to run into all kinds of problems try to get any GAMES running with all 8 cores... however as far as I understand, milkyway should detect all 8 and start using them.

I suppose there could be another problem - if you have 8 cores and also are running two work units on each (preference setting = n2)(recommended) then you are going to be crunching 16 work units at once. This will take about one minute to 1:30 to complete 16 (or less if you overclocked all the cores). Because boinc only communicates every two minutes with the MW server for more tasks (at least on my setup) - and it only has up to 24 tasks at time - a system with 4x4870x2's might process more tasks than it can download. But i might be wrong, maybe if it runs out of work it speeds up the communication interval at least temporarily and will download more work.

If you think you can justify spending a fortune on this build and only intend to use it for milkyway then go for it. I am not aware of any other BOINC projects that will run 8 cores. Mabye GPU grid? Folding@home should.

I would also find it funny if you built it and then MW project shut down. :)

Running four of the best 5000 series cards would just be insanity. I cant recommending spending the $4000+ AUD to purchase the cards for a system like that. (But the 6.0 teraflops per card - compared to 4870x2's 2.4 - is nice)(figures from leaked specs sheet from a few months back).

---

Edit: (where has the edit button gone?)

I have measured the AC input to my computer it seems my estimates were a bit off. I might make a post about it later because its still a small saving.

I am sure though that the 275W i mentioned in my last post must have been a TDP value since the actual is no where near that high (MW adds only 150W AC at default settings).

Unfortunately it is difficult to determine the power consumption of the card... I am not sure if simply finding the wattage of each core (VxI for each set of stats in GPU-Z etc) represents the true power consumption of the card. And I'm not aware of any simple physical way to measure it. Even measuring AC power consumption and trying to convert that to a DC value is difficult especially since powersupply conversion efficiency is non-linear.

You could probably get away with a single high power PSU. I suppose they're all overload protected anyway and you probably have a few lying around. Worth keeping a close eye on the PSU's voltage stats while under load though to ensure its providing stable power.

---

Laviathan, Thank you for your detailed post, much appreciated.

I would also find it funny if you built it and then MW project shut down. :)

Yes that would be hilarious :-) I had a good laugh on that one :)

For the hydropower machine I use 4 700 W powersupplies, with a sync box for the on-off. Power consumption fluctuates with all cards running per project, for GPUGrid it was around 1300 Watts, for MW it is stable at 737 Watts for the entire machine with a peak to 1000, I have a power meter attached. For the new setup I would use two 1000W Silverstones, I have elaborately researched the various models and types.

Money to burn, well, not really but a careful consideration. Some people go on vacations, I save the money and build rigs like these.

I actually do run it as a heater, starting september temps go down here in Holland, until around april with freezing winters so the heating cost saved will partially compensate for the energy bill. Green crunching so to speak.

I will not be running games on it (loud 'ahhhh' and head-shaking from the audience).

Interesting point with the workunit turnaround times. I do think that Boinc lets you keep a 'supply' of work measured in time rather than workunits, so it should not be too much an issue. GPUGrid used not to supply spare workunits until it became apparent that the project would benefit from it, now there is an optimal supply of work units.

6 TFlops per card, now, that IS fast.

From the list of top computers in MW, I see that UL1 has quite a few 16 CPU machines, I wonder what GPUs he is using. I'll have to ask...

Ah, AL1 says in a post

I'm sorry, but I have to correct you on this: I'm running 'only' three single 4870s, no 4870X2s. I once had an 4870X2, but I wasn't able to get the GPUs to run at more than 50% load each and so it turned out to be slower than a single 4870. I also tried running four single 4870s on one board, but this only looked fine on paper...in reality something slowed this configuration down and therefore it was less productive than using three cards...

there you go.

Regards,
Hydro

---

Team Bletchley Park.

Hydropower? Holland? Irony alert :)
But don't you need hills for hydropower?
Unless you live in south Limburg where you can occasionally coast for a distance on your fiets without building up speed beforehand.
But even there it would be really marginal for hydropower.
Here in Australia we have hydropower. We have hills, dams, power stations, but rarely enough rain to fill the dams to generate the power. And that's in the wetter south-east part of the country. So most of our electricity comes from coal.

---

Cheers,

PeterV

.

While testing my power consumption today I also noticed that the power draw fluctuates. There seems to be a average value and then every 15 seconds it will nearly double for a split second before slowly going back down to the average and occasionally it goes a bit lower. My measurements were an average of 25 readings..

My current budget and situation does not let me spend huge amounts of money on computers as much as I would like. Not nearly as much as I see some of the top contributors to this project are.

I use my computer equipment as a heater too. However I live in a subtropical zone so most of the year it is actually a bad thing.

I am trying to get my hands on another cheap 4870x2 and plan to edit their bios heavily and try to get into the top 20 user RAC for this project.

Its probably a good thing your not planning to game on it. I was just reading about the scaling of 4x 4870x2 systems and it appears that in almost all games there is VERY little benefit to having a second 4870x2, and in some cases a single card is better. Even fewer games can properly use 3 and even then the benefits would probably be very small. And it seems running a 4th card results in extreme bottlenecks and the entire system becomes limited by the CPU (no matter how expensive it is or how high it has been overclocked).

In regards to the list of top computers in MW, I'm more interested in how some users have got 64 processors on one system. Must be a bug, system info hack, or he works for intel and procured a prototype engineering sample :P

Verstapp, another problem (aside from no rain) is that most of the significant hydro power resources here have already been tapped. Also, it is often not simple or economical to convert a non-power-generating dam into a power generating one.

Maybe in a few decades we'll have some nuclear power stations here and we wont be paying $0.20 per kW/h and wont be destroying the environment at the same time.

---

But not until we've burned all the coal... or shipped it all overseas.
By which time the climate will be so screwed up that Eucumbene will be full one year, empty the next.
>laviathan
Big fish in a small toilet? :)

---

Cheers,

PeterV

.

Hello Peter, Verstapp sounds like Verstappen, known from the area around St. Odilienberg. 'Ich hub och ne tied in roermung en hearle gewoend'. :)

Love Australia. I visited last year for work, had a good view of Sydney and surrounding areas, Adelaide, a quick stay in Darwin and Alice.

The Hydropower machine was supposed to run on watercooling, hence the name, until I found out that the air did just as well and saved over 1K Euro on the build.

---

Team Bletchley Park.

Hi Hydro,

Yes, it is Verstappen, but this branch is from Gennep with some others in Venray. No relation to Jos the [ex-]F1 driver.
The shortened version is because one of my previous ISPs only allowed 8 character logins.
I had thought of going hydro too, until about 4 years ago someone introduced me to Zalman and I've never looked back.

Anyway, see you around. PM me if you want.

---

Cheers,

PeterV

.

Has anyone built a 4-card HD4870X2 machine ? If so, what are the pitfalls to watch?

Some demo systems featuring 4 HD4870X2 cards where shown last year. But I don't think there are many systems around, as crossfire is limited to 4 GPUs by the driver (either four single GPU cards or two X2 cards). But you want to disable crossfire either way, because it gives better performance for GPGPU applications (memory content of the cards doesn't need to be mirrored). So I don't know if this limit to 4 GPUs applies also for a configuration as individual cards. But one year ago I have seen some complaints that the Linux driver does not support more than 4 GPUs. No idea if that has changed or if it is different for the Windows driver.

Generally, I would recommend X2 cards with 4 video outputs (two per GPU). Some people report serious problems deactivating crossfire on cards with only 2 outputs. As both connectors hang on one GPU, it can get hard to use both GPUs on such a card without crossfire (driver gets not loaded for the GPU without connectors). With a single card one can usually work around this by setting up the primary display on the GPU without connectors (remote access to the machine by VNC recommended, as one may not see anything on the connected monitor afterwards). Having X2 cards with 4 connectors and putting on a dummy plug for each GPU should be the most reliable solution. At least that works for two X2s. Maybe someone like UL1 has something to add here. He (and some others too) runs again 4 GPUs and not only 3, as one doesn't loose that much with multiple GPU setups lately (app and maybe also the driver has improved in that direction).

I guess you know alredy something about the general problems associated with building a stable system running 4 graphics cards with 8 GPUs sucking something close to 1 kW (83A @ 12V !) out of the PSU(s), not even looking to the remaining parts of the system. So I won't tell you anything about that ;)

Are there other projects besides MilkyWay and FAH that specifically target ATI cards ?

Collatz@home comes to my mind. CPU, CUDA and ATI are supported there. Their ATI app is not supporting multiple GPUs very well yet, but there is already a new version on the way, which will bring it to the MW level (actually even slightly in front of it ;).

@laviathan: yes, that 64 cpu thing had caught my eye too. I can only imagine they rigged the response, use a virtual system and clustered it or they use real systems running in a cluster configuration.

@Cluster Physik: from the NVidia adventure I can tell that over there the max number of GPUs supported is 8, eventhough SLI supports only 4. I would expect ATI to have done the same if the demo systems have 8 GPUs. I'd be running it under XP64.

Which brands would have those 4 outputs ?

Yes, the powersupply jungle is treacherous :) Finding the right one is an important key to success. Hydropower works very well generally.

Thanks for the pointer to collatz.

I read more on multiple workunits per GPU for ATI, Does the current CUDA MW app support multiple workunits as well ? If so, how do I activate that feature ?

---

Team Bletchley Park.

Which brands would have those 4 outputs ?

I thought Sapphire had some, but now I can find only the corresponding HD4850X2.

I read more on multiple workunits per GPU for ATI, Does the current CUDA MW app support multiple workunits as well ? If so, how do I activate that feature ?

No, it's not available on nvidia cards. And it may be also removed from a future ATI version as it brings only 1 or 2 percent performance. But it complicates the application and it gets basically unnecessary when we have ATI support from the BOINC client.

Just found the sapphire link 5 minutes ago :) (thanks)
http://www.tomshardware.com/reviews/radeon-hd-4870-x2,2073-2.html

I initially found a link to the HD4850x2, which had 4 connectors, but that seemed to blow the hot air into the case, not something I'd want.

A pity for the multi-wu capability. I could imagine that the two or more GPU threads would run alternating to the main CPU performing tasks on the other thread. Like pipelining in a CPU, this wouuld be pipelining tasks to, in and back from the GPU. fill the bucket, give the bucket to the firefighter, empty the bucket by the firefighter, hand back the empty bucket. 4 stage pipeline.

---

Team Bletchley Park.

It makes sense there would be a problems trying to run two 4870x2s in crossfire (on 2 video out cards).

I have never seen a 4870x2 with 4 outputs though.

I am confident that if you spent alot of time tinkering with driver configurations and settings you could get all 4 cores running MW on two dual-output cards but it would probably be alot of work (I'm willing to try it if I can get a second card). I plan to get a second 4870x2 from ebay and if I fail getting it to run then I can sell it on ebay again and should get my money back.

I know the F@H people have had nightmares trying to get it to work on all 4 cores. But that was a few months ago when Catalyst 9.3 was only beta. (By the way, they never got it going as far as I know).

---

I just got a 2nd 4870x2 for $130 and will let you know if I can getting it working nicely on all cores. But this may take some time :P

---

I am confident that if you spent alot of time tinkering with driver configurations and settings you could get all 4 cores running MW on two dual-output cards but it would probably be alot of work (I'm willing to try it if I can get a second card). I plan to get a second 4870x2 from ebay and if I fail getting it to run then I can sell it on ebay again and should get my money back.

I know the F@H people have had nightmares trying to get it to work on all 4 cores. But that was a few months ago when Catalyst 9.3 was only beta. (By the way, they never got it going as far as I know).

I just have seen that someone (I know and trust) runs a HD4870X2 in crossfire without loosing significant performance at all. He runs Win7x64 with Cat 9.8. In fact, ATI promised some months ago their driver team would do something about the crossfire issue in GPGPU applications. Looks like it paid off already.

One would not want to crossfire all cards. But maybe it runs quite well with several cards when each card has only its internal crossfire activated but not between the cards (just don't use the crossfire bridges)?

Well, I have a Dual Sapphire 4850X2 setup with two unpopulated PCIe slots waiting to be populated.
I recently updated to Catalyst 9.8 and I'm running it on Vista 64 Bit with 8 GB RAM and a 940 Phenom X4 940.

I've tried with "dummy plugs" on the 9.3 driver but that didn't work well (only 2 out of three dummys worked)

Currently I'm going crossfire and CCC and GPU-Z report only 80% load on each GPU.
I definately will go for more gpus in that system and will disable crossfire in the future.

For the PSU Issue, I currently have a OCZ 700W PSU which works fine and I have another 500W PSU that I will use in addition to the PSU inside my system.

CP already has given me great feedback on my setup and I'm sure there's much to tweak.

Also, the metal which covers the entire card really is a bad design in my opinion. I removed it from one of the cards and the gpu temp went down 10°C.

ttfn,

Sysfried (you can find my host in the top 20 right now)

Thanks for that info, I have heard that with Vista it is hard to get cards working right (also NVidia) so great to hear you actually have two double cards running. I am amazed you can get by with the 700 W.

Actually, I found no problems with Vista at all...

Vista 64 Bit Enterprise Edition
ATI Drivers
and then follow the manual with the file copy you have to do in the windows\system32 folder.

That did it.

Even Boinc 6.10.0 & 6.10.3 work without problems (and of course the older 6.something )

Cheers,

Sysfried

I assume you've seen this thread: http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=789

The guy here uses 8 nVidia GPUs (4x295) for folding.

In terms of ATI card related fun-ness, there shouldn't be a barrier when it comes to 4 X2 cards going full pelt (as long as the conditions, air, power etc are good).

---