I was thinking about how easy it is to get old computers for pretty much nothing or very little money. "Hey, why not get 16 or so old PCs and put them together in a Linux cluster!" I thought last night.

First off, how hard is it to set up a Linux cluster for a Linux newbie? I would think if I got 16 P2 PCs at 250 MHz and 64 MB of RAM going, that would be a ton of computer power! 250 MHz times 16 gives you 4 GHz! For RAM I might use the clusterr as a web server to handle the digg effect if I could get fast enough Internet or just to play around with.

I know I have the space. The heat build up is a whole different issue though.

Can anyone tell me how hard Linux clusters are to set up and use? Will all Linux apps work with Linux Clusters or do they have to be specially coded? If so, what apps will work out of the box?

Quote: Originally Posted by Sleepy_Sentry I was thinking about how easy it is to get old computers for pretty much nothing or very little money. "Hey, why not get 16 or so old PCs and put them together in a Linux cluster!" I thought last night. First off, how hard is it to set up a Linux cluster for a Linux newbie? I would think if I got 16 P2 PCs at 250 MHz and 64 MB of RAM going, that would be a ton of computer power! 250 MHz times 16 gives you 4 GHz! For RAM I might use the clusterr as a web server to handle the digg effect if I could get fast enough Internet or just to play around with. I know I have the space. The heat build up is a whole different issue though. Can anyone tell me how hard Linux clusters are to set up and use? Will all Linux apps work with Linux Clusters or do they have to be specially coded? If so, what apps will work out of the box?

clusters
check these links

http://www.clusterbuilder.org/pages/...question-6.php

http://www.aspsys.com/software/operating_systems/

http://www.gridbus.org/~raj/csc433/ClusterOS.pdf

http://www.centos.org/

Also ** NAME DROPPING TIME** you might try and ask GruntMan from over at Gruntville for some advice as clusters are his business.

I built a Linux Cluster about a year ago with xbox's! It was fun! Anyways its not hard. And here is a good guide to help you out.
http://www.ram.org/computing/linux/linux_cluster.html

rocks clusters is also easy to set up....

http://www.rocksclusters.org/wordpress/

One of my teachers claims he tried CentOS with his son (can you guess which teacher this is, Sir.Smoke.Alot?) and said it was very easy and they had a small cluster up and running. He said he had problems "controlling" them, but he couldnt seem to go into detail. Im not sure how accurate this is, though because today while some people and I were looking at an add for a Sun Fire T1000, he walked by and claimed that he used to work on those 20 years ago. EGOMANIAC!

Sleepy, I asked GrunTMaN about your questions and here is his answer, He builds clusters for a living so he's definatly someone to listen IMO.

Quote: It would work fine, the problem for him will be efficiency. Like he said, it's 16 computers that add up to 4GHz. In other words it is 16 computers doing the work of one, slower. The fsb is only 100, he'll be connecting them through 10/100 at best. So a slow bus, with slow ram and slow interconnects is going to leave him with a very slow cluster that will break the bank to run. If he lives at home, he better ask mom and dad before using a 36.4 Amps of power 24/7. If he is a novice and would like to do it for the fact of learning it, I would also recommend Rocks to do it with. There is no set-up or config, it just loads up and works, or not Then when he is more familiar with what is involved, he can move to suse (I say Suse because they offer a lot of support for clustering hardware and interconnects and such, most Linux OS's will work but Suse has most of what you need built in.)and configure them and the compilers himself and try out some of the software packages to tweak it like torque and what not. Feel free to quote me, i didn't have time to look up my Apex log in

I like this cluster (http://mini-itx.com/projects/cluster/) he used linux and gave a little bit of help how to do it.

There is a ClusterKnoppix live CD available for download. Very easy to use. Runs off the CD.

(don't know if there is an actual option to install it permanently, but we did some testing with it. Pretty spiffy.)

Depends on the specific application that you want to run though. When we did the testing with ClusterKnoppix, on two Dell Dimension (I think that they were GX280s or something like that), we had to load it up with 8 processes in order for the Cluster part of it to kick the processes over to other systems (for a 50% load balancing). Below that, it stayed on the local system, unless it ran for a long time, then it sent the whole thing over to the slave system.


Sometimes, if you're doing multiple things, it might be better just to do them as separate independent machines.

For something like F@H or anything computationally intensive, it won't really speed it up much because as far as I know, programs like that aren't made to be able to be split into multizone/multidomain for distributed processing.

i.e. "If it's going to take 8 hours, it's going to take 8 hours."

Quote: Originally Posted by HigHTecHReDNecK Sleepy, I asked GrunTMaN about your questions and here is his answer, He builds clusters for a living so he's definatly someone to listen IMO.

Thanks a ton man! I would give you rep, but apparently I need to "spread some rep around" before I can give to you again.

And now that I think of it, I might run into some problems if I ever do cluster a large amount of PCs. I don't want my power company to think I'm growing marijuana under plant lights and call the police .

Hey, I actually work on the tier 2 project at ucsd. So i've spent most of the last year building and mantaining large and cumbersome clusters, and i will tell you right now with all my experience that the rocks clusters are generally the best idea (i'm not just saying it because sdsc develops it). If you use rocks 4.x you're in for a treat, very easy setup. Assuming you have the hardware setup and your networking squared away, you could have your cluster running in three hours. Though i'll advise you now, i recently had to build a 10 node cluster out of old p3 500 mhz processors and it was a nightmare. The parts you'll be working with are just too old and too unmanageable. It's possible, as i've done it, but not fun, and it turned out to be barely worth the time. And just to clear it up, rocks is essentially a highly modified centos build, you can technically use a centos os disk as a base for your rocks installation. So if you do build a rocks cluster centos is awesome for any multi home machines you might set up.

Anyone interested in building a cluster should definitely look into rocks. Once you learn it well enough you can do some amazing things, and setup some badass clusters. What's most important is to learn how to build custom rolls and kernels, once you do that you can do just about anything you want with rocks.

P.S. if you're a newbie to linux this may not be a great idea if not simply because cluster computing is not fluff and fold. It's a field that's constantly under development, and as such there's not alot of automation or documentation as alot of other things. Rocks clusters are about as user friendly as clusters can be, but there is still a learning curve. Once you have your cluster you'll have to learn how to manage your jobs, you'll probably want to get a fairly fast network going as it can be a huge bottleneck (we're hooked up to the teragrid), and you'll have to know your networks inside and out as cluster computing is largely networking. If it's just for fun then, go for it, it would be a great learning experience.

Also information at ClusterMonkey.net. I need to get off my butt and build one too.