Status of Fly

Here’s the post with all the pics. For the two people who are going to read it. But hey, I took all the pictures, so I might as well put them somewhere.

The blades don’t have UPS backup, but oh well. Everything else does now, thanks to the freed up UPSes from the old nodes.

More details on software and hardware in the next post, but there’s the pics. All I’ve got time for today!

I was trying to figure out how to do this cross-PXE kickstart thing, because the nodes in rack 1 of the cluster don’t have CD-ROM drives. You can cross-kickstart in ROCKS, but not with PXE; you have to actually go to each node and boot it with its appropriate boot CD. I searched the web and found a post on the subject by Brandon Davidson, a (the?) Systems Administrator for the University of Oregon Neuroinformatics Center. I emailed him and asked him if he had gotten it working. He emailed me back saying that he was going to make a virtual instance of ROCKS 5.1 and make me a new base roll…. and a few days later, he had the .iso up for me to test. I haven’t tested it yet, but he did, and I have high hopes that it will work. Thanks Brandon!

There will be pictures later, but for now:

-old 2GB nodes are out

-new blades are in, in chassis, electrical and network all done

-Opterons are also racked up, with power and network

-Cabling is all worked out and much neater than it’s ever been!

Now I’m going to get the remote console for the blades working, fix the broken cluster nodes (compute-1-11 just died, I hope it’s a power supply), add the Opterons to the existing cluster as rack 3 (they work out of the box), and wait for Chris to be done comping so I can reinstall the cluster and make sure the blades work with ROCKS 5.1 (they don’t appear to work, at least out of the box, with ROCKS 4.2.1, which is what we currently have installed).

The Opterons I speak of are the five free 2×2.0Ghz/4GB nodes from Jared. They appear to be about as fast as the nodes I just removed from the cluster, but they have more RAM and are 64-bit, so I guess I’ll use them.

Dan and I went up to Dartmouth last week and got some new hardware: the previously mentioned blades, as well as some (free!) older Opteron machines that are nonetheless pretty decent and will probably be added to the cluster as well: four dual 2.0Ghz (I think) machines with 4GB of RAM each. These are, of course, x86_64 (as are the blades), so, since all of rack 1 is also 64-bit, now I kindof want to create an x86_64 distro so that we can take advantage of the faster speed and greater memory-per-process capabilities of 64-bit RenderMan in particular, but probably other things as well. This is going to require some hacking, as the head node is going to remain 32-bit, just because there isn’t any other node I can stuff enough disk into. ROCKS has support for this, but only if you boot the nodes from CD the initial time… I don’t have CD-ROM drives in many of my nodes in rack 1, so I’m going to have to hack the PXE boot… not a big deal, I don’t think.

Right now I’m waiting for a new plug in the server room, because though I checked whether I had the right plug in there already, I didn’t check the voltage…. the ones in there are 120V and I need 240V. D’oh! This should happen tomorrow, theoretically.

On my plate now is to remove all the old 2GB nodes from the rack and use their spare parts to fix the 4GB nodes that are down, and then physically place the new nodes in the rack.

Then I’ve got to back up the old config and install ROCKS 5.1

Fun! I’ll keep posting as I make more progress.