Let me tell you a little story - the story of how I came to be involved in beowulfery in the first place.
One's goal in a Monte Carlo calculation (at least the ones I do as a physicist) is to importance sample a Markov chain driven by a suitable stochastic selection rule. The result is built up from many, many ``independent, identically distributed'' samples generated by the Markov processC.1
However, the Markov process does not in general produce completely independent states for sampling in each step. In the problems of interest the number of steps that must be taken to generate one independent sample tends to increase with the size of the system studied (according to a scaling law of its own). It can take a lot of samples to generate high precision results for large systems which have the greatest relevance to the physical phenomena one is simulating. It can take a very large number of steps in the Markov process to generate this large number of independent samples.
By ``large numbers'' here I mean that at one point five or six year ago I ran the calculation on over 100 Sun Sparc 5's for nearly a year, continuously. This added up to well over 2 GFLOP-years of computation. I'd be running on them yet if it weren't for the fact that Solaris (2.3) proved utterly incapable of managing my calculation in the background and a foreground X session simultaneously and even more unfortunately, my background process usually ``won''. This irked the students at Duke University that these compute clusters technically ``belonged'' toC.2. So I got booted from the student clusters and immediately started to design Duke's first beowulf, before I even knew that the beowulf project existed.
The design process was simple. SunOS might have worked as an operating system for the cluster - I'd used Sun's and PVM and expect for years at that point doing these calculations and with SunOS they ran nicely in the background without annoying users working at the console. Solaris was out; I still haven't forgiven Sun for taking an operating system that, really, was no worse than linux 2.0.x and transforming it into something hideous. Linux I'd used and had marvelled that it was so wonderfully functional - every bit as good as SunOS (which at the time was arguably the world's best operating system - I'm not dissing linux at all).
Intel Pentium Pro's had also just been introduced and really for the first time could compete with any of Sun or SGI or DEC's affordable workstations. So I added a linux-dual PPro cluster onto our next grant proposal. It was actually something of a relief to discover the beowulf project (via a visiting physicist from Drexel). I no longer had to wonder if linux was sufficiently stable to support a compute cluster as it had already done it.
However, I didn't have to worry too much, because my task was ECG. I had managed it for years using an unholy mix of submitting jobs one at a time on lots of hosts on our lan, then as a master-slave tasks (see below) via PVM, then (to avoid loading the campus network backbone, which I was far more worried about than loading Sparc 5's that had cycles to spare compared to my desktop SunOS Sparc 2 that could run my calculation in the background and X in the foreground without difficulty) via expect scripts and /bin/sh scripts. I knew that as long as I could construct a system that had a network, some reasonably fast floating point, and rsh that I could get useful work done.
Finally, the systems arrived, I popped linux 2.0.0 in Slackware onto them (2.0.0 was still so new that it crinkled when you rubbed it, but it did support SMP operation and I had bought dual PPro's exclusively) and, after a week or two of pounding my head against the hardware trying to get an Adaptec 2940 to workC.3I'd learned a lot about the kernel, had for the first time in my life managed to actually break a filesystem by (necessarily) shutting a system down without syncing it, had gotten pretty good at the Slackware floppy-based install, had learned to flash an Adaptec BIOS, had moved up several kernel revisions to 2.0.11 or so, and had a very small stack of dual PPro's on switched 100BT (which at that time was very expensive) running a parallel calculation.
I'd also joined about six linux lists, including at some point then or soon thereafter, the beowulf list. The beowulf list proved to be a godsend. There were other people out there who were using linux-based COTS computers to build supercomputer-class clusters. They were going through the same things I was. Making the network cards behave (some didn't). Deciding whether single CPU boxes were ``better'' or ``worse'' than dual CPU boxes (answer: it depends, as discussed in the chapter on bottlenecks). Learning to be envious of those who had really big piles of systems with snazzy superfast networks like Myrinet (in spite of the fact that I, at least, didn't really need a superfast network).
The folks on the beowulf list have taught me, step by step, most of what I know about cluster computing. Sometimes the lessons were pleasant and fruitful, other times I got by skin blasted off by nuclear flames, but both ways I learned and eventually got to where I could contribute some of what I'd learned back to the list.
In case I haven't made it sufficiently clear yet, I am not a real computer scientist, I'm just a humble physicist. There are plenty of folks on the beowulf list who are real computer scientists, some of them Bell-prizewinning scientists at that (I've seen the one hanging on Don Becker's wall, for example). To all of them I am humbly grateful, and to them I dedicate this bookC.4.