AMD Processors
Decrease font size
Increase font size
Topic Title: How to build your own Screwball Farm
Topic Summary:
Created On: 08/14/2005 01:24 AM
Status: Read Only
Linear : Threading : Single : Branch
Search Topic Search Topic
Topic Tools Topic Tools
View similar topics View similar topics
View topic in raw text format. Print this topic.
 08/14/2005 01:24 AM
User is offline View Users Profile Print this message

Author Icon
Mime
Troll Hunter

Posts: 8517
Joined: 10/06/2003

Along with the bins of cables, manuals, old heatsinks and various other junk, spare parts are something that geeks tend to acquire over time. However, unlike the other junk spare parts can actually be useful if you have enough of them and put them to the appropriate task. I decided that the appropriate task was to build a network of diskless terminals dedicated to running Folding@Home. This guide is intended for any and all of my geeky bretheren who might be interested in doing the same.

An obvious question at this point is why I called it the Screwball Farm. The answer to that is exactly the reason I explained above. The entire network is built almost completely out of spare parts which had been collecting dust on shelves and filling up empty space in various bins and drawers for an undetermined amount of time. As a result many parts of the network don't always work as expected and sometimes stopped working altogether. Nonetheless, I decided it was better to put up with these quirks and hiccups in order to put the parts to some kind of useful work and thus the Screwball Farm was born. If you're interested in seeing how a project like this can develop in a "real world" setting you can check out my project thread below.

http://forums.amd.com/index.php?showtopic=34972

Prerequisites for the guide
I was basically a Linux newbie when I decided to build my folding farm, so it is possible for Linux newbies to build their own farm like I did. However, in order to prevent this guide from being even more scary-big than it already is, I'm not going to explain the basics of using Linux. If I say you'll need to enter a certain command "as root" or "you'll need to edit your /etc/hosts file" you'll need to know what that means. Also, I'm not going to explain much about basic tools like cd, cp, ls and su, so if you don't know what any of that means you'll need to check out a tutorial on basic Linux commands. Feel free to do so and follow this guide at the same time, but be aware that it's going to take a lot more work to learn Linux and LTSP both at the same time.


Before we get started...
As I usually do I'll be providing a little background to help get things started. For those familiar with Linux and networking or those who already have a machine with a working Linux distro installed you can skip ahead and go directly to the hardware setup for each terminal.

You'll need to be logged in as root in order to edit many of the configuration files involved in this project. There's a general rule in Linux-dom that root access is to be used only when necessary. One reason for that is to prevent someone from making a simple mistake that ends up hosing the entire system. Also, Linux expects users with root access to know what they're doing, so you won't be prompted with as many "are you sure you want to do that?" messages as you usually are in windows. I don't mean to scare anyone off, and there are some guards in place, but the moral of the story here is that when you're logged in as root you will be able to mess things up that you really(really! do not want to be messed up. A little caution can go a long way.


Picking a Linux distribution for your LTSP server
When it comes to picking a Linux distro a lot of it is just personal preference. Some people like Fedora, some prefer Ubuntu, and others are partial to more "unix-like" distros like Slackware and Gentoo. All of them should make for a solid system once properly installed. In this guide I'll be using Fedora Core 4, but it's been my attempt to provide a guide that will extend relatively easily to other distros as well, so hopefully you'll be able to follow along without much trouble if you decide to go with something else. Package names and/or versions will probably be different but the overall procedure should stay more or less the same. In the cases where there are significant differences I've provided some links below to alternate install guides targeted for specific distributions that I've used while installing LTSP.

LTSP on Slackware
http://www.linuxquestions.org/...s/sh...threadid=234286
http://wiki.ltsp.org/twiki/bin/view/Ltsp/Slackware

LTSP on Gentoo
http://www.gentoo.org/doc/en/ltsp.xml
Note: At the time of this writing the ltsp ebuild(net-misc/ltsp) creates an invalid directory structure in /tftpboot which breaks net-booting via PXE. The correct directory structure is explained in the LTSP wiki. Also, the ebuild script will install LTSP to /opt/ltsp-4.1 just in case you might already have a different version of LTSP installed at the default location of /opt/ltsp. If that's not the case, then it may be helpful to create a symlink at /opt/ltsp that points to this directory.

LTSP on Ubuntu
Currently there's work being done to integrate LTSP directly into ubuntu. It's a completely different kind of LTSP than what I describe below that has the upstream codename of "MueCow". The overall install procedure looks like it'll be simplified a great deal, and be more easily supported which can only be good things. However, at the time this guide was last updated(August 2006) the project is still very much experimental, so I'm not recommending it to anyone who doesn't have a strong background in linux. For those who do and are feeling a bit adventuresome feel free to poke around at the Ubuntu Linux forums(linkage) or the Ubuntu Wiki(linkage) and give it a shot.

If anyone comes across a similar case with a different distro let me know and I'll update the guide accordingly.

About versions...
The version of LTSP being used in the guide is 4.1. Barring a radical overhaul of the way LTSP works, most of the edits made to the various configuration files below will largely be independent of version changes in LTSP, with one possible exception. Minor things are sure to change, such as the keywords used to invoke certain features in lts.conf(the main LTSP configuration file), but the overall structure should stay more or less the same. The one exception is rc.sysinit. If you try to copy the rc.sysinit I provide below into some other installation of LTSP, there's a good chance everything will become hopelessly screwed up since that file typically is not changed when setting up the network. The modifications made to rc.sysinit aren't very complicated at all, so if you want to try using a different version of LTSP and feel like trying your hand at hacking it on your own, then by all means give it a shot.

Hardware setup for the terminals
Each of your LTSP terminals will boot over the network and download their kernel from the server. Motherboards with onboard LAN often work the best since only very, very old motherboards will not be able to net-boot using PXE and you won't have to mess with PCI cards to get them started. However, if you don't have any of those nor any NICs with onboard bootroms you don't have to worry since the boot process can be changed a little bit depending what you've got to work with.

If you don't have any bootable NICs handy you can head over to rom-o-matic.net and download an etherboot rom image that's been built for your network card. The rom image can then be put on a floppy or burned to a CD. The floppy or CD will then do the work of initalizing the system and getting it ready to download the kernel from the server.

Something else you might want to hunt down are the MAC addresses that you'll need later while configuring DHCP. A MAC address is a unique identifier sort of like an IP address, but a MAC address is embedded into the hardware of a network card. That means it'll be the same every time you turn on the machine no matter what. That makes it useful for jobs like this when there's no OS running on the machine to provide any other kind of interface to the system. Usually you'll find the MAC address on a sticker on the back of the network card. If you're using onboard LAN you'll find the MAC address either on a sticker on the last PCI slot or somewhere else on the board. If all else fails and you can't find that blasted sticker anywhere you can hook the terminal up with a live network cable and a monitor and power it up. The machine will fail when it tries to boot over the network, but it'll display the MAC address on the screen so that you can write it down and pop it into the DHCP configuration file when the time comes.


BIOS tweaks for the terminals and server
It's also a good idea to play around in the BIOS a bit before you let your machines loose to be dedicated crunchers. For both the terminals and the server it can be useful to set the BIOS to power the system up after power loss so that you won't have to run around turning everything back on if the power goes out. Setting the option "Halt On" to "no errors" on each terminal will prevent the BIOS from hanging up during POST if you don't have a keyboard or video card in the system. Also, in some cases you'll need to enable an onboard network boot rom in the BIOS before you can use it.


Requirements for running LTSP
The LTSP server is what controls the configuration of each terminal and the process of booting each terminal over the network. LTSP requires a few basic services in order to make that happen properly. Those are DHCP, TFTP, NFS, XDMCP, and optionally DNS. A detailed description of how each of those services are used can be found in the LTSP documentation. You can check to make sure each of those services are installed by opening a terminal window and entering the following commands below.

CODE
[root@screwball ~]# rpm -q dhcp
dhcp-3.0.2-12
[root@screwball ~]# rpm -q bind
bind-9.3.1-8.FC4
[root@screwball ~]# rpm -q nfs-utils
nfs-utils-1.0.7-8
[root@screwball ~]# rpm -q tftp-server
tftp-server-0.40-6
[root@screwball ~]# rpm -q portmap
portmap-4.0-65

It's ok if the version numbers are slightly different, but if any of these commands respond with "package not installed" or simply plop you back at the command line without responding at all, then the package need to be installed. The easiest way to do that in FC4 is to use yum(yes, they named it yum) which is similar to apt-get that Debian users know and love.

For example, entering this command as root would download and install NFS.

CODE[root@screwball ~]# yum install nfs-utils

If you needed to install NFS you might find that it requires other packages in order to work properly. yum should be able to get those for you also, but it might pester you with the kind of "is this ok?" questions that geeks typicallly say "yes" to without hardly even reading them.

It's also helpful to have each of those services set to load when the server boots so that you don't forget to start one of them and wonder why your network has suddenly taken a dive. The GUI for starting and stopping services is under Desktop->System Setting->Server Setting->Services(check the man page for chkconfig if you prefer the command line). Putting a check in the box next to the service name will make sure that the service starts when the system boots. Don't worry about it for now if any of these services won't start. We'll deal with that later. The service name of bind is named.

As a side note, an alternate DNS server could be dnsmasq. It's a nice little daemon useful for jobs like this on small home networks. If you're new to linux, one advantage to using bind is that it can be installed when you install Fedora which removes the need to install it manually either by using an RPM or by compiling from source(which often is a pitfall for many a beginner). However, configuring bind can be a little bit troublesome for beginners as well since it contains a ton of features that this project will probably never need. I actually prefer dnsmasq over bind for this project just for that reason.

Installing LTSP
After you've installed the required services you can get started on installing and configuring LTSP. The download page is here...

http://wiki.ltsp.org/twiki/bin/view/Ltsp/DownLoads

Personally, I like using the LTSP installer rather than the ISO. Using the LTSP installer to download the packages for you is less complicated than having to extract or mount the ISO, and it'll probably be less painful if you've got a slow connection.

After you've downloaded the RPM, run the command as root shown below to extract the RPM and install the package all in one shot.

CODE[root@screwball ~]# rpm -ivh ltsp-utils-0.11-0.noarch.rpm

Again, the version number may be different. 0.11-0 was the latest release at the time of this writing.

Afterwards, you can run the installer by typing "ltspadmin". The ltspadmin installer requires that it be run as Superuser which requires you to enter the command "su -" rather than just "su". If you keep getting the message "command not found" when trying to run ltspadmin that's probably the reason why.

You'll need to configure the installer, but that's a simple process where you'll again be asked the usual confirmation questions which most geeks hit "yes" to without hardly even reading them. Once you've started the installation and are presented with a list of packages to install, hit 'A' to select all packages and 'Q' to start the installation process. Installing all the packages in one shot might seem like a sledgehammer type of approach, but it's still the best option because a full install doesn't take up that much space and it's a drag to back up and install the others if you miss something.

Once you start configuring LTSP, you'll be given the choice between "Show status for all services" or "Configure the services manually". Pick configure manually. A short summary of each option is listed below.

1. Runlevel: Chooses the runlevel of the server. The description given in ltspadmin describes this one pretty well. I always leave it at runlevel 5 and then determine the behavior of the terminals later by using screen scripts(which I'll cover a bit later).

2. Interface Selection: If your server has more than one NIC this is where you tell LTSP which one it will be using.

3. Configure DHCP: This option allows you to start and configure the DHCP service that LTSP uses for network configuration. The configuration file DHCP uses is found at /etc/dhcpd.conf and it'll need to be edited later to fill in the details of your network.

4. Configure TFTP: TFTP is used to download the kernel that will run on each terminal from the server. Very rarely have I needed to manually configure TFTP. Setting it up usually is just a matter of hitting "Y" when ltspadmin asks you if you want to start it.

5. Configure Portmapper: The portmapper acts as a sort of switchboard. It coordinates which ports are being used by what applications on the server in order to avoid traffic jams on the network. I've had the same experience here as with TFTP. Usually you can just start it up and leave it alone.

6. Configure XDMCP: XDMCP is the X Display Manager Control Protocol. The same deal holds true here, at least as far as this project is concerned. If you're not going to be booting into a Linux desktop, then you won't have to mess with this one at all.

Options 8 through 11 all create templates of various configuration files that tell LTSP everything it needs to know in order to behave properly on your network. You can either go ahead and have ltspadmin generate all these files for you and tweak them later, or leave them all alone and start from scratch.


Editing /etc/hosts
The hosts file contains the name-to-address mappings for your local network. My hosts file looks something like this...

CODE
127.0.0.1        localhost       localhost.localdomain
#for loopbacking, leave this line alone

192.168.1.11     screwball       #LTSP server



We're starting off easy. If the IP of your server isn't 192.168.1.11(and if you want its hostname to be something other than screwball), then you'll need to change that in order to fit your network. You can do this by going to Desktop->System Settings->Network and clicking on the hosts tab, or by editing /etc/hosts as root in the editor of your choice. I have my server set up to assign dynamic IP addresses to each terminal. If you want your server to assign static IPs, then you'll need to include a line for each terminal containing its IP and hostname as well.


Editing /etc/exports
One of the coolest features of LTSP is that all the files generated by the folding client can be stored on the server hard drive. This allows you to take a peek at how each terminal is doing in exactly the same way as you could if the client was running locally on the server. Before it can do that though the server needs to know where to put all those files and how it should treat them. The NFS service is what does that and the exports file is what NFS uses to figure out everything it needs to know about your network.

There's a GUI interface for this also(if you have it installed) which you can find at Desktop->System Settings->Server Settings->NFS or you can login as root and edit the file directly using the editor of your choice. My exports file looks something like this...

CODE
/home                          192.168.1.11/255.255.255.0(ro,sync,no_root_squash)
/opt/ltsp                      192.168.1.11/255.255.255.0(ro,sync,no_root_squash)
/var/opt/ltsp/swapfiles        192.168.1.11/255.255.255.0(rw,async,no_root_squash)
/opt/fah/screwball001          192.168.1.11/255.255.255.0(rw,async,no_root_squash)
/opt/fah/screwball002          192.168.1.11/255.255.255.0(rw,async,no_root_squash)
# ... and so on for each terminal on the network
# Notice the blank line below!



Again, change the server IP of 192.168.1.11 to whatever you need it set to. The options of ro and rw should be fairly self explanitory as read-only and read/write. The others of sync and async correspond to the option in the GUI of "Sync write operations on request". The sync option enables this feature and the async option disables it. The no_root_squash option corresponds to enabling the GUI option of "Treat remote root user as local root". You'll need to login as root in order to create directories in /opt on the server, so if this option is not enabled you might run into file permissions problems with the folding client not being able to write files to the directory. If security is a big issue, then you might want to find a way to make NFS happy without using this option, but for this project it probably won't be much of a problem.

Each terminal will run a separate copy of the folding client, so the directories of /opt/fah/screwball001 and /opt/fah/screwball002 and so on are where the folding client for each terminal are stored on my server. screwball001 and screwball002 are the names I picked, but you can call your terminals pretty much anything you want as long as you use the same names in all the different configuration files. You can also pick pretty much anything you want for the directory on where to store the folding clients. Since /opt is usually used for software that's not installed with the distro I decided to store all the directories in /opt/fah.

The directories of /home, /opt/ltsp, and /var/opt/ltsp/swapfiles can all stay put. The only reason you'd need to change something is if you installed LTSP to some directory other than the default of /opt/ltsp.

Note: If you're editing /etc/exports by hand there must be a blank line at the end of the file. I have no idea why that is, but there must be a blank line at the end of the file. If do not put a blank line at the end of the file, then you could very easily end up pulling all your hair out and crying on the floor trying to figure out why NFS won't work.


Editing /etc/dhcpd.conf
If you're getting a little fried at this point now would be a good time to take a break and get some pizza or something because this is often where the real fun starts.

If you're still reading this, then I guess you're ready to go ahead. LTSP uses DHCP to provide the IP addresses to each terminal. The dhcpd.conf file contains all the information needed for the DHCP server daemon to behave properly while running on the server.

One common source of problems is when a router with DHCP enabled is serving IPs from the same range that the LTSP server will be. In most cases, a home network will not need more than one DHCP server, and unless you're very tricky about it having more than one DHCP server on the same physical network will almost always cause you problems anyway. The best thing to do in this case is to turn off DHCP in your router. Once you get DHCP configured properly, the LTSP server can take the role of assigning IP addresses to each of your terminals as well as the rest of the network.

In order to make this a little easier, I made a sample dhcpd.conf and put way too many comments in it. Once you're done editing the file copy it to /etc/dhcpd.conf I'll go over it here as well to provide a little more explanation. If you're using a copy of dhcpd.conf that ltspadmin generated for you, then that file will probably look a little different.

At the top of the file, you'll see a few lines that look like this...

CODEddns-update-style         none;

option option-128 code 128 = string;
option option-129 code 129 = text;

default-lease-time 21600;
max-lease-time 21600;

All that stuff you can probably leave alone and be ok. These options just pass a little more information about your terminals and network on to DHCP and also define how long each IP is "leased" to a given terminal.

Next up is to tell DHCP some details about the network and the LTSP server.

CODEoption root-path         "192.168.1.11:/opt/ltsp/i386";
option domain-name       "localhost.localdomain";

subnet 192.168.1.0 netmask 255.255.255.0 {
  option domain-name-servers    192.168.1.11;
  option broadcast-address      192.168.1.255;

  option routers                192.168.1.1;
  option subnet-mask            255.255.255.0;

  range 192.168.1.10 192.168.1.254;
}

The root-path option is where DHCP can find the LTSP software on your network. Again, change the server IP to whatever you need to set it to. If you have a domain name, you can fill that in for the value of option domain-name. Otherwise, leave it as localhost.localdomain.

The subnet declaration is where things might start to get a little sticky. The IP of my LTSP server is 192.168.1.11 which means it lives on the 192.168.1.0 subnet. Every subnet that's tied in to the LTSP server must have it's own subnet block otherwise DHCP will become very unhappy. If you want to work some more separation into your network, this is where you'd need to let DHCP know about it by including multiple subnet blocks. Most of the time though if you have only one physical network, then you're going to be working with only one subnet. I have everything set up on one subnet here and it works just fine.

Now let's take a look at the options inside the subnet block. I use a standalone router to provide Internet access to the server and the terminals, so the value of option routers is the IP of my primary router. The values of option broadcast-address and option subnet-mask are pretty standard for home networks.

I have my network set up to assign dynamic IPs to the terminals since I think it makes dhcpd.conf a little cleaner that way. The range statement is what defines the pool of IP address that DHCP will pull from when assigning IPs to the terminals. However, if static IPs fit your setup better you can bend DHCP to your will in that way also. Next up are the group and host declarations, where we tell DHCP a little bit about each terminal on the network.

CODEgroup {
    use-host-decl-names    on;
    option log-servers     192.168.1.11;

    host screwball001 {
        hardware ethernet  00:0C:76:90:93:FD;
        filename           "/lts/2.4.26-ltsp-3/pxelinux.0";
    }
    host screwball002 {
        hardware ethernet  00:40:F4:B0:27:D1;
        filename           "/lts/vmlinuz-2.4.26-ltsp-3";
    }
    # ... and so on for each terminal...
}


The big string of garbage on the hardware ethernet line is the MAC address for each terminal that you collected earlier. The filename parameter tells LTSP where to find the kernel that the terminal will use when it boots. You'll notice that the filenames listed for each terminal are different. In the example above, screwball001 is using a bootable NIC, and screwball002 is booting from a CD or floppy. The details on why different boot processes require different kernels aren't particularly important, but remember how you set the terminals up. Pointing DHCP towards the wrong kernel is a sure-fire way to bring everything to a screeching halt.

If TFTP starts with the -s flag, then the path you'll need to pop in to dhcpd.conf will be relative to /tftpboot. That is, if the actual path on the server is /tftpboot/lts/vmlinuz-2.4.26-ltsp-3 then the path you'll need to enter in dhcpd.conf is /lts/vmlinuz-2.4.26-ltsp-3(notice that "/tftpboot" is not included here) If you're not sure about TFTP, try running ltspadmin again and this time pick "show status of all services" after choosing to configure LTSP. It should show you what's going on with TFTP.

Barring any errors that might pop up, at this point you're pretty much done(yay) with the configuration of DHCP. The only thing left to do is to create the leases file by issuing the command below as root...

CODE[root@screwball ~]# touch /var/lib/dhcp/dhcpd.leases

Editing /opt/ltsp/i386/etc/lts.conf
lts.conf is the main configuration file for LTSP. The entries in lts.conf provide the last bits of information needed about the LTSP server and terminals. Near the top of the file should be a block that looks something like this...

CODE[Default]
        SERVER              = 192.168.1.11
        XSERVER             = auto
        X_MOUSE_PROTOCOL    = "PS/2"
        X_MOUSE_DEVICE      = "/dev/psaux"
        X_MOUSE_RESOLUTION  = 400
        X_MOUSE_BUTTONS     = 3
        USE_XFS             = N

The server IP should be familiar by now, and most of the other entries pertain to how the terminal should behave when using a graphical interface. I've kept them in there for the sake of compatibility even though I don't often use any of the terminals in a graphical interface. The USE_XFS statement is whether or not to use a font server, which for this project you won't need.

Next up is the configuration of each terminal.

CODE
[screwball001]
        NIS_DOMAIN         = screwball.ltsp
        NIS_SERVER         = 192.168.1.11
        LOCAL_APPS         = Y
        SCREEN_01          = shell
        RCFILE_01          = startfah

[screwball002&

-------------------------
Containment Breach

Do not meddle in the affairs of archers, for they are subtle and quick to anger.
Statistics
112018 users are registered to the AMD Processors forum.
There are currently 0 users logged in.

FuseTalk Hosting Executive Plan v3.2 - © 1999-2014 FuseTalk Inc. All rights reserved.



Contact AMD Terms and Conditions ©2007 Advanced Micro Devices, Inc. Privacy Trademark information