This manual page describes the format of the Beowulf-config file /etc/beowulf/config. This file defines the structure of a Scyld ClusterWare cluster, and provides a central location for many of the operational parameters.
The Beowulf-config file contains the settings for beoboot, node initialization, BProc communication parameters, and other aspects of cluster operation.
The syntax of the Scyld configuration files is standardized, and is intended for human editing with embedded comments. Tools are provided for reading and writing from common programming and scripting languages, with writing retaining comments and formatting.
While /etc/beowulf/config may be edited with any text editor, it is best changed using the beonetconf or beosetup programs. However, some parameters may not be editable using those programs, and must be set carefully when editing the file, as incorrect editing may leave the cluster unusable.
Regardless of which option you use to edit the Beowulf-config file, the changes don't take effect until Beowulf services are restarted. You can restart Beowulf services by rebooting the master node, or by executing either /ect/rc.d/init.d/beowulf restart or service beowulf restart.
The config file format is a line-oriented sequence of configuration entries. Each configuration entry starts with a keyword followed by parameters. The line is terminated by a newline or '#'. The latter character starts a comment.
The keyword and following parameters have the same syntax rules: they may be preceded by whitespace and continue to the next whitespace or the end of the line.
Keywords and following parameters may include whitespace by quoting between a matching pair of '"' (double quote) or ''' (single quote) characters. A '\' (backslash) removes the special meaning of the following quote character.
Note that comments and newlines take precedence over any other processing, thus a '#' may not be used in a keyword or embedded in a parameter, and a backslash followed by a newline does not join lines.
Each configuration option is contained on a single line, with keyword and optional parameters. Blanks lines are ignored. Comments begin with an unquoted '#' and continue to the end of the line.
The Beowulf-config file is intended to be parsed using the beoconfig utility.
The interface directive is used to specify the name of the interface that connects the master node to the compute nodes. This is used by the cluster services and management tools such as the BProc master daemon and the Beoserv daemon. Common values are eth1 (the second ethernet device) and myri0 (for Myrinet). If present, entries after the interface name specify the IP address and netmask that the interface should be configured to.
The iprange directive specifies the range of IP addresses to be assigned to nodes. If the optional nodenumber is given, the first address in the range will be assigned to that node, the second address to the next node, etc. If no node number is given, the address assignment will begin with the node following the node that was last assigned. If no nodes have been assigned, the assignment will begin with node 0.
The ip directive assigns one or more IP addresses to a single node. The addresses must be in dotted notation (eg. 192.168.1.100). If the optional nodenumber argument is given, the specified IP address will be used for that node. If no node number is given, the addresses will be assigned beginning with the node number following whatever node was assigned last. If no nodes have been assigned, the IP address assignment will start with node 0.
The bootfile directive is used to specify the name of the phase 2 image that should be used. If a node number is given, then the specified phase 2 image will be used for that node. If no node number is given, then the specified phase 2 image will be used for all compute nodes that have not been specified in any other bootfile entries. The nodenumber can also be a list of comma separated node ranges (eg. 1-5, 10-20).
The bprocport directive is used to specify the TCP port that the compute node connects to on the master for BProc.
The fsck directive is used to specify the file system checking policy to be used at boot time. The valid policies are never, safe, or full.
The file system on the compute nodes will not be checked on boot.
The file system on the compute nodes will go through a safe check everytime the compute node boots.
The file system on the compute nodes will go through a full check everytime the compute node boots. The full check might possibly remove files from the filesystem if they cannot be repaired.
The kernelcommandline directive is used to specify any options you wish to have passed to the kernel on the compute nodes. These are the same options that are normally passed with 'append=' in lilo, or on the lilo prompt while the machine is booting (eg. 'kernelcommandline apm=power-off'.)
The kernelimage directive is used to specify the full path to the kernel that should be used when creating the phase 2 boot images for the compute nodes.
The ignore directive is used to specify a MAC address (eg. 00:11:22:AA:BB:CC) that beoserv should ignore RARP requests from. Multiple ignore directives are allowed.
The insmod directive is used to specify a kernel module to be loaded (usually a network driver). Options for the module may be specified as well.
The libraries directive is used to specify a list of libraries that should be cached on the slave nodes when they boot. The library path can be a directory or file. If a directory name is specified, everything in that directory will be copied over, but none of the sub-directories under that directory will be copied over.
The logfacility directive specifies the log facility that the BProc master daemon should use. Some example log facility names are daemon, syslog, and local0 (see the syslog documentation for more information.) The default log facility is daemon.
The mcastbcast directive is used to tell beoserv to use broadcast instead of multicast when transmitting files over the interface. This is useful when network equipment has trouble with heavy multicast traffic.
The mcastthrottle directive is used to control the rate at which data is transmitted over the specified interface. The rate is given in megabits per second. This is useful when the compute node interfaces cannot keep up with the master interface when sending large files.
The mkfs directive specifies the policy to use when building a Linux file system on the compute nodes. The valid policies are never, if_needed, or always.
The filesystem on the compute nodes will never be recreated on boot.
The filesystem on the compute nodes will only be recreated if the filesystem check fails.
The filesystem on the compute nodes will be recreated on every boot. fsck will be assumed to be set to never when this is set.
The modprobe directive is used to specify the name of the kernel module to be loaded with dependency checking along with any specified module options.
The modarg directive is used to specify options to be used for modules that are loaded during the boot process without options. This is useful for specifying options to modules that get loaded during the PCI scan.
The moddep directive is used to specify module dependencies. The first module listed is dependent on the remaining modules in the space separated list. The first module will be loaded after all other listed modules. Module dependency information is normally automatically generated by the beoboot script.
The node directive is used to assign MAC addresses to node numbers. The user is the username of the user who owns the node (this can also be "any"), and the group is the name of the group who owns the node (this can also be "any"). There should be one of these lines for each node in your cluster. Note the following:
If a vaule is not provided for the nodenumber argument, the first node entry is node 0, the second is node 1, the third is node 2, etc.
The value off can be used for the MACaddress argument to leave a place holder for that node number.
To skip a node number, use the value node or node off for the MACaddress argument.
To skip a node number and make sure it will never be automatically filled in by something later in the future, use the value node reserved for the MACaddress argument.
The nodes directive is used to specify the total possible number of nodes in the cluster. This should normally be set to match the iprange. However, if multiple ipranges are specified, then this value should represent the total number of nodes in all the iprange entries.
The pci directive is used to specify what driver should be used in support of the specified PCI device. A device is identified by a unique vendor ID and device ID pair. The vendor and device ID's can be either in decimal or hexadecimal with the "0x" notation. You should have one of these lines for each PCI ID (a vender ID combined with a device ID) for each device on your compute nodes that is not already recognized. Any module dependencies or arguments should be specified with moddep and modarg.
The server directive is used to specify the port number that beoserv uses for the specified transport protocol (see the transportmode directive.) A different port number must be used for each protocol type. The allowable values for the protocol are mc (multicast) or tcp (unicast). The default port numbers are 10001 for multicast and 1556 for unicast.
The transportmode directive is used to specify the transport protocol to be used by beoserv for serving IP addresses, boot files, and libraries to the compute nodes. The valid transport protocols are mc (multicast) or tcp (unicast). The default value is tcp.
# Beowulf Configuration file # Network interface used for Beowulf # Only first argument to interface is important interface eth1 192.168.1.1 255.255.255.1 # These two should probably agree for most users iprange 192.168.1.100 192.168.1.107 nodes 8 # Default location of boot images bootfile /var/beowulf/boot.img kernelimage /boot/vmlinuz-2.4.17-0.18.12.beo kernelcommandline apm=power-off # Default libraries libraries /lib /usr/lib # Default file system policies. fsck full mkfs if_needed # beoserv settings server mc 10001 server tcp 1556 transportmode mc # Default Modules bootmodule 3c59x 8139too dmfe eepro100 epic100 hp100 natsemi bootmodule ne2k-pci pcnet32 sis900 starfire sundance tlan bootmodule tulip via-rhine winbond-840 yellowfin # Non-kernel integrated drivers bootmodule e100 bcm5700 # gm # PCI Gigabit Ethernet. # * AceNIC and SysKonnect firmwares are very large. # * Some of these are distributed separate from the kernel bootmodule dl2k hamachi e1000 ns83820 # acenic sk98lin node 00:50:8B:D3:25:4D node 00:50:8B:D3:07:8B node 00:50:8B:D3:31:FB node 00:50:8B:D3:62:A0 node 00:50:8B:D3:00:66 node 00:50:8B:D3:30:42 node 00:50:8B:D3:98:EA