Configuration of Host Hardware and OS for Coordinating Nodes ============================================================ This document describes the setup of a virtual machine host on which will operate a DataONE Coordinating Node. The steps take for setup and configuration up to the point where a clean Ubuntu client is installed are described. Hardware -------- .. table:: Specifications of the hardware being installed for the UNM node. ================== =============================================== Attribute Value ================== =============================================== Brand, Model Dell, R905 Processors 4x Quad Core Opteron 8378 2.4GHz (Shanghai), 4x512K L2 Cache, 6MB L3 Cache, HyperTransport 1Ghz Memory 64GB 800MHZ ECC (16X4GB) Disks 5x 600GB 10K RPM Serial-Attach SCSI 3Gbps 3.5-in HotPlug HardDrive Disk Controller PERC6i SAS RAID Controller Internal with Battery NICs 4x Broadcom NetXtreme II 5708 1GbE Onboard NICs with TOE. Configured with LAG enabled. ================== =============================================== .. table:: Specifications of the hardware being installed for the UCSB node. ================== =============================================== Attribute Value ================== =============================================== Brand, Model Dell, R905 Processors 4x Quad Core Opteron 8380 2.5GHz (Shanghai), 4x512K L2 Cache, 6MB L3 Cache, HyperTransport 1Ghz Memory 64GB 800MHZ ECC (16X4GB) Disks 5x 600GB 15K RPM Serial-Attach SCSI 3Gbps 3.5-in HotPlug HardDrive Disk Controller PERC6i SAS RAID Controller Internal with Battery NICs 4x Broadcom NetXtreme II 5708 1GbE Onboard NICs with TOE. Configured with LAG enabled. ================== =============================================== Disk Configuration ~~~~~~~~~~~~~~~~~~ Disks are configured as RAID 1+0 (mirror + striping) with one hot spare, providing 1.2TB. OS Installation ~~~~~~~~~~~~~~~ Boot from Ubuntu 9.10 64 bit server distribution. .. Note:: The OS was upgraded to 10.04 on 2010-05-10 with no apparent side effects besides the need to set the correct host name (reverted to "ubuntu") .. table:: OS installation configuration values at UNM =============== ================================= Attribute Value =============== ================================= OS Ubuntu 10.04 64 bit server Server name host-unm-1 Server FQDN host-unm-1.dataone.org IP Address 129.24.0.18 Netmask 255.255.255.0 Gateway 129.24.0.1 DNS 1 129.24.0.52 DNS 2 Partitions swap 16GB LVM everything else. =============== ================================= .. table:: OS installation configuration values at UCSB =============== ================================= Attribute Value =============== ================================= OS Ubuntu 10.04 64 bit server Server name host-ucsb-1 Server FQDN host-ucsb-1.dataone.org IP Address 128.111.220.52 Netmask 255.255.255.0 Gateway 128.111.220.1 DNS 1 128.111.220.18 DNS 2 128.111.220.16 Partitions /boot 240MB LVM everything else. =============== ================================= .. table:: OS installation configuration values at ORC =============== ================================= Attribute Value =============== ================================= OS Ubuntu 10.04 64 bit server Server name orcvm1 Server FQDN orcvm1.dataone.org. host-orc-1.dataone.org IP Address 160.36.134.68 Netmask 255.255.255.240 Gateway 160.36.234.65 DNS 1 160.36.134.65 Partitions /boot 240MB LVM everything else. =============== ================================= Partition setup at UCSB - during install, select "Guided - use entire disk and setup LVM", then enter "40 GB" when asked how much space to use - after install extend swap and make partition for kvm:: $ sudo lvextend /dev/host-ucsb-1/swap_1 -L 16G $ sudo lvcreate -l 100%FREE -n kvm host-ucsb-1 $ sudo swapoff -a $ sudo mkswap /dev/host-ucsb-1/swap_1 $ sudo swapon -a $ sudo mkfs.ext4 /dev/host-ucsb-1/kvm - kvm partition mounted at the default ubuntu location /var/lib/libvirt/images and symlinked to /kvm, so as to not need to change /etc/libvirt config files The partitions (UNM):: # parted -l Model: DELL PERC 6/i Adapter (scsi) Disk /dev/sda: 1199GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 32.3kB 1199GB 1199GB primary boot, lvm 2 1199GB 1199GB 255MB extended 5 1199GB 1199GB 255MB logical ext2 Model: Linux device-mapper (linear) (dm) Disk /dev/mapper/dataone-kvm: 1103GB Sector size (logical/physical): 512B/512B Partition Table: loop Number Start End Size File system Flags 1 0.00B 1103GB 1103GB ext4 Model: Linux device-mapper (linear) (dm) Disk /dev/mapper/dataone-home: 40.0GB Sector size (logical/physical): 512B/512B Partition Table: loop Number Start End Size File system Flags 1 0.00B 40.0GB 40.0GB ext4 Model: Linux device-mapper (linear) (dm) Disk /dev/mapper/dataone-root: 40.0GB Sector size (logical/physical): 512B/512B Partition Table: loop Number Start End Size File system Flags 1 0.00B 40.0GB 40.0GB ext4 Model: Linux device-mapper (linear) (dm) Disk /dev/mapper/dataone-swap: 16.0GB Sector size (logical/physical): 512B/512B Partition Table: loop Number Start End Size File system Flags 1 0.00B 16.0GB 16.0GB linux-swap(v1) After the OS installation, setup for running KVM guests. Detailed instructions are available at the `Ubuntu site`_ :: $ sudo apt-get install kvm libvirt-bin ubuntu-vm-builder bridge-utils $ sudo adduser `id -un` libvirtd .. _Ubuntu site: https://help.ubuntu.com/community/KVM Edit ``/etc/network/interfaces`` to support bridging so the KVM guests can use external IP addresses:: $ cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet manual # The bridge network for KVM auto br0 iface br0 inet static address 129.24.0.18 netmask 255.255.255.0 network 129.24.0.0 broadcast 129.24.0.255 gateway 129.24.0.1 bridge_ports eth0 bridge_stp off bridge_fd 0 bridge_maxwait 0 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 129.24.0.52 dns-search nmepscor.net Modify the default configuration for ufw_ to enable forwarding by editing ``/etc/default/ufw`` and set:: DEFAULT_FORWARD_POLICY="ACCEPT" Then enable ssh and start the firewall:: sudo ufw allow ssh sudo ufw enable Network config at UCSB with two LACP bonded ports on an LACP enabled switch with bridging:: $ cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto bond0 iface bond0 inet manual bond-slaves none bond-mode 4 bond-miimon 100 auto eth0 iface eth0 inet manual bond-master bond0 bond-primary eth0 eth1 auto eth1 iface eth1 inet manual bond-master bond0 bond-primary eth0 eth1 auto br0 iface br0 inet static address 128.111.220.52 netmask 255.255.255.0 gateway 128.111.220.1 broadcast 128.111.220.255 bridge_ports bond0 bridge_fd 0 bridge_maxwait 0 .. _ufw: https://help.ubuntu.com/10.04/serverguide/C/firewall.html