TS93-55 - 192.168.1.93 10.0.0.55
TS94-56 - 192.168.1.94 10.0.0.56
Virtual Ip - 192.168.1.95
Each machine has 2 network cards: one for public and one for private traffic. The public interfaces connect to my main switch on the 192.168.1.X network. The private interfaces are connected via a crossover cable on the 10.0.0.X network.
Once you have installed Ubuntu 8.04 Server, you need to install some build tools.
apt-get install build-essential binutils cpp gcc autoconf automake1.9 libtool \
autotools-dev g++ make flex
In theory, that should get all of the development tools installed that you'll need.
Next we need to get the entire kernel source code. Just the kernel headers won't cut it. We need to compile the kernel so that it builds some of the scripts needed to compile the DRBD driver. We'll also install the ncurses library so that menuconfig works.
apt-get install libncurses5-dev linux-source-2.6.24
Then extract the kernel source:
cd /usr/src
tar -xvf linux-source-2.6.24.tar.bz2
cd /usr/src/linux-source-2.6.24
Next, lets clean up any unneeded files (which there shouldn't be any the first time):
make mrproper
Before you can build the kernel, you need to copy your existing kernel build configuration into the kernel source directory:
cp /boot/config-2.6.24-16-server /usr/src/linux-source-2.6.24/.config
Now we run the menuconfig which will read in our kernel build configuration and build some version files. As soon as the GUI appears, just exit immediately. You don't have to change any of the settings.
make menuconfig
Finally we need to prepare the kernel and compile it. This will take quite some time.
make prepare
make
Now that we have the kernel source compiled and ready to go, let's get the DRBD source.
cd /root
wget http://oss.linbit.com/drbd/8.2/drbd-8.2.5.tar.gz
tar -xvf drbd-8.2.5.tar.gz
cd /root/drbd-8.2.5
We need to build the DRBD driver and specify the path to the kernel source, then install the driver in the /lib path:
make KDIR=/usr/src/linux-source-2.6.24
make install
Once the driver is compiled, we need to move/copy it to the appropriate lib directory:
mv /lib/modules/2.6.24.3/kernel/drivers/block/drbd.ko \
/lib/modules/2.6.24-16-server/kernel/drivers/block
Next we need to start the driver and tell Linux to load it the next time it boots:
modprobe drbd
echo 'drbd' >> /etc/modules
update-rc.d drbd defaults
Now that everything is installed, verify the driver is loaded:
lsmod | grep drbd
It might be a good idea to reboot and make sure it loads.
At this point, you should set up the /etc/drbd.conf, which mine looks like this:
global {
usage-count no;
}
common {
protocol C;
syncer {
rate 30M;
al-extents 1801;
}
startup {
wfc-timeout 0;
degr-wfc-timeout 15;
}
disk {
on-io-error detach;
# fencing resource-and-stonith;
}
net {
sndbuf-size 512k;
timeout 60; # 6 seconds (unit = 0.1 seconds)
connect-int 10; # 10 seconds (unit = 1 second)
ping-int 10; # 10 seconds (unit = 1 second)
ping-timeout 5; # 500 ms (unit = 0.1 seconds)
max-buffers 8000;
max-epoch-size 8000;
cram-hmac-alg "sha1";
shared-secret "secret";
}
}
resource r0 {
on TS93-55 {
disk /dev/mapper/ts-ts--ts1-1;
address 10.0.0.55:7788;
device /dev/drbd0;
meta-disk internal;
}
on TS94-56 {
disk /dev/mapper/ts-ts2-1;
address 10.0.0.56:7788;
device /dev/drbd0;
meta-disk internal;
}
}
With the configuration set, you need to restart/reload DRBD, create the meta disk, and bring the drive up:
/etc/init.d/drbd restart
drbdadm create-md r0
drbdadm up r0
* If any error like the following one occurs:
TS93-55 # drbdadm create-md r0
md_offset 1069215744
al_offset 1069182976
bm_offset 1069150208
Found ext3 filesystem which uses 1044160 kB
current configuration leaves usable 1044092 kB
Device size would be truncated, which
would corrupt data and result in
'access beyond end of device' errors.
You need to either
* use external meta data (recommended)
* shrink that filesystem first
* zero out the device (destroy the filesystem)
Operation refused.
Command 'drbdmeta /dev/drbd0 v08 /dev/sdb5 internal create-md'
terminated with exit code 40
drbdadm aborting
try
dd if=/dev/zero of=/dev/sda5 bs=1M count=128
* And then you can successfully create your resource on both machines:
[paras1]# drbdadm create-md r0
Wait the end of the synchronization.
At this point you can view the DRBD status:
TS93-55:~$ cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12
0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r---
ns:221871972 nr:7160 dw:3856764 dr:227396211 al:763 bm:17504 lo:0 pe:0 ua:0 ap:0
resync: used:0/31 hits:13841287 misses:13628 starving:0 dirty:0 changed:13628
act_log: used:0/1801 hits:961638 misses:790 starving:0 dirty:27 changed:763
chris@app1:~$ /etc/init.d/drbd status
drbd driver loaded OK; device status:
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12
m:res cs st ds p mounted fstype
0:r0 Connected Primary/Secondary UpToDate/UpToDate C
* Now it’s time to create your EXT3 filesystem on the DRBD device! (only in the case there was no existing data on /dev/sdb5)
[TS93-55]# mke2fs -j /dev/drbd0
You're now ready to mount /dev/drbd0 and put data on it. There's still other things you need to do if you plan on using Heartbeat for failover monitoring.
Step1:
sudo apt-get install heartbeat
Step 2.
Now we’re going to configure the /etc/ha.d/ha.cf file for our machine. Here is what I’ve put into the /etc/ha.d/ha.cf file ON EACH MACHINE:
vim /etc/ha.d/ha.cf
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
initdead 120
bcast eth0
udpport 694
auto_failback on
node TS93-55
node TS94-56
Step 3.
Now configure the /etc/ha.d/authkeys file ON EACH MACHINE for what kind of security and file checking you want, I don’t care about security in this example so I put this is the file since it’s the fastest:
auth 2
2 crc
Step 4:
vim /etc/ha.d/hareresource
TS93-55 drbddisk::r0 Filesystem::/dev/drbd0::/pgsql/data::ext3 192.168.1.95 postgresql-8.2 apache2
Step 5.
Make sure heartbeat and the service(s) you’re watching DO NOT start at boot, otherwise things get really ugly if when you screw up:
chkconfig heartbeat off
chkconfig httpd off
/etc/init.d/httpd stop
(on both machines)
Step 6.
Alright, it’s finally time to test your failover configuration. First, we need to start heartbeat on the primary machine:
TS93-55# /etc/init.d/heartbeat start
Then, start it on the secondary machine
TS94-56# /etc/init.d/heartbeat start
You should now be able to ping the cluster IP (192.168.1.95). You can also check that the /dev/drbd0 filesystem is mounted on the primary node using df. Check the /var/log/messages file on either machine for debugging information
For more informations follow the links below
http://gcharriere.com/blog/?p=1
http://www.cb1inc.com/2008/05/18/installing-drbd-8.2.5-on-ubuntu-8.04-hardy-heron
http://writequit.org/blog/?cat=61
Thursday, April 23, 2009
Subscribe to:
Posts (Atom)
