Temporary Server Portforward

Sometimes it is useful to redirect all tcp traffic to port 80 from one server to another. This can be important when you decide to change your DNS entries from one server to another and you don’t want to leave the old webserver server running.

  echo 1 > /proc/sys/net/ipv4/ip_forward
  iptables -F
  iptables -t nat -F
  iptables -X
  iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination SOURCE_IP:80
  iptables -t nat -A POSTROUTING -p tcp -d SOURCE_IP --dport 80 -j SNAT --to-source DESTINATIONIP
  iptables -t nat -A POSTROUTING -j MASQUERADE

GlusterFS / Async Writes

Over the weekend i decided to experiment and setup a glusterfs between my root server(plexus) and my fileserver at home(hydra).

I won’t describe how to setup glustefs. There are various good guides available.

  gluster volume create gv0 replica 2 hydra:/opt/gfs/brick1/gv0 plexus:/gfs/brick1/gv0
  gluster volume set gv0 network.ping-timeout 5
  gluster volume start gv0

  root@plexus:/gfs# mount -t glusterfs localhost:/gv0 gv0/
  root@hydra:/gfs# mount -t glusterfs localhost:/gv0 gv0/

At first glance i hoped that small writes would be cached and asynchronously flushed to the filesystem. I tried to increase the performance.cache-size and performance.write-behind-window-size option but did not succeed in increasing the write performance.

As you can see the write performance is basically predetermined by my max upload speed.

  # Write 1MB
  root@hydra:/gfs/gv0# dd if=/dev/zero  of=testfile bs=1K count=1024
  1024+0 records in
  1024+0 records out
  1048576 bytes (1.0 MB) copied, 3.36612 s, 312 kB/s

But luckily there are branching filesystems like unionfs that is for example used for live CDs. Such filesystems work in that way that they overlay multiple filesystems into a single virtual one.

I created a new gv0-writecache folder which can be written to. The follow command creates a unionfs filesystem that is mounted to the /gv0/union folder.

  mkdir -p /gfs/union
  unionfs-fuse  -o default_permissions -o allow_other /gfs/gv0-writecache=RW:/gfs/gv0=RO /gfs/union/

Succeeding writes to the union folder are now very fast and the files are stored within the /gfs/gv0-writecache folder.

  root@hydra:/gfs/union# dd if=/dev/zero of=test2 bs=1M count=10
  10+0 records in
  10+0 records out
  10485760 bytes (10 MB) copied, 0.0454509 s, 231 MB/s

After that step we only need to make sure to sync those files from the writecache to the glusterfs mount.

  root@hydra:/gfs/gv0-writecache# rsync  --remove-source-files -av /gfs/gv0-writecache/* /gfs/gv0

I also tried to use aufs instead of unionfs-fuse but i got various strange errors.

Resizing raid5 with mdadm, lvm and luks for encryption on debian wheezy

This guide will show you how to setup a simple mdadm, lvm test environment in which you can experiment with changes you want to execute on a real disk system.

Setup the test environment

You can of course use your own disks. In my case i did a trial run of the needed steps by using losetup to map a data file to a loop device.

  dd if=/dev/zero of=diska bs=1M count=300
  dd if=/dev/zero of=diskb bs=1M count=300
  dd if=/dev/zero of=diskc bs=1M count=300
  dd if=/dev/zero of=diskd bs=1M count=300
  losetup /dev/loop0 diska
  losetup /dev/loop1 diskb
  losetup /dev/loop2 diskc
  losetup /dev/loop3 diskd

  # Create raid 5 with one disk missing
  mdadm -Cv /dev/md2 -l5 -n3 /dev/loop0 /dev/loop1 missing

  # Add luks cryptolayer ontop
  cryptsetup luksFormat /dev/md2
  cryptsetup luksOpen /dev/md2 cryptotest

  # Add lvm structure 
  pvcreate /dev/mapper/cryptotest
  vgcreate testvg /dev/mapper/cryptotest
  lvcreate -l 100%FREE -n testlv testvg

  # Create filesystem
  mkfs.ext3  /dev/mapper/testvg-testlv
  mount /dev/mapper/testvg-testlv test/

You can now copy your data onto the raid 5 and when done add the missing disk to the final raid. You can of course use three disks from the start.

Add missing disk to the raid 5

  # Add missing device
  mdadm --add /dev/md2 /dev/loop2

Growing the raid 5 by adding a forth disk

Once your data has been copied you may want to add the now free disk to the raid of md2.

  # Add disk and resize md2
  mdadm --add /dev/md2 /dev/loop3
  mdadm --grow --raid-devices=4 /dev/md2

  # Check the sync process and the final result
  cat /proc/mdstat
  mdadm --detail /dev/md2
  umount /tmp/test/test

  # Resize the crypt layer and check the result
  cryptsetup resize cryptotest
  fdisk -l /dev/mapper/cryptotest

  # Resize physical volume and check the result
  pvresize /dev/mapper/cryptotest 
  pvdisplay

  # Resize logical volume and check result
  lvextend -l +100%FREE /dev/mapper/testvg-testlv
  lvdisplay 

  # Resize filesystem
  e2fsck -f /dev/mapper/testvg-testlv
  resize2fs /dev/mapper/testvg-testlv
  mount /dev/mapper/testvg-testlv test

Cleanup of the test environment

Of course you should not do these steps on the real raid. These steps are just needed when you want to cleanup your testenvironment.

  umount /tmp/test/test
  lvremove testvg testlv
  vgremove testvg
  pvremove  /dev/mapper/cryptotest 
  cryptsetup close /dev/mapper/cryptotest 
  mdadm --manage /dev/md2 --stop
  losetup -d /dev/loop0
  losetup -d /dev/loop1
  losetup -d /dev/loop2
  losetup -d /dev/loop3

MySQL 5.6 Debian Wheezy Installation

Just a small snippet on how to setup the MySQL 5.6 database on debian wheezy.

apt-get install libaio1
groupadd mysql
useradd -r -g mysql mysql
wget -O mysql-5.6.16-debian6.0-x86_64.deb http://downloads.mysql.com/archives/mysql-5.6/mysql-5.6.16-debian6.0-x86_64.deb
dpkg -i mysql-5.6.16-debian6.0-x86_64.deb
cd /usr/local
ln -s /opt/mysql/server-5.6 mysql
cd mysql
scripts/mysql_install_db --user=mysql
chown -R root .
chown -R mysql data
cp support-files/mysql.server /etc/init.d/mysql
mkdir -p /etc/mysql/conf.d/
cp my.cnf /etc/mysql/
echo $'!include /etc/mysql/my.cnf\n!includedir /etc/mysql/conf.d/' > my.cnf
update-rc.d mysql-5.7 defaults
service mysql start
./bin/mysql_secure_installation