32 bit to 64 bit Zimbra Server Migration on Ubuntu.

Posted December 14th, 2012 by bryanr

I recently succeeded in migrating our Zimbra server to a 64bit operating system.  If you are tasked with doing the same, I hope this post helps you out, because a ran into a few problems.  I had to use 4 different guides to finish the migration on Ubuntu; apparently the migration on Ubuntu requires more steps than other operating systems.  If you are reading this you already know that Zimbra no longer supports a 32bit operating system for it's newest release, Zimbra 8.  In our environment, we moved from Zimbra 7.2.1 on Ubuntu 8.04 32bit to Zimbra 7.2.1 on Ubuntu 10.04 64bit.

Here are the guides that I used to complete the move:






For the most part, I followed the first guide above, which is pretty straight foward.  But it omits several steps that were required in our migration.  Here is what I did:

Make certain you are installing the exact version of Zimbra on the new server that was installed on your old server!

Install 32bit runtime libraries on Ubuntu as root.

apt-get install ia32-libs

Install the just the packages on new host.

./install.sh -s

Copy /opt/zimbra/zimbramon of the 64bit installed stuff to a tmp dir

cp -rp /opt/zimbra/zimbramon /opt/zimbramon-64bit

Remove /opt/zimbra on new host.

rm -rf /opt/zimbra

On old host, rsync /opt/zimbra AS ROOT to the new host.

nice -n +19 rsync --progress -avzH -e ssh  /opt/zimbra/ root@NEWHOSTIP:/opt/zimbra

Continue to sync the zimbra directory until you are ready to move.


Move Day-

STOP SERVICES ON OLD HOST. Failure to stop services can result in data corruption on the target host.

zmcontrol stop

Perform last rsync as root on old host.

nice -n -20 rsync -avzH -e ssh --delete --progress /opt/zimbra/ root@NEWHOSTIP:/opt/zimbra

Copy the zimbramon-tmp over the /opt/zimbra/zimbramon dir

cp -rp /opt/zimbramon-64bit/* /opt/zimbra/zimbramon 

Repair permissions on new host.

/opt/zimbra/libexec/zmfixperms --verbose --extended 


Export the LDAP data on the old host:

a. Create an LDAP dump directory. As root, type:

mkdir /backup

b. As root, type:

chown zimbra:zimbra /backup

c. Backup the LDAP data, as zimbra, type:

/opt/zimbra/libexec/zmslapcat /backup

d. Backup LDAP config data, type:

/opt/zimbra/libexec/zmslapcat -c /backup

e. On new host, do:

mkdir /backup

f. From old host, copy over the ldap.bak file:

scp /backup/ldap.bak root@newserver.com:/backup 

g. From old host, copy over the ldap-config.bak file:

scp /backup/ldap-config.bak root@newserver.com:/backup 


Rename new host to old name and change IP. If you are not changing the IP, make sure that you have updated DNS and that the hostname resolves to the correct IP. (This varies by distro- /etc/hosts /etc/sysconfig etc). You may want to reboot the host just to be sure the name change takes.

Restore the LDAP data to the new server. As zimbra, type on new server:

rm -rf /opt/zimbra/data/ldap/hdb/*
rm -rf /opt/zimbra/data/ldap/config/*

If this is an ldap master with replicas:

rm -rf /opt/zimbra/data/ldap/accesslog/*
mkdir -p /opt/zimbra/data/ldap/hdb/db /opt/zimbra/data/ldap/hdb/logs

If this is an ldap master with replicas:

mkdir -p /opt/zimbra/data/ldap/accesslog/db /opt/zimbra/data/accesslog/logs

Create the new directory structure and restore database tuning

cd /opt/zimbra/data/ldap
mkdir -p hdb/db
mkdir -p hdb/logs


chown -R zimbra:zimbra /opt/zimbra/data/ldap


/opt/zimbra/openldap/sbin/slapadd -q -n 0 -F /opt/zimbra/data/ldap/config -cv -l /backup/ldap-config.bak


/opt/zimbra/openldap/sbin/slapadd -q -b "" -F /opt/zimbra/data/ldap/config -cv -l /backup/ldap.bak



this time without the -s flag. This will upgrade the files under /opt/zimbra that need to be corrected for the new hardware. Select Y to upgrade. Do not check the database as you will receive some errors.

update the ssh keys after the install is finished, as zimbra, type:



zmcontrol restart

Make sure that the UID for the zimbra user is the same as before. If it has changed, edit /etc/passwd to make it match the zimbra user UID on the old server.

If you use secondary disks, you can mount them on your new operating system without any problems.  The filesystem is not changed from 32bit to 64bit.

Now your /opt/zimbra/* is the same on both hosts (or upgraded for new hardware by install.sh), and you have imported the LDAP DB, your hostname should be the same as before. You should be able to start Zimbra on the new host.


LVM is not as useful in a virtual server environment

Posted June 17th, 2011 by bryanr

It wasn’t that long ago that I decided to partition a couple of our new servers using the LVM file system.  I have read that using the LVM (logical volume manager) is much for flexible than using standard ext3 or ext4 partitions.  This is true when you are using a physical server with physical hard drives, because when you run out of disk space, your only option is to add additional hard drives.  Using logical volumes makes it very easy to add new disks to existing volume groups and expand the volumes.

But, when your servers are in a virtual environment, you can easily make the hard drive larger with a setting change.  Expanding volumes and expanding ext partition both require server downtime too, so there aren’t many advantages.  Let’s just compare the steps taken to expand both file systems on a virtual server.

For logical volumes:

1. Increase the size of the vdisk, and reboot.

2. On guest OS, create a new partition to use free space (‘fdisk -l’ and ‘fdisk /dev/sda’), and reboot.

note: the new lvm partition must have an ID type “8e” in fdisk

note: if you are using and extended partition, you will have to boot to a partition utility to resize it first (gparted works great)

3. Create a physical volume on the new partition with pvcreate.

4. Extend the existing volume group to the new partition with vgextend.

5. Expand the logical volume to fill the volume group (use ‘lvextend -l +100%FREE /dev/vg/lv’ to use the remaining space so you don’t have to calculate the exact size.  ‘lvresize’ can also be used.)

6. Expand the filesystem to fill the logical volume (‘resize2fs -p /dev/vg/lv’)

note: use ‘lvdisplay’ to find the correct path to your volume groups

Here is the same process for standard partitions:

1. Increase the size of the vdisk, and reboot.

2. Boot to a partition utility and resize your current partition to fill the new disk size (I use a gparted boot cd iso).

note: if you have multiple partitions, you will have to move the partition around on the disk to resize in certain situations.

I can conclude the standard partitioning is much easier to resize.

Here is a good reference blog for LVM resizing:


Our Apache2 config changes for Drupal web server

Posted June 17th, 2011 by bryanr

Drupal is the tool of choice for our web site team, but it using alot of dynamic page content.  Consequently, that increases the memory usage for the apache server processes.  Each process on our production server uses 30-50MB.  The server currently has 4GB total memory, so that means it should be able to handle 80-130 web connections without swapping to disk.  This simple calculation is something that should be done for every web server to set the MaxClients limit.  Here are some other settings that we are using for Apache2 on our Drupal servers:

Timeout 30

KeepAlive On

MaxKeepAliveRequests 100

KeepAliveTimeout 5

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
StartServers          5
MinSpareServers       5
MaxSpareServers      10
MaxClients          100
ServerLimit         150
MaxRequestsPerChild   2000

Settings the MaxRequestsPerChild variable to 2000 means that each Apache process will be “recycled” 2000 times before it is forced to spawn a new Apache process.  The default is 0, or unlimited, which can lead to memory leaks.

Also set the KeepAliveTimeout low (2-5), to keep processes available for new connections instead of waiting on existing connections.

Here are some good resources that I used:




Quick host lookup tip

Posted June 9th, 2011 by bryanr

If you would like to query for a particular record type for a domain, then this tip is for you.  In my specific case, I needed to make sure that my mail server was reading the txt record correctly for our domain.  You can use this host command to do so:

#host -t txt <domainname>

You should be able to do the same for any domain if you want to see if they have an SPF record in place to cut down on spoofed mail.

Domain level Apache configuration changes in Plesk

Posted June 1st, 2011 by bryanr

Plesk doesn’t provide an easy way to make virtualhost configuration changes in Apache.  When Plesk is installed the default Apache settings are left untouched.  Plesk adds a .conf file to Apache’s conf.d directory that specifies “Include” parameters for the each domain’s configuration file.  The Plesk config files are over written whenever changes are made via Plesk.  If you want to manually make any changes to the Apache configuration for a website, you need to create a vhost.conf file for it in the following directory:


This directory also contains the Apache config files for the domain (these files have names ending with httpd.include).  When the Plesk domain settings are re-saved, Plesk will include the vhost.conf file in the resulting domain httpd.include file.  The easiest way to force the addition of the vhost.conf parameters is to run the following command:

/usr/local/psa/admin/sbin/httpdmng --reconfigure-domain <domain>

I recently had to change the loglevel to debug for a particular website and I was not allowed to add the variable to the site’s .htaccess file.  Adding a vhost.conf file that included “loglevel debug” did the trick.

Mysql Charset and Collation for Moodle

Posted May 31st, 2011 by bryanr

We have migrated and updated our Moodle databases several times over the years, which has resulted in inconsistent database collations.  I am by no means a database expert, but I know how to find the answers to the problems at hand.  Here are some of the steps that I used to upgrade our moodle site from 1.9 to 2.0.

Create a new empty database for the upgraded site:

mysql> CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci;

Use mysqldump to create a backup file and then import the backup into your empty database.

# mysqldump -u user -p  moodledb > data.sql

# mysql -u user -p -h localhost –default-character-set=utf8 moodledbnew < data.sql

Then to make sure you don’t get collation errors during the moodle upgrade process perform one of the following (the second method worked well for me):

1. Backup the database; Dump the database to a .sql or text file; Perform a global search and replace all instances of utf8_general with utf8_unicode; Run a query on the database replacing the existing code

2. In the following script, replace “database_name” with the name of your database and edit the statement template in the parameters of the CONCAT function to have the target character set (and/or collation) of your choice. After that you can just take the statements and batch execute them in the MySQL client of your choice (eg. phpmyadmin, Navicat, the official MySQL client, etc.). Of course, you’re strongly advised to make a backup of your database before you start messing around with character set (and/or collation) conversions.

SELECT  CONCAT('ALTER TABLE `', t.`TABLE_SCHEMA`, '`.`', t.`TABLE_NAME`, '`  CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;') as stmt
FROM `information_schema`.`TABLES` t
AND t.`TABLE_SCHEMA` = 'database_name'

Of course, you will need to download the new moodle version, copy over your themes, modules, and blocks, and adjust your web server settings appropriately.

How to view the running/loaded config for MySQL

Posted May 31st, 2011 by bryanr

I had a heck of a time finding out how to see what the parameters were currently loaded for MySQL on our servers.  By default, when you install MySQL server from the Ubuntu repositories, the my.cnf config file specifies only a few variables.  And the rest just use a default value (my assumption).  We recently had a problem with one of our websites that was being bombarded with registration requests; the database became unresponsive because of the number of sleeping processes that used up all of the system memory.  Since MySQL uses a default wait_timeout value of 28800 (8hrs), the sleeping processes were allowed to eat up system resources.  There were no timeout parameters listed in the my.cnf file at all.  So, if a setting is not specified in your my.cnf file, how do you see what is loaded by default?  Just run the following command:

# mysqld –help
# mysqld –verbose –help

You will have to specify a user when you run the command.

Yourls.org (Your Own URL Shortner)

Posted April 19th, 2011 by petel

We are always looking for quick and easy solutions to problems. Here at Essdack we have an internally used system for our essk.me service built on Drupal. Which works great. In the last few weeks, I have been looking for a solution for a test project with Spockholm that is a little more robust with features like filtering, public or private options, Json returned in shortneing and lengthening, perfomance, and url preview to name a few. We have been experimenting with a free open source url shortening and lengthening service called yourls (Your Own Url Shortner). This service was pretty easy to set up and had many of the features of a tinyurl or bit.ly. We found with a high volume service the database needed to be modified and indexes used. Otherwise, it is a great solution. Kudos to the developers. Check it out at Yourls.org

Using SPF to reduce spam

Posted April 14th, 2011 by bryanr

Recently, our users have been receiving a lot of backscatter email, which are bounce messages from email sent to invalid addresses that the user did not actually send in the first place.  I hope that made sense…  The messages that were sent out used our employee’s email address as the sender.  There is really nothing we can do to stop these spammers from imitating our email addresses and sending these messages.  What we can do is setup spam rules on our email server to identify the messages as spam, and other email admins would have to do the same for their email systems.

The first step is to create an SPF record for your DNS zone, there are some great tools online that make this very easy.  I like http://www.openspf.org.  The SPF setup tool that they provide make the it a quick process.  Once you have the record contents, add it to you DNS zone file as a text record.  I would recommend that you use “-all” in you record, which states email from your domain should only come from the servers specified in the SPF record.  This addition will trigger a hard fail when the SPF record is checked and the message is delivered by a server that is not listed in the record.  If you want to be more lenient, you can use “~all”, which states that servers not specified in you SPF record are not preferred and it will trigger a soft fail.  The problem with using a soft fail is that many spam servers will not add spam points for soft fail unless a custom spam score is added.

We use Zimbra with Spamassassin, and installing SPF query tools was simple.  You will need to find instructions for your specific system.  With Spamassassin, just add your SPF spam scores and other custom scores to /opt/zimbra/conf/spamassassin/local.cf.

Here is a sample:

score SPF_SOFTFAIL 2.000
score SPF_FAIL 5.000
score SPF_HELO_FAIL 5.000

You will want to change the score based on your required spam score.

Here is a quick how to tip.  Scripting between servers is handy for routine file transfers or backups, but in order to make scripts unattended, you will have to add your server credentials to your script in plain text or setup authentication keys for you server.  The authentication keys are very easy to create, but you must always remember to create your key on the client.  Then, you copy the key to ~/.ssh/authorized_keys on the destination server for the user you will that you specify in your script.  I know that seems backwards, but since the server has the client’s key, it has permission to login without a password.  So, moving on, here is the command to create the key on you client; you will then distribute this key to your servers.

sudo ssh-keygen -t dsa -f /root/.ssh/id_dsa -C "your comment"

Note: your root user home directory might have a different path than I specified above.

Next, you need to add the key to the authorized_keys file on the server; here is an easy method:

sudo cat /root/.ssh/id_dsa.pub | ssh root@remote_address 'cat - >> ~/.ssh/authorized_keys'

You should be able to login to the server from the client without using a password now, just make sure you run your command using sudo.