sammitch blog

Testing ZFS Compression Levels

We’re about to replace one of the backup servers at work, and in the process get our feet wet with ZFS. In addition to RAIDZ, something that got my attention was ZFS compression. With our existing backup procedures the mail servers [as large as 600GB with millions of files] are simply fed through tar with gzip compression and stored as a massive, unfriendly backup.tar.gz file. This, coupled with the now relatively small partition size of 1.2 TB was giving us problems with both storing more than a couple days’ worth, and the fact that it was taking in excess of 48 hours to perform a backup.

For years I had been itching to use rsync to back up the mail servers, but the main problem was that the backups would not be compressed, and a secondary problem was how to store more than a single copy of the filesystem. ZFS compression solved the main issue, and a quick spin through the man pages showed that the –delete, –backup, and –backup-dir directives would solve the remaining hurdle.

Given that we would need ZFS to compress several terabytes of data we wanted to test out the various levels of compression available to us in advance. To this end I set up a test machine with the following specs:

• Dell PE2950
• 2x Dual-Core Xeon CPUs
• 4GB RAM
• 5x SAS 10k disks
• FreeBSD 8.2 AMD64

One disk is used for the OS, and the other 4 were used to create a RAIDZ1 pool. Under this pool I initially created 3 ZFS datasets with GZIP compression levels of 1, 4, and 8. For test data I pulled 29GB worth of mail data [maildir format, approx 435k files] from one of our servers to an uncompressed section of the pool, and then copying them to one of the compressed datasets. After seeing the results I created 3 more datasets for GZIP-2, GZIP-3, and LZJB compression.
Continue reading…

Binary to Text Using Bash

So I was looking at some job postings when I came across one with a block of binary data in the description. I was curious to see if it was anything actually relevant to the posting that might help my application, something clever slipped under the HR dept’s nose, or complete gibberish that someone thought looked cool.

Here it is for reference:

01001100 01101111 01101111 01101011 01101001 01101110 01100111 00100000 01100110 01101111 01110010 00111010 00100000 01001001 01010100 00100000 01000010 01110101 01110011 01101001 01101110 01100101 01110011 01110011 00100000 01000001 01101110 01100001 01101100 01111001 01110011 01110100 00100000 01110111 01101000 01101111 00100000 01100011 01100001 01101110 00100000 01110100 01110010 01100001 01101110 01110011 01101100 01100001 01110100 01100101 00100000 01001001 01010100 00100000 01101100 01101001 01101110 01100111 01101111 00100000 01101001 01101110 01110100 01101111 00100000 01110000 01101100 01100001 01101001 01101110 00100000 01101100 01100001 01101110 01100111 01110101 01100001 01100111 01100101 00100000 01110100 01101111 00100000 01100011 01110010 01100101 01100001 01110100 00100000 01110011 01101111 01100110 01110100 01110111 01100001 01110010 01100101 00100000 01110011 01101111 01101100 01110101 01110100 01101001 01101111 01101110 01110011 00100000 01100110 01101111 01110010 00100000 01100011 01101100 01101001 01100101 01101110 01110100 01110011 00001101 00001010

A little Googling, and some finagling with the commands [apparently bc on my system freaks out if obase is not declared first] I came up with this command:

for item in `cat bin.txt `; do echo “obase=10;ibase=2;$item” | bc | awk ‘{printf(“%c”, $1)}’; done

Which translated the block of binary octets in bin.txt to this, complete with spelling mistake:

Looking for: IT Business Analyst who can translate IT lingo into plain language to creat software solutions for clients

Turned out kind of lame, but at least I made a handy little one-liner to share.

Creating aggregate CPU usage graphs in Cacti

A quick TL;DR on Cacti: It is a PHP/mySQL/RRDtool solution for monitoring server performance via SNMP and/or various scripts. After a bit of a learning curve it can be customized quite handily. http://www.cacti.net/

While commiserating with a gent on IRC about Cacti, he mentioned that he didn’t like how Cacti split the graphs for multi-core CPUs into a graph per CPU. This makes it harder to get a read on how much CPU power the server is actually using at any given time. Since I had recently figured out the basics of making custom graphs I thought I’d see if I could make one for this.

Continue reading…

Domain Level Quotas for Postfixadmin

Coming up on a year ago I started working on a new mail product for work. We chose to use Postfix given its reputation and the features it offers. As it stood, and still stands, the only passable web-based administration console is Postfixadmin. The software, while very good, seemed centered around managing a relatively small number of domains in an environment where the domain admins are usually the server admins. I began modifying the code to work with huge numbers of domains, and tighten controls in areas where history has taught me our users like to cause trouble. I submitted patches to allow server/super-admins to control the creation of catchall aliases and unlimited quotas, and the big one was adding domain-level quota support. Continue reading…

Spring Sunset Downtown

So it’s finally spring up in Edmonton, hopefully a real spring now that the freak blizzard a few weeks ago has all melted. I got lucky today and the clouds finally cleared out in time for the sunset and I had some fantastic light to work with.

This one was made of just 3 shots which came together very well this time. The last time I had worked on the sunset pic I noticed that this lens and/or camera does have a bit of a vignette, and I’ve found a balance of filters to get rid of it almost perfectly. There was just a tiny bit of fudging to fill in a gap where there wasn’t any  more photo to go around, but it’s nothing important so I hope no one notices.