sammitch blog

HTTP Compression Self-Test in PHP

I’ve noticed that it can be difficult for many people to determine whether or not their server software is already compressing their output for them independently of what they have in their code. Most browsers do not present this information without an add-on, and Apache and IIS do not provide any information via the $_SERVER array as to whether compression is turned on, so the only way to check is to make a request against your server and inspect the response.

I’ve created the following script to simplify this process, just drop it somewhere on your server and request it with your web browser.. Continue reading…

Simple PHP PDO Wrapper Class

For the last few years I’ve been using PHP Data Objects [PDO] in all of my projects, mainly for the use of parameterized queries to avoid SQL injection. Now with PHP pushing to remove the old mysql_* functions and get the community into PDO or mySQLi I’ve found myself sharing this code more and more often. So I’m finally posting it up here so I don’t have to keep copy/pasting this after editing out all the stuff that’s specific to me.

As a bonus, this also implements the Singleton design pattern, so you can use a static Mysql_DB::getInstance() function anywhere in you code to grab the instance of the database with very little hassle.
Continue reading…

Calculating the Number of Weekdays in a Given Interval

Not having the most to do on a given workday at this new job I’ve spent quite a bit of time on StackOverflow.com answering questions, mostly in the PHP and mySQL tags. Usually it’s just playing “find the missing semicolon” or “type the function name into the documentation site’s search”, but once in a while something interesting gets asked. This question really got me going last week, and I was more than a little bummed that it didn’t get much attention, so I’m posting it here.

The question was asked under the mySQL tag, so it’s written a little funky to be able to be calculated for each row in a result set. [just replace @start and @end with the appropriate field names] But this can also be easily translated into other languages with the following notes:

1. mySQL’s DATEDIFF() function is not inclusive, hence adding 1 to the result.
2. mySQL’s DAYOFWEEK() function returns values 1 [Sunday] through 7 [Saturday].
3. mySQL’s IF() function is IF( [logical expression], [value if true], [value if false]).

SET @start  = '2012-09-30';
SET @end    = '2012-11-03';

SELECT
    @raw_days   := DATEDIFF(@end, @start)+1 'raw_days',
    @full_weeks := FLOOR(@raw_days / 7) 'full_weeks',
    @odd_days   := @raw_days - @full_weeks * 7 'odd_days',
    @wday_start := DAYOFWEEK(@start) 'wday_start',
    @wday_end   := DAYOFWEEK(@end) 'wday_end',
    @weekend_intrusion  := @wday_start + @odd_days 'weekend_intrusion',
    @extra_weekends     :=
        IF(@wday_start = 1, IF(@odd_days = 0, 0, 1),
            IF(@weekend_intrusion > 7, 2,
                IF(@weekend_intrusion > 6, 1, 0)
            )
        ) 'extra_weekends',
    @total_weekends     := @full_weeks * 2 + @extra_weekends 'total_weekends',
    @total_workdays     := @raw_days - @total_weekends 'total_workdays'

Results:

+----------+------------+----------+------------+----------+-------------------+----------------+----------------+----------------+
| raw_days | full_weeks | odd_days | wday_start | wday_end | weekend_intrusion | extra_weekends | total_weekends | total_workdays |
+----------+------------+----------+------------+----------+-------------------+----------------+----------------+----------------+
|       34 |          4 |        6 |          1 |        6 |                 7 |              1 |              9 |             25 |
+----------+------------+----------+------------+----------+-------------------+----------------+----------------+----------------+

Try it yourself on SQLfiddle.com.

Testing ZFS Compression Levels

We’re about to replace one of the backup servers at work, and in the process get our feet wet with ZFS. In addition to RAIDZ, something that got my attention was ZFS compression. With our existing backup procedures the mail servers [as large as 600GB with millions of files] are simply fed through tar with gzip compression and stored as a massive, unfriendly backup.tar.gz file. This, coupled with the now relatively small partition size of 1.2 TB was giving us problems with both storing more than a couple days’ worth, and the fact that it was taking in excess of 48 hours to perform a backup.

For years I had been itching to use rsync to back up the mail servers, but the main problem was that the backups would not be compressed, and a secondary problem was how to store more than a single copy of the filesystem. ZFS compression solved the main issue, and a quick spin through the man pages showed that the –delete, –backup, and –backup-dir directives would solve the remaining hurdle.

Given that we would need ZFS to compress several terabytes of data we wanted to test out the various levels of compression available to us in advance. To this end I set up a test machine with the following specs:

• Dell PE2950
• 2x Dual-Core Xeon CPUs
• 4GB RAM
• 5x SAS 10k disks
• FreeBSD 8.2 AMD64

One disk is used for the OS, and the other 4 were used to create a RAIDZ1 pool. Under this pool I initially created 3 ZFS datasets with GZIP compression levels of 1, 4, and 8. For test data I pulled 29GB worth of mail data [maildir format, approx 435k files] from one of our servers to an uncompressed section of the pool, and then copying them to one of the compressed datasets. After seeing the results I created 3 more datasets for GZIP-2, GZIP-3, and LZJB compression.
Continue reading…

Binary to Text Using Bash

So I was looking at some job postings when I came across one with a block of binary data in the description. I was curious to see if it was anything actually relevant to the posting that might help my application, something clever slipped under the HR dept’s nose, or complete gibberish that someone thought looked cool.

Here it is for reference:

01001100 01101111 01101111 01101011 01101001 01101110 01100111 00100000 01100110 01101111 01110010 00111010 00100000 01001001 01010100 00100000 01000010 01110101 01110011 01101001 01101110 01100101 01110011 01110011 00100000 01000001 01101110 01100001 01101100 01111001 01110011 01110100 00100000 01110111 01101000 01101111 00100000 01100011 01100001 01101110 00100000 01110100 01110010 01100001 01101110 01110011 01101100 01100001 01110100 01100101 00100000 01001001 01010100 00100000 01101100 01101001 01101110 01100111 01101111 00100000 01101001 01101110 01110100 01101111 00100000 01110000 01101100 01100001 01101001 01101110 00100000 01101100 01100001 01101110 01100111 01110101 01100001 01100111 01100101 00100000 01110100 01101111 00100000 01100011 01110010 01100101 01100001 01110100 00100000 01110011 01101111 01100110 01110100 01110111 01100001 01110010 01100101 00100000 01110011 01101111 01101100 01110101 01110100 01101001 01101111 01101110 01110011 00100000 01100110 01101111 01110010 00100000 01100011 01101100 01101001 01100101 01101110 01110100 01110011 00001101 00001010

A little Googling, and some finagling with the commands [apparently bc on my system freaks out if obase is not declared first] I came up with this command:

for item in `cat bin.txt `; do echo "obase=10;ibase=2;$item" | bc | awk '{printf("%c", $1)}'; done

Which translated the block of binary octets in bin.txt to this, complete with spelling mistake:

Looking for: IT Business Analyst who can translate IT lingo into plain language to creat software solutions for clients

Turned out kind of lame, but at least I made a handy little one-liner to share.