Archived entries for python

SSH to an EC2 instance Without Knowing the Hostname

If you’re not ponying up an extra few bucks per month for an EC2 Elastic IP it can be a pain in the ass to reach your EC2 instances, particularly after they’ve rebooted and gotten a new IP and hostname. Fortunately it is possible, though not obvious, to retrieve the public DNS name of an EC2 instance based on its Instance ID and Availability Zone.

To this end I’ve created two scripts, a Python script [ec2-get-dns] that uses boto to retrieve the hostname, and a bash script [ec2-ssh] that gets the hostname and connects via SSH.

Shortcuts

Oh wait, you say it’s a pain in the ass to remember instance IDs? Really? I always know that i-35213355 is the web server and i-54632856 is the database server, but OK. This also reads named shortcuts stored in a file [.ec2list] in your home directory.

#shortcut-name:region:instance-id[:username]
web-server:us-west-2:i-35213355
db-server:us-west-2:i-54632856:db-user

As you can see above the username is optional. Connecting to the ‘web-server’ entry will default to ‘ec2-user’ while ‘db-server’ will use ‘db-user’. Continue reading…

Quick Look: Switching from PHP to Python

For the last few years it seems no one would shut up about how great Python is.

It’s easy, and intelligent, and intuitive, and so great, and it cleaned my house, did my taxes, satisfied my wife, raised my children, yadda, yadda, yadda…

Apparently this is also the language they’re teaching to new CompSci students, and even some high school students as well.

So you could understand why it was so frustrating that I couldn’t pick it up even after 2 or 3 attempts. The lack of bracket-enclosed code blocks, the ‘everything is an object with methods’ philosophy, and 3 different kinds of array turned every script I looked at into an unreadable mess, and every script I tried to write into an error-ridden monument to how much I didn’t get it.

Until 2 weeks ago, that is. I had just been hired as a PHP dev when our lead turned around and said something along the lines of “PHP is bad, all future projects will be Python-based”. Now was as good a time as any to learn. I picked up a task that was to take a simple PHP upload script with a problem [eg: PHP can’t handle file uploads over 2GB] and re-write it in Python. If you have a Python learning block I suggest you do something like this instead of half-heartedly trying “Hello World!” scripts.

Now I’m not exactly a Python pro, but here are a few tips to help you make the move. Continue reading…

Handling Large Requests in Twisted

So working on a bit of code for a video media related project I was given a PHP script that handled uploads, and told to move it to Python because of some funky things in PHP that prevent it from accepting file uploads larger than 2GB properly. I was strongly encouraged to use the Twisted framework as that’s what they’d like to move everything to in the future.

Twisted seemed as easy as advertised, but I found that every time I uploaded a file the memory usage ballooned up two or three times as large as the file I was uploading. I did some Googling and some asking around and mostly came up with references to 10-year-old Bug 288, suggestions that my issue would no longer be an issue when 288 is fixed, and apologies.

With some help from the guys in #twisted.web on FreeNode I tracked down the calls and found that while it’s hard-coded into Twisted to direct any incoming request larger than 100,000 B to a tempfile.TemporaryFile if the detected Content-Type is multipart/form-data it’s sent to Python’s cgi.parse_multipart() function which reads the entire request into memory twice, plus a little extra. Continue reading…



Copyright © 2009–2010. All rights reserved.

RSS Feed.