Latest Entries

Read-Only Elasticsearch Proxy using Nginx

I’ve recently set up an Elasticsearch/Logstash/Kibana cluster for a project and found Elasticsearch’s complete lack of authentication/authorization a bit alarming. I wasn’t exactly exposing these endpoints publicly, but I also didn’t want an inquisitive/careless/mischievous employee deleting documents or indexes.

However, the one exception I did want to make was to allow modifications to documents in the ‘kibana-int’ index so that users could create and modify dashboards.

So I’ve come up with the following Nginx config file for this:

upstream elastic_cluster {
	server elasticsearch-01.fooblah.com:9200;
	server elasticsearch-02.fooblah.com:9200;
}

server {
	listen 9200;
	server_name kibana.fooblah.com;
	client_max_body_size 50m;

	proxy_redirect off;
	proxy_set_header Connection "";
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header Host $http_host;
	proxy_pass_header Access-Control-Allow-Origin;
	proxy_pass_header Access-Control-Allow-Methods;
	proxy_hide_header Access-Control-Allow-Headers;
	add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
	add_header Access-Control-Allow-Credentials true;

	# disallow write/modify to all indices
	location / {
		limit_except GET POST HEAD OPTIONS {
			# you could use auth_basic options here to allow HTTP-Auth
			deny all;
		}
		proxy_pass http://elastic_cluster;
	}
	# re-allow write/modify to kibana-int index for dashboard storage
	location /kibana-int/ {
		proxy_pass http://elastic_cluster;
	}

}

Hopefully someone finds this useful!

Converting OpenSSH keys to PuTTY format via command line

The information on converting to and from PuTTY key formats without starting up the PuTTYgen GUI is quite few and far between, as I’ve found.

The key bit is that PuTTY is available as a package for most Linux distros, and comes with PuTTYgen as well.

To convert your existing key, simply use:

puttygen openssh_private_key -o output_filename.ppk

And if you’d like to generate new OpenSSH and PuTTY keys, use something like:

username=testy
fullname="Testy McTest"
ssh-keygen -t rsa -b 2048 -C "$fullname's Key" -N "" -f $username && \
puttygen $username -o $username.ppk

Which will generate files testy, testy.pub, and testy.ppk.

I’d imagine that the PuTTYgen utility will accept the same command format on Windows as well if you’ve already got your OpenSSH keys handy there.

Create and mount a file as a disk in Linux

If you need to test something where a disk needs to be mounted, but don’t want to go through the hassle of actually attaching a physical disk or provisioning a virtual one you can simply create a ‘loopback’ device in linux.

I’ve been working on project where in the future I might want to export a big pile of disks via NFS, but I don’t feel like provisioning a bunch of temporary devices for it. There are a few other tutorials out there on how to do this, but they either have extraneous commands [ie losetup] or don’t deal with actually creating the image. Continue reading…

Encoding and Decoding Large Numbers in PHP

I found an interesting question on StackOverflow today. Not necessarily because I have any interest in BitCoins [I don't], but because the asker needed to convert the format of some very large numbers. In this case the number turned out to be 58 digits in base 10, and PHP has a tendency to crap out somewhere after 11 to 19 digits.

The first problem I tackled was writing a proper base58 encoding function since the ones floating around the internet seem to be spotty at best. But why limit it to just one encoding? I’ve written the same function a handful of times already, all with different, strange, and sometimes scrambled bases. I’d definitely been doing it wrong.

This is what I came up with:

function arb_encode($num, $basestr) {
	$base = strlen($basestr);
	$rep = '';

	while($num > 0) {
		$rem = $num % $base;
		$rep = $basestr[$rem] . $rep;
		$num = ($num - $rem) / $base;
	}
	return $rep;
}

Continue reading…

Copy and Paste Files Between SSH Sessions

I’ve found that the permissions in the directories on the servers I’ve been working on recently are not very friendly to using scp or rsync, [root owns the dir, but sshd PermitRootLogin = No] but I need to copy files around regularly. I’ve used a simple cat file | base64 to embed file contents in scripts before, so why not pair it up with tar to move many files?

I’ll save the sob story where I found that tar by default pads with a LOT of null bytes, but that’s why -z and -b 1 are your friends.
Continue reading…



Copyright © 2009–2010. All rights reserved.

RSS Feed.