The information on converting to and from PuTTY key formats without starting up the PuTTYgen GUI is quite few and far between, as I’ve found.
The key bit is that PuTTY is available as a package for most Linux distros, and comes with PuTTYgen as well.
To convert your existing key, simply use:
puttygen openssh_private_key -o output_filename.ppk
And if you’d like to generate new OpenSSH and PuTTY keys, use something like:
ssh-keygen -t rsa -b 2048 -C "$fullname's Key" -N "" -f $username && \
puttygen $username -o $username.ppk
Which will generate files testy, testy.pub, and testy.ppk.
I’d imagine that the PuTTYgen utility will accept the same command format on Windows as well if you’ve already got your OpenSSH keys handy there.