How to setup Apache as a Tomcat proxy

In this post we're going to setup Apache to act as a proxy for the Tomcat application server on Ubuntu. First off we need to install the "tomcat6" package from the Ubuntu repositories, which is as simple as:

sudo apt-get install tomcat6
and answering "Y" to download Tomcat along with all of its dependencies. To make sure that the Tomcat server is running, try to open up port 8080 on the machine in your browser. If all is well you will see the Tomcat server Welcome page. If not, you may need to start up the server, which can be done with:

sudo service tomcat6 start

Next we need to install the Apache HTTP server, which "apt-get" also makes easy for us:

sudo apt-get install apache2

Again, just enter "Y" when asked whether to download the package and all of its dependencies.

In order to enable Apache to act as a proxy for Tomcat, we're going to need to make use of the "proxy" and "proxy_http" modules. Unfortunately these two modules don't come enabled by default, so we're going to have to enable them and restart apache for the changes to take effect:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo service apache2 restart

Now we need to tell the proxy module how to proxy the requests and where to proxy them to. For this I've created a "tomcat-proxy" file under /etc/apache2/sites-available/, which we're going to enable using Apache's a2ensite command. The file itself looks like the following:

ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 1000
TimeOut 1000
#
# Configure the mod_proxy
#
ProxyPass / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/

After editing the file we enable the site and reload Apache's configuration:

sudo a2ensite tomcat-proxy
sudo service apache2 reload

And that's it! If everything's gone to plan we should be able to hit up port 80 on our server and get the Tomcat welcome page.

Note that this isn't the only way to configure Apache as a proxy. A more sophisticated way is to make use of the AJP protocol/module, which is custom designed to work with Tomcat.

To get Apache proxying to Tomcat using the AJP protocol, we have to enable the Apache module and restart Apache for the changes to take effect:

sudo a2enmod proxy_ajp
sudo service apache2 restart
Next we have to enable the AJP connector in Tomcat. This is done in the /etc/tomcat6/server.xml file. If you edit this file you'll need to uncomment a line, which looks like:

<connector port="8009" protocol="AJP/1.3" redirectport="8443"></connector>

and restart Tomcat:

sudo service tomcat6 restart

The next step is to simply go back to our configuration file under /etc/apache2/sites-available/tomcat-proxy and change the protocol and port of the URL's we supplied the ProxyPass and ProxyPassReverse directives:

ProxyRequests Off
ProxyPreserveHost On
ProxyTimeout 1000
TimeOut 1000
#
# Configure the mod_proxy
#
ProxyPass / ajp://127.0.0.1:8009/
ProxyPassReverse / ajp://127.0.0.1:8009/

You'll notice we've replaced "http" in the URL with the custom "ajp" protocol. Restart Apache and hitting port 80 on the server should redirect you to the Tomcat welcome page as before.

What's the difference between the two approaches to proxing? Functionally there's not really any difference that the user gets to see. However, the AJP is a binary protocol, compared to the regular HTTP proxy method that the first approach uses. This should mean less data passed between the proxy and application server as well as lower latencies. Look out for a future post benchmarking the two approaches to see what the real-world difference in performance between the two approaches is.

How to export your Blogger posts to Wordpress

I've been trying to export my Blogger posts to Wordpress for some time now. The reason was to assure myself that if in the future I wanted to migrate away from Blogger for whatever reason, there was an easy way to transfer all of the content to an alternative system. In order to do this, I was trying to get the Blogger Importer plugin to work for ages, but it constantly errored out, giving a message about Google denying the request due to it being malformed.

I had almost given up on finding a way to import the content, when a google search led me to the "Importing Content" section of the Wordpress Codex. This was useful as it lead me to this page:

http://blogger2wordpress.appspot.com/

which usefully converts your blogger export file into the Wordpress format and gives it to you as a downloadable file. After downloading, I just went to the "Tools", "Import" section of the Wordpress admin console, selected the "Wordpress" link, installed the plugin, selected the file and viola! The posts were imported along with the images which had been saved as attachments. The only missing feature that I've found is that the import didn't import the labels that accompanied each post.

Writeable Rsync server with authentication

In my previous post we talked about how to setup a simple read-only rsync server. In this post,  we'll be taking that simple read-only example and expanding it to allow multiple users, each with their own credentials.

In order to do this, we need to modify the /etc/rsyncd.conf file, and change the following lines:

read only = no
auth users = bozo
secrets file = /etc/rsyncd.secrets

Note that "bozo" is the username of a fake user we're going to create. We can see in this configuration that we're pointing to a file under /etc/rsyncd.secrets. We're going to have to create this file and populate it with the credentials for any users we have created. In this case, we populate it with bozo's username and password:

bozo:clown

We also have to set the permissions on this file to make sure that it's only readable by the root user, using the chmod command:

chmod 600 /etc/rsyncd.secrets

Now, usually we would run the "reload" command to send a message to the server to reload its configuration, but when we do this for rsync, we get the following message:

$ service rsync reload
 * Reloading rsync daemon: not needed, as the daemon
 * re-reads the config file whenever a client connects.

Which is very useful indeed. Now when we connect from the client side, we have to do so using the credentials which we've just created. The command looks like:

rsync -r bozo@192.168.1.10::public/ .


This will ask us for a password, which we know is "clown" from before after which the copy should start as usual. If you want to test out the write capability of the server, we just need to create a file in our current directory and then execute the rsync command going the other way:

rsync -r . bozo@192.168.1.10::public/

In the future we might want to automate the rsync process in order to have it run as a cron job or other automated job. This means that we won't have a human there to enter the password. This can be gotten around with by using the "--password-file" option of the rsync command, like so:

rsync --password-file=~/.rsync_pass -r . bozo@192.168.1.10::public/

Note that as with the rsyncd.secrets file mentioned previously, you'll have to change the permissions on this file to ensure that it's not world readable. The file itself just needs to contain the password to use and nothing else.

Simple read-only rsync server on Ubuntu

If you haven't heard of it rsync is a piece of software which allows you to keep files in sync over a network, while only copying across the "changes" from one copy to the next. The advantages of this are that a lot less data needs to be transferred than would have to be done with something like FTP or SFTP. This attribute of rsync also makes it perfect for things like backups which don't change much from one iteration to the next.

Installing rsync is as simple as:

sudo apt-get install rsync

Although, I've found that with the server version of Ubuntu, it's already installed after installing the OS.

By default, the server doesn't come configured or enabled to start at boot. In order to configure it, we will need to copy across the example rsync configuration into the /etc directory and modify the /etc/default/rsync file:

sudo cp /usr/share/doc/rsync/example/rsyncd.conf /etc/

Modify the /etc/default/rsync file to look like the following:

# defaults file for rsync daemon mode

# start rsync in daemon mode from init.d script?
#  only allowed values are "true", "false", and "inetd"
#  Use "inetd" if you want to start the rsyncd from inetd,
#  all this does is prevent the init.d script from printing a message
#  about not starting rsyncd (you still need to modify inetd's config yourself).
RSYNC_ENABLE=true

# which file should be used as the configuration file for rsync.
# This file is used instead of the default /etc/rsyncd.conf
# Warning: This option has no effect if the daemon is accessed
#          using a remote shell. When using a different file for
#          rsync you might want to symlink /etc/rsyncd.conf to
#          that file.
# RSYNC_CONFIG_FILE=

# what extra options to give rsync --daemon?
#  that excludes the --daemon; that's always done in the init.d script
#  Possibilities are:
#   --address=123.45.67.89 (bind to a specific IP address)
#   --port=8730 (bind to specified port; default 873)
RSYNC_OPTS=''

# run rsyncd at a nice level?
#  the rsync daemon can impact performance due to much I/O and CPU usage,
#  so you may want to run it at a nicer priority than the default priority.
#  Allowed values are 0 - 19 inclusive; 10 is a reasonable value.
RSYNC_NICE=''

# run rsyncd with ionice?
#  "ionice" does for IO load what "nice" does for CPU load.
#  As rsync is often used for backups which aren't all that time-critical,
#  reducing the rsync IO priority will benefit the rest of the system.
#  See the manpage for ionice for allowed options.
#  -c3 is recommended, this will run rsync IO at "idle" priority. Uncomment
#  the next line to activate this.
# RSYNC_IONICE='-c3'

# Don't forget to create an appropriate config file,
# else the daemon will not start.

The only variable that's really changed from the default is the "RSYNC_ENABLED" which has been set to "true".

If we have a look at the config file under /etc/rsyncd.conf, we can see that we're allowing read-only access to the /var/www/pub directory to any user:


# sample rsyncd.conf configuration file

# GLOBAL OPTIONS

#motd file=/etc/motd
#log file=/var/log/rsyncd
# for pid file, do not use /var/run/rsync.pid if
# you are going to run rsync out of the init.d script.
# pid file=/var/run/rsyncd.pid
#syslog facility=daemon
#socket options=

# MODULE OPTIONS

[public]

comment = public access
path = /var/www/pub
use chroot = yes
# max connections=10
lock file = /var/lock/rsyncd
# the default for read only is yes...
read only = yes
list = yes
uid = nobody
gid = nogroup
# exclude =
# exclude from =
# include =
# include from =
# auth users =
# secrets file = /etc/rsyncd.secrets
strict modes = yes
# hosts allow =
# hosts deny =
ignore errors = no
ignore nonreadable = yes
transfer logging = no
# log format = %t: host %h (%a) %o %f (%l bytes). Total %b bytes.
timeout = 600
refuse options = checksum dry-run
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

Now all we need to do is to create the folder and start up the rsync server:

sudo mkdir -p /var/www/pub
sudo service rsync start

In order to test the server, you can just drop any files into the /var/www/pub directory and then download them using:

rsync -r [hostname/IP address]::public/ .


e.g. rsync -r 192.168.1.10::public/ .

This will copy across all of the files from /var/www/public into your current directory. Note that if you leave out the dot at the end, it will merely display the list of files under /var/www/pub. Another thing to note is that by default the Rsync server uses TCP port 873 to communicate with the rsync client, so you may have to open up this port on your firewall if it is blocked.

pwgen - Generate random passwords on Linux

There's a useful package in Debian/Ubuntu called pwgen, which allows you to generate random, human pronouncable (this is moot) passwords.

It works simply by running the 'pwgen' binary:

$ pwgen Teboo0sh Rahz3Jee aeWae1mn isheL9oo Ahbubo6o fie7ow7L eij3Re0i ieCheh2A oSae0pah uGu1Co0k Pa0PhieZ riope6Ie IeC6aiYi zie4Yahx Yoh0quae yab2iCae Ooqu2wei chel2ohG EeSh5jok hoxoZa7o He8gaale gao6EiSh Uo8loh1b Phie2gie Ehei7ais yeicoo4Z Een1ohcu duZ9ook6 aQuu3wei YuW4gaen soh8ueCh Phohwai5 bi9bu4Li ieWah7ae Aip5Ohv0 lieM1aiG raeF6voe Fooduo9a pohqu3Da Ahn0iRio Uwaech6U ne8Quu9b AhV3oNee zieG1thi Shai1Chu Zae0pie1 aet1geFe Ko8wi4go

It also comes with some useful options such as: -y which adds a random character to each password -N which allows you to specify the number of passwords generated (by default the entire terminal is filled up with passwords) -H which allows you to generate repeatable passwords by using a file and a piece of text to seed the random number generator You can install pwgen on Debian/Ubuntu using:

apt-get install pwgen

For a full list of options have a look at 'man pwgen'.