Movable Type on Ubuntu with Nginx and fcgiwrapper

The recent proliferation of static site generators has piqued my interest in leaving Wordpress which is sometimes unresponsive. Admittedly, this isn't hard given my short attention span. I thought, "Here it is, the reason why my obscure writings won't turn into a hugely influential blog is that the website is too slow. It needs to be static pages so it can scale to handle the the audience of the whole internet.", or something like it.

While deciding which python site generator use, I came across this discussion at HN.

http://news.ycombinator.com/item?id=89663

The comment that caught my eye was the link to Jeff Attwood's explanation as to why he uses Movable Type.  His explanation is along the lines that he chooses the best tool for the job. What I heard was: the benefits of a static site generator, without glueing everything together and writing html and markdown. This commenced the beginning of another wild goose chase, better known as the path to yak shaving. Here are the brief working instructions to Movable Type on Ubuntu with fcgiwrapper and nginx. 

1. Download MT5...

  $ wget http://movabletype.org/stable/MT-5.0-en.zip

2. Unzip it

e.g. unzip MT-5.0-en.zip mt

3. Make some directories and then add symbolic links:

cd /var/www/
mkdir cgi-bin
mkdir html
ln -s /home/ubuntu/mt  /var/www/cgi-bin/mt
ln -s /home/ubuntu/mt-static /var/www/html/mt-static

4. Install nginx, spawn-fcgi and fcgiwrap

sudo apt-get install nginx spawn-fcgi libfcgi0ldbl
sudo apt-get install build-essential libfcgi-dev

5. Modify /etc/init.d/fcgiwrap to set the user and group to www-data or ubuntu. I know it's hacky in a bad way and security risk, but it works.

#FCGI_APP Variables
FCGI_CHILDREN="1"
FCGI_SOCKET="/var/run/$NAME.socket"
FCGI_USER="ubuntu"
FCGI_GROUP="ubuntu"
# Socket owner/group (will default to FCGI_USER/FCGI_GROUP if not defined) FCGI_SOCKET_OWNER="ubuntu" FCGI_SOCKET_GROUP="ubuntu"

 6. Create a socket file for fcgiwrap and set its permissions such that fcgiwrap can access it.

touch /var/run/fcgiwrap.socket
sudo chown ubuntu:ubuntu /var/run/fcgiwrap.socket

7. Nginx config file. This was the biggest pain point for me until I found link [4]:

server {
     server_name mt.bhoung.com;

     root /home/ubuntu/blog/;

     location / {
        index index.html index.htm;
     }

     location ~ ^/cgi-bin/.*\.cgi$ {
     gzip off; #gzip makes scripts feel slower since they have to complete before getting gzipped
     fastcgi_pass unix:/var/run/fcgiwrap.socket;
     fastcgi_index index.cgi;
     fastcgi_param SCRIPT_FILENAME /var/www/$fastcgi_script_name;
     fastcgi_param QUERY_STRING $query_string;
     fastcgi_param REQUEST_METHOD $request_method;
     fastcgi_param CONTENT_TYPE $content_type;
     fastcgi_param CONTENT_LENGTH $content_length;
     fastcgi_param GATEWAY_INTERFACE CGI/1.1;
     fastcgi_param SERVER_SOFTWARE nginx;
     fastcgi_param SCRIPT_NAME $fastcgi_script_name;
     fastcgi_param REQUEST_URI $request_uri;
     fastcgi_param DOCUMENT_URI $document_uri;
     fastcgi_param DOCUMENT_ROOT $document_root;
     fastcgi_param SERVER_PROTOCOL $server_protocol;
     fastcgi_param REMOTE_ADDR $remote_addr;
     fastcgi_param REMOTE_PORT $remote_port;
     fastcgi_param SERVER_ADDR $server_addr;   
     fastcgi_param SERVER_PORT        $server_port;
     fastcgi_param SERVER_NAME        $server_name;
   }
   
   # for static files 
   location /mt-static {
       root /var/www/html;
   }
}

8. As I was importing my entries from wordpress I hit a file size limitation in nginx. 

# vi /usr/local/nginx/conf/nginx.conf
client_max_body_size 2M;

* I know I've skipped a few steps like the MySQL and the mt-config.cgi file... I'll get back it a bit later

[1] http://www.movabletype.org/documentation/installation/
[2] https://help.ubuntu.com/community/FcgiWrap

[3] http://library.linode.com/web-servers/nginx/perl-fastcgi/ubuntu-10.04-lucid

[4] http://wiki.nginx.org/SimpleCGI
[5] http://www.cyberciti.biz/faq/linux-unix-bsd-nginx-413-request-entity-too-large/
[6] http://www.livejournal.com/doc/server/lj.install.perl_setup.modules.html

Google Maps Overlays with TileMill QGIS and R

Check it out: http://bhoung.s3-website-ap-southeast-2.amazonaws.com/

I had been meaning to replicate the cool maps that Ben Boyer made when he was at the Chicago Tribune (population density map of Chicago: http://media.apps.chicagotribune.com/census-2010/census-demo/index.html) .

I'm glad to be able to say that I successfully completed his tutorial a few weeks ago, with Australian data. I finally found a few minutes to upload the modified code onto Amazon's S3 Buckets service. The main obstacle to following the tutorial for anyone interested was trying to generate the images using Mac OSX 10.8.2. As most of the pieces of software are constantly being updated, it was easiest to fire up Ubuntu and run invar, an open source python script to generate the images, rather than trying to install it on the Mac....

No surprises as to what population density looks like in Australia, but it's pretty nonetheless. Extensions would be look at other kinds of data, perhaps at the postcode level. Also, implementing the layering with dots as per some of the Chicago Tribune's other maps would be great too.

Update: So the time has come for me to replicate these maps with new data. I'll document as I go this time round.

1. Merging spatial data with shapefiles can be done more easily than using postgres with either R (http://gis.stackexchange.com/questions/6839/best-way-to-add-attribute-data-to-a-shape-file) or the mmqgis plugin in QGIS.

2.Carto to conver tilemill mml  file to an xml file

3. install pip and then invar

https://github.com/mapnik/mapnik/wiki/UbuntuInstallatio

  • sudo apt-get install python-mapnik mapnik-utils libmapnik0.7
  • sudo apt-get install python-pip
  • sudo apt-get install python-greenlet
  • sudo pip install invar
  • ivtile -h
  • ivtile project.xml . -9.3 96.9 -43 159 10 16 -p 2

notes for common lisp / emacs / slime

I've carefully watched Marco Baringer's screencast on slime and emacs again.
This time, I've taken notes, and better understood what is happening. It has helped to resolve my misunderstanding of how common lisp operates with slime/swank.

For some reason, I had ignored the importance of why swank and slime was required and or beneficial to common lisp.

*
**
***

C is Control
M is Alt

Look up a command:
C-c C-d d is slime-describe-symbol
C-c C-d h is hyperspec-lookup

C-x C-s is save work without exiting

C-M-X is slime-eval-defun - finds top level form that contains the point/cursor and evaluates it.

C-x C-e is slime-eval-last-expression - takes form preceding cursor and evaluates it.

inspect package using slime-inspector...
First, (find-package :morse)
This returns a symbol.
M-x slime-inspect
* i.e. the symbol representing the package. Hit enter on highlighted text to read descriptions.
n for next, l to go back.

C-M-f
C-M-b
C-M-n
C-M-p
forward and back s-expressions. next and previous which seem really similar.
C-M-u
C-M-d
up and down nested levels.

C-t transpose s-expressions.


M-n
M-p
next or previous note / warning

C-c C-t to trace function in file
then (untrace) in REPL.

C-c < slime-list-callers
see who calls this function

C-c > is slime-list-callees

ggmap

ggmap is a fantastic package on R. I've previously blogged about rgooglemaps but ggmap supercedes (and appears to even build on it - looks to be the case when you install it). It's the equivalent of ggplot for maps and it is very easy to use. In this example I overlay some data points on a map. You may have seen David Kahle's earlier work using ggplot with Houston's crime data here.

 

Retail Volumes Apr' 2012

From all reports, the retail (and services) market in Melbourne has been pretty miserable. It doesn't bear out in the official numbers though, yet.

Choropleth Map Time Lapse with R statistics, ggplot and ffmpegx

Melbourne Unemployment 2008 to 2011 - Revisited I have previously created a choropleth map showing unemployment rates for Victoria. After seeing Drew Conway's animated clock via R-bloggers, I thought I'd give it a go with my map on unemployment rates. Not surprisingly, unemployment rates remain low in the inner-east of Melbourne, while the north and west, and outer south east tend to have higher rates.

AFL teams' personalities 2010 & 2011

The content of this blog post is the relationship between a team's performance and it's perceived favouritism. Some handle it better than others.

These graphs show betting odds (converted to probabilities) against the actual results (as measured by the margins of the matches). The colour of the points indicate home and away games where home games are those where the opposing team is not from the same state.

Notes:

Geelong goes against the odds away and with the odds at home.
PA wins at home when close-ish. NM wins close games.
Adelaide uniformly wins and loses away, and drops more games than expected at home.
In order of flakeyness: Stk, Ade, Gee, Ess, PA, Haw, Car, NM, WB


Here's the same for 2011:

AFL 2012

The season proper is just around the corner. I wonder how the ladder positions have trended over time?