Migrating from Dynamic To Static

Migrating from Dynamic To Static

Sat 16 May 2015

Finally! My site is now transferred to a new platform. In this article I will outline my motivations, the process and what problems I stumbled into.

My history

Most of my sites, including this one has traditionally been powered by the open source Joomla CMS.

I first got into Joomla when it was at at version 0.x, thus follow the painfull upgrade paths to 1.5, 2.5 and now 3. Ease of upgrade have not been the main focus for the Joomla CMS and Im tired of wasting time on upgrades.

Joomla CMS is written in php which also led to an endless headache of keeping up to date with current versions of php.

The challenges

As most php cmses Joomla also requires an database to store its content in, which means another piece of software that I had to maintain with stability and security in mind.

Another note regarding stability and security is that my process separation mechanism (suphp) and web application firewall (modsecurity) was draining all the resources on my little vps, causing other subsystems on the server to malfunctioning.

I also had a hard time enjoying creating content with the javascript based wysiwyg editor that Joomla use. The editor did strange things with my formatting which meant I often had to edit the html by hand to get the formatting I needed. There is alternative editors available for download but they all mess things up, and they also make yet another piece of code to maintain. Offline editing of content is also a no go, restricting creation of content to the times Im in front of my computer.

The sum of all this was too much administration and not enough content creation.

Something needed to change!

Improvement list

  • First goal is to get rid of the constant upgrade race. I need a way to reduce attack vectors on my server.
  • Second goal is to get a decent content editor to work with. I need full control of the output and ease of adding codesnippets in my articles. I already know MarkDown, so that is a choice if possible.
  • Third goal is to get all my previous written content into the new platform with a minimum of work.

Getting rid of Joomla, php and mysql would help reach the first goal. I looked at some python based systems: Django, Web2Py and Flask. Interesting systems, but still complex and with similar security conserns.

Migrating data from Joomla to Pelican

Joomla content (articles) are stored in the database as various rows containing html code. I.e there is a title, alias and text column which together makes up the content article.

What I needed to do was to export and convert Joomla content to a format that Pelican could work with.

Exporting from Joomla

I used the excellent j2xml plugin (http://extensions.joomla.org/extension/j2xml) that allowed me to export all my articles as XML.

After I exported my articles to XML I started investigate the format and layout of the XML file so that I could write a "converter" for it.

Converting to Pelican

After examining the XML dump from Joomla, and reading a lot of Pelican docs I started to get a sense for what I needed to get a successful convertion.

I ended up with a python snippet that load the XML, parse it and writes out html files in a format that Pelican understand. You can get the script from my github page: https://github.com/kbotnen/joomla2pelican

I decided to use the lxml library to parse the XML, and I made a class, HtmlArticle, that would represent an article. The script then creates a collection of HtmlArticle objects that is written to individual files in Pelican's 'content' folder.

Migrating theme from Joomla to Pelican

My original theme was created with Artisteer and exported as a Joomla template. The templateengine jinja2 that Pelican is using is a total different beast.

I had to read the docs and recreate my layout and css by hand.

It took some time to grasp the concept, but when I did it was an easy task to recreate the look that I want.

Conclusion

I have used Pelican for about 5 months now and gained some experience with it.

  • One of the main reason to switch was to get the fun back into content editing.

I see that when I write in markdown the focus is on the content, and not what it's going to look like on the website. I dont have to fight html tags that is automagically inserted by the editor anymore! I can also create content while offline, and publish it as soon as I get a Internet connection.

  • Updating changes while developing was slow.

Due to the number of articles that had to be generated I had to separate all the old Joomla articles to its own folder "imported". The "imported" folder is ignored when I do "make html" with dev settings, but if I do "make deploy" all the "imported" content is generated too.

  • I had to create a script that deployed my new articles to the production server, but when the script was created all subsequent deployments is hazzlefree.

  • Another reason to switch was to get rid of all the pain related to upgrading.

I have removed php and suphp from the server. I dont need any database connection, and the only components I have to update is the OS and the Webserver.

Im happy what I did the switch, and if I get the time, I will migrate more of my sites to Pelican.

Tagged as : joomla pelican