Migrating from Drupal to Hugo

TL;DR: Migrating my website from Drupal 7 to Hugo

Jump directly to the end titled Migration to Hugo

Initial website

Looking back at my website’s history, the domain was first registered sometime in 2003. Back then, it was mostly a couple of html pages. Being (and still) a novice in web, my website was mostly on ideas from others. IIRC, for the bare html one, I took a lot of look wise details from Miss Garrels’ website.

First blog

My initial blog was self-hosted with a blogging software in PHP, named PivotX The website for it still works, so hopefully the project is still alive. It was pretty good a tool for the purpose. Very lean and had support for data backends in both, MySQL and flat files. The latter was important to me as I wanted to keep it simple.

Drupal

My first interaction with Drupal was with its WSOD. That was it until I revisited it when evaluating different FOSS web tools to build a community site for one of my previous employer.

Back then, we tried multiple tools: Jive, Joomla, Wordpress and many more. But finally, resorted to Drupal. What the requirement was was to have something which would filter content under nested categories. Then, of the many things tried, the only one which seemed to be able to do it was Drupal with its Taxonomy feature, along with a couple of community driven add-on modules.

We built it but there were other challenges. It was hard to find people who were good with Drupal. I remember to have interviewed around 10-15 people, who could take over the web portal and maintain it, and still not able to fill the position. Eventually, I ended up maintaining the portal by myself.

Migrating my website to Drupal

The easiest way to deal with the maintenance was to have one more live portal running Drupal. My website, which back then, had ambitious goals to also serve an online shopping cart, was the perfect candidate. So I migrated my website from PivotX to Drupal 6. Drupal had a nice RSS Import module which was able to pull in most of the content, except the comments on each article. I think that is more a limitation of RSS Feeds. But the only data import path I could find back then was to import content through RSS Feeds.

Initially, Drupal looked like a nice tool. Lots of features and a vibrant community made it very appealing. And I always desired to build some skills Hands-On (that’s how the job market likes it; irrespective of the skills, it is the hands-on that they evaluate) by using Drupal both, at the employer’s community portal and my personal website.

Little did I know that running/maintaining a website is one aspect; where as extending it, is another (mostly expensive) affair.

Drupal 7

That was the first blow. For a project serving as a platform, Drupal was a PITA when dealing with migrations. And it is not about migrations to a different platform. Rather an upgrade from one major release to another.

Having been using Debian for quite some time, this approach from Drupal brought back memories from the past, of when using Red Hat Linux and SuSE Linux distribution; where upgrades were not a common term, and every major release of the distribution people were mostly recommended to re-install.

Similar was the case with Drupal. Every major release, many (core) modules would be dropped. Many add-on modules would lose support. Neither the project nor the community around it, was helpful anymore.

But somehow, I eventually upgraded to Drupal 7. I did lose a lot of functionality. My nested taxonomy was gone and my themes were all broken. For the web novice that I am, it took me some time to fix those issues.

But the tipping point came in with Drupal 8. It took the pain to the next level repeating the same process of dropping modules and breaking functionalities; never did I hear much of backward compatibility on this platform.

Hugo

For quite some time I kept looking for a migration path away from Drupal 7. I did not care what it was as long as it was FOSS, and had an active community around it. The immediate first choice was WordPress. By this time, my web requirements had trimmed down. No more did I have outrageous ideas of building all solutions (Web, Blog, Cart) in a single platform. All I did was mostly blog and had a couple of basic pages.

The biggest problem was migration. WP has a module, that does migration. But, for whatever annoying reason, the free version of it would only pick 7 articles from the total. And it did not import comments. So the annoyance and my limitations with web technologies was still prone to with WP. This migration path did not enthuse me much: it was more like a Hindi idiom: आसमान से गिरे और खजूर में अटके

I also attempted Jekyll and Hugo. My limited initial attempts were disappointing. Jekyll had an import module, which IIRC did not work proper. Similar was the case with Hugo, which has a module listed on its migration page, drupal2hugo, which sets a disappointment in the beginning itself.

With nothing much left, I just kept postponing my (desperate) plans to migrate.

Migration to Hugo

Luckily, I was able to find some kind soul share migration scripts to help migrate from Drupal 7 to Hugo. Not everything could be migrated (I had to let go of comments) but not much was I in a position to wait more.

With very minimal changes to adapt it to my particular setup, I was able to migrate most of my content. Now, my website is running on markdown generated with Hugo. More than the tool, I am happy to have the data available in a much standard format.

If there’s one thing that I’m missing on my website, it is mostly the commenting system. I would love to have a simple way to accept user comments integrated into Hugo itself, which would just append those comments to their respective posts. Hopefully soon, when I have (some more) free time.

<?php
define('DRUPAL_ROOT', __DIR__);
include_once(DRUPAL_ROOT . '/includes/bootstrap.inc');
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
$nids = db_query('SELECT DISTINCT(nid) FROM {node}')
  ->fetchCol();
$nodes = node_load_multiple($nids);
foreach($nodes as $node) {
  $front_matter = array(
    'title' => $node->title,
    'date' => date('c', $node->created),
    'lastmod' => date('c', $node->changed),
    'draft' => 'false',
  );
  if (count($node->taxonomy_vocabulary_2[LANGUAGE_NONE])) {
    $tags = taxonomy_term_load_multiple(
      array_column(
        $node->taxonomy_vocabulary_2[LANGUAGE_NONE],
        'tid'
      )
    );
    $front_matter['tags'] = array_column($tags, 'name');
  }
  if (count($node->taxonomy_vocabulary_1[LANGUAGE_NONE])) {
    $cat = taxonomy_term_load_multiple(
      array_column(
        $node->taxonomy_vocabulary_1[LANGUAGE_NONE],
        'tid'
      )
    );
    $front_matter['categories'] = array_column($cat, 'name');
  }
  $path = drupal_get_path_alias('node/'.$node->nid);
  if ($path != 'node/'.$node->nid) {
    $front_matter['url'] = '/'.$path;
    $content_dir = explode('/', $path);
    $content_dir = end($content_dir);
  }
  else {
    $content_dir = $node->nid;
  }
  $content = json_encode(
    $front_matter,
    JSON_PRETTY_PRINT|JSON_UNESCAPED_SLASHES|JSON_UNESCAPED_UNICODE
  );
  $content .= "\n\n";
  $tmp_file = '/tmp/node.html';
  file_put_contents($tmp_file, $node->body['fr'][0]['value']);
  $body = shell_exec('html2markdown '.$tmp_file);
  unlink($tmp_file);
  //$body = $node->body['fr'][0]['value'];
  $content .= $body;
  $dir_name = '/tmp/hugo/content/'.$node->type.'/'.$content_dir;
  mkdir($dir_name, 0777, true);
  file_put_contents($dir_name.'/index.md', $content);
}

See also