1 Hakyll :heart: Pandoc

Since I don’t write Haskell code professionally anymore it takes me longer to get into the right rhythm. This post is intended for my future self more or less and should serve as a cheatsheet for Hakyll development. I’ve already written a high level overview of how to edit content and build the website in this README. Here I want to go deeper into how to construct new compilers and how to apply them in a context to templates.

Under the hood Hakyll integrates natively with Pandoc – the swiss-army knife of file converters. Pandoc is also written in Haskell and can convert files between a wide variety of file formats and can be extended with custom Lua filters. All this has some configuration complexities associated with it and below I also discuss the various configurations and extensions used to enable the auto-generation of table of contents and \(\LaTeX\) math support on this website.

2 Website rules

2.1 Matching clause, routes and compilers

Writing rules in the DSL¹ is simple enough. A rule needs three things to be valid; a matching clause for the source file, a route for the compiled output and the compiler itself which is responsible for the transformation between the input and the output.

We match all files with the .css extension in the css directory and its subdirectories (had we used * only files in the css directory would match). The idRoute preserves the filename as is but other routes may change the e.g. the extension or something more complex. For example we want to match files with .scss extension, compile them and pipe the compressed results to a file with the .css extension for browser compatibility.

In some cases instead of a source file to match we just create an output directly. This is the case with index.html which is dynamically generated from from the latest blog posts.

2.2 Templates and their contexts

Optionally the compiler may load a template to the input and interpolate variables from a context. We do so by chaining multiple compilers together into a more feature rich compiler. The output of a previous compiler is interpolated into the $body$ variable of the template. Other variables are made available to the template by defining them in the context. Before templates can be loaded they need to be compiled using the templateCompiler.

Contexts are monoids and can therefore be easily combined to create bigger contexts where the earlier definition of a non-empty (mempty) field takes precedence.

pageTitleField :: String -> Context String
pageTitleField key = 
   aliasContext alias metadataField <> -- use page title from metadata
   pathTitleField key               <> -- or read from the path
   constField key "Crypto and Code"    -- alternatively use this
   where
      alias x | x == key = "title"
      alias x            = x

A special case of the string context is the function context which defines a variable which value is a function which can be called in the template.

2.3 Metadata, flow control and list iteration

Metadata can be placed in the front matter of the markdown formatted as YAML. The metadata can be made available both to the compiler and to the template in case the metadataField :: Context a is applied to the template. Tags can eiter be comma separated or as a valid YAML list.

---
title: This is the blog title
tags: tag1, tag2
tags: 
   - tag1
   - tag2
summary: |
  Introduction to the blog content
---

This is the blog body

Special function variables are used to define conditional branches to display and iterate over items. The variable names used within the $for(blogs)$ loop are defined in the blog list field. Everything between $sep$ and $endfor$ will be used as a separator between items, i.e. it is not included for the last element.

3 Advanced features

Using the pandocCompilerWith some options can be passed in which affect the behavior of Pandoc. In addition extensions can be enabled which unlock additional features.

pandocCompilerWith :: ReaderOptions -> WriterOptions -> Compiler (Item String)

blogCompiler :: Compiler (Item String)
blogCompiler = do
   ident <- getUnderlying
   toc   <- getMetadataField ident "withtoc"
   pandocCompilerWith blogReaderOptions (maybe defaultOptions blogOptions toc)
   where
      defaultOptions = defaultHakyllWriterOptions
      blogOptions = const blogWriterOptions

3.1 Auto-generated table of contents

3.2 \(\KaTeX\ \) to render \(\LaTeX\) math

Pandoc can transform math to MathJax when configured correctly. KaTeX is able to render the output by embedding the javascript and CSS.

<link rel="stylesheet" href="/katex/katex.min.css">
<script defer src="/katex/katex.min.js"></script>
<script type="text/javascript" script defer src="/katex/auto-render.min.js" 
  onload="renderMathInElement(document.body);"></script>

Equations can be place on their own lines.

$$ \ln x = \int_{-\infty}^x \frac 1 y \, dy $$

or

\[ \ln x = \int_{-\infty}^x \frac 1 y \, dy \]

becomes

\[ \ln x = \int_{-\infty}^x \frac 1 y \, dy \]

Writing $x \equiv a \pmod{b}$ or \( x \equiv a \pmod{b} \) prdouces \(x \equiv a \pmod{b}\).

3.3 Decks and presentations

Gooogle decks can easily be embedded using an HTML snippet and requires no special handling on Hakyll’s side. I wrap the standard <iframe> tag in a div tag with the a special class which makes the presentation responsive.

<div class="responsive">
<iframe src="https://docs.google.com/presentation/d/e/<ID>/embed
  ?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" 
  allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>
</div>

.responsive {
  overflow: hidden;
  padding-bottom:56.25%;
  position: relative;
  height: 0;
  iframe {
    left: 0;
    top: 0;
    height: 100%;
    width: 100%;
    position: absolute;
  }
}

reveal.js – the HTML presentation framework – uses standard HTML tags to define the presentation and supports themes. Enabling reveal.js is achieved by modifying the deck’s base template and adding a few options to the decks metadata.

---
reveal: true
theme: league
---

3.4 Paginating posts

Blog posts are paginated by constructing a Paginate object with buildPaginateWith and then creating rules for every page.

type PageNumber = Int

data Paginate = Paginate
   { paginateMap        :: M.Map PageNumber [Identifier] -- used in blogCtx
   , paginateMakeId     :: PageNumber -> Identifier 
   , paginateDependency :: Dependency
   }

buildPaginateWith
    :: MonadMetadata m
    => ([Identifier] -> m [[Identifier]])  -- group items into pages
    -> Pattern                             -- items to paginate
    -> (PageNumber -> Identifier)          -- identifiers for the pages
    -> m Paginate

To access pages pagination links are added to the relevant context. paginateContext returns a default paginate context which provides standard pagination fields but I like to alias those fields to give them a bit more friendly names.

3.5 Support for tags and categories

Tags are supported in a similar manner to pagination. The Tags object is constructed with either buildTags or buildTagsWith. Hakyll provides functions to sort tags, render them and adding to a context. Rules for each tag must be created using tagsRules in order for tags to work.

data Tags = Tags
   { tagsMap        :: [(String, [Identifier])]
   , tagsMakeId     :: String -> Identifier
   , tagsDependency :: Dependency
   }

-- buildTags takes a pattern for loading resources and 
-- a mapping from the tag name to its identifier
buildTags :: MonadMetadata m => Pattern -> (String -> Identifier) -> m Tags

-- getTags returns the tag field from the metadata as a list of strings
-- the field value can either be tags separated by a comma or a valid YAML list
getTags :: MonadMetadata m => Identifier -> m [String]

Tags can be rendered in a standard way using renderTags, as a tag cloud using renderTagCloud or in a custom way using tagsFieldWith.

4 Rolling your own

4.1 Collecting rules in the Rules monad

The compiler rules live inside the Rules monad which is an instance of the MonadMetadata type class which enables monads to retrive the Metadata of the source file identified by the Identifier. Metadata is a wrapper around a YAML object with functions to lookup it’s values as either strings or lists of strings.

data Rules a

instance Monad Rules
instance Functor Rules
instance Applicative Rules
instance MonadMetadata

class Monad m => MonadMetadata m where
   getMetadata    :: Identifier -> m Metadata
   getMatches     :: Pattern -> m [Identifier]

-- getAllMetadata returns all metadata associated with a pattern
getAllMetadata :: MonadMetadata m => Pattern -> m [(Identifier, Metadata)]

-- getItemUTC tries to extract and parse the time from the published field 
-- or from the filename. 
getItemUTC :: MonadMetadata m	=> TimeLocale	-> Identifier	-> m UTCTime	

-- match matches a pattern and adds a rule for it.
match :: Pattern -> Rules () -> Rules ()

-- hakyll runs the rules in the IO monad
hakyll :: Rules a -> IO ()

4.2 Identifiers and patterns

data Item a = Item
   { itemIdentifier :: Identifier
   , itemBody       :: a
   } deriving (Show, Typeable)

instance Item
instance Item
instance Item
instance Show a => Show (Item a)
instance Binary a => Binary (Item a)

4.3 Writing Compilers and loading Items

The other monad to implement the MonadMetadata type class is the Compiler monad which as the name implies compiles matched source files and takes care of dependencies between rules such that if one rule relies on the results of another rule they are executed in the correct order. The Compiler monad is an instance of Alternative which makes it easy to combine potentially failing compilers (using empty, fail, or throwError) compilers which are tried in sequence until one of them succeeds or else the combined compiler fails.

Compiler a values have access to a range of functions which can retrieve the identifier, the source and path of both the matched file and any file identified by an Identifier. Keep in mind that for such function to succeed there needs to exist a rule for that Identifier in the Rules monad. Otherwise Hakyll doesn’t have any knowledge of its existence.

data Compiler a -- a is the the type of the output - usually String

instance Monad Compiler
instance Functor Compiler
instance Applicative Compiler
instance Alternative Compiler
instance MonadMetadata Compiler
instance MonadError Compiler

-- compile adds a compiler to the Rules monad. Note that the Compiler value is Item a
compile :: (Binary a, Typeable a, Writable a) => Compiler (Item a) -> Rules ()

-- makeItem lifts a value into the the Compiler monad.
makeItem :: a -> Compiler (Item a)

-- getRoute returns the route for a specified item
getRoute :: Identifier -> Compiler (Maybe FilePath)

Parser to try to extract and parse the time from the published field or from the filename. See dateField for more information. Exported for user convenience.

-- getResourceBody returns the full contents of the matched source file
-- as a string without metadata preamble.
getResourceBody :: Compiler (Item String)

-- load an item compiled elsewhere.
load :: (Binary a, Typeable a) => Identifier -> Compiler (Item a)

-- loadAll loads a dynamic list of items.
loadAll :: (Binary a, Typeable a) => Pattern -> Compiler [Item a]

4.4 Snapshots to define a pipeline stage

When loading a compiled resource it usually has the base layout templates applied to it making it unusable in dynamic lists. Snapshots are the solution to that problem. During the compiler construction a snapshot of the resource can be saved in middle of the pipeline allowing you to later load that stage of the compiled resource.

4.5 Working with contexts and templates

Contexts define variables which can be accessed and interpolated in a template. There are three ways to define a Context.

1. Use the Context data constructor.
2. Use field functions, e.g. field, constField or listField.
3. Derive a new context from another context using mapContext.

The Context data constructor is rarely used but necessary if the the context needs to be able to return mempty which is implimented as a failing Compiler. In the Compiler monad failure can be signalled with empty, noResult or throwError.