Delivering XML Sitemaps with Servant

With a growing amount of articles on my site - which is running on Servant code - I was thinking about adding those articles to list those articles in a XML Sitemap to simplify and speed up indexing in search enginges (i.e. Google, Bing, etc.). Regarding SEO for websites XML sitemaps are important and sometimes underrated.

A sitemap is a XML (eXtensible Markup Language) document which contains links to the content of your website. Furthermore we can add additional information about each link in your sitemap, those are the lastmod and changefreq.

Servant does not support the delivering of XML sitemaps out of the box. But as well there is a lib for achieving this. The package servant-xml in comibnation of the lib Xmlbf gives us a powerful tool to build sitemaps.

Basically the idea is to create some content which is then added to the class ToXml from Xmlbf.

To begin with, let's define AppSitemap and SitemapEntry:

newtype AppSitemap = AppSitemap { entries :: [SitemapEntry] }

data SitemapEntry = ArticleEntry ArticleEntity

Here we created AppSitemamp to be able to collect entries and return this type in a servant API definition (we will see this in a moment).

The SitemapEntry is an ADT which contains (for now) only one Data constructor ArticleEntry which accepts an ArticleEntity. We won't look closer to ArticleEntity, let's just assume it is an entity from database which will contain the data we need to create a sitemap entry.

To add other entries to our sitemap, we need just add another data constructor:

data SitemapEntry = ArticleEntry ArticleEntity
  | TagEntry TagEntity
  | CategoryEntry CategoryEntity
  | StaticEntry Text

We can now add our types to the ToXml class:

newtype AppSitemap = AppSitemap { entries :: [SitemapEntry] }

data SitemapEntry = ArticleEntry ArticleEntity

instance ToXml AppSitemap where
  toXml (AppSitemap {..}) = element "urlset" [] (map toXml entries)

instance ToXml SitemapEntry where
  toXml (ArticleEntry article) = 
    element "url" [] (concat [loc, changefreq])
    where
      loc = element "loc" [] (generateUrl article)
      changefreq = element "changefreq" [] "monthly" 

The implementation is pretty straight forward. We pattern match and use basic element function from Xmlbf to create nodes in a XML structure and glue them together.

Finally we can create an API in the typical Servant way:

module Ersocon.HTTP.Web.Sitemap where

import ClassyPrelude
import Ersocon.Capability.Resource.ArticleCapability
import Ersocon.HTTP.Sitemap (AppSitemap (..), SitemapEntry (..))
import Ersocon.Server.Definition
import Servant
import Servant.XML

--------------------------------------------------------------------------------

-- | Sitemap
type SitemapAPI = "sitemap.xml" :> Get '[XML] AppSitemap

sitemapAPI :: Proxy SitemapAPI
sitemapAPI = Proxy

--------------------------------------------------------------------------------

-- | The server that runs the SitenmapAPI
sitemapServer :: (MonadIO m, ArticleResourceM m) => ServerT SitemapAPI (AppT m)
sitemapServer = sitemapHandler

--------------------------------------------------------------------------------

-- | Business logic of the sitemap (fetch data, etc...)
sitemapHandler :: (MonadIO m, ArticleResourceM m) => AppT m AppSitemap
sitemapHandler = do
  (articles, total) <- fetchPublicArticles
  pure $ AppSitemap {entries = map ArticleEntry articles}

This is almost the easiest part. We can now use our prepared data types to construct a sitemap from our entities. We set the return type of our "sitemap.xml" route to AppSitemap with the content type XML. The lib servant-xml will take care of this and construct a proper XML response for us!