Publishing Websites with Emacs and Org Mode: Publishing Functions

This first step gathers all project variables and stores them in a list. If the requested project is a meta-project defined with :components (“component1” “component2”), then org-publish processes them sequentially.

The :preparation-function parameter is a list of one or more functions that run before publishing commences. The output of these functions is ignored in the further process, unless they change or create files in the base directory. These functions can, for example, create category pages, check that every file has a title, date, and keywords, or check that all images have alt tags.

Preparation functions take the project properties as their single argument, as defined in org-publish-project-alist. You can declare custom variables for use in a bespoke publishing function, for example :base-url https://peterprevos.com.

This demo function prints the name of each file that will be published.

(defun my-prep-loop-source-files (project-plist)
  “Demo preparation function using PLIST project properties.”
  (let ((files (org-publish-get-base-files
                (cons “temp-project” project-plist)))
        (base-url (plist-get project-plist :base-url)))
    (dolist (file files)
      (message “Will publish source file: %s for %s” file base-url))))

Emacs stores a project-specific timestamp cache on disk. The cache prevents redundant compilation by storing the source file modification timestamps, cross-references, and output mappings. Org stores the cache in the directory mentioned in org-publish-timestamp-directory.

You can force Emacs to publish all files and ignore the cache with the universal argument (C-u M-x org-publish-all) or (org-publish-all t), or for a single project use org-publish-project.

A sitemap lists all essential pages on your site in an organised structure. There are two main types of sitemaps: XML sitemaps to guide search crawlers through your site, and HTML sitemaps for the convenience of your visitors. Org has no built-in functionality to generate the search-engine standard sitemap.xml protocol, but it can generate Org versions and export these to the desired format out of the box.

When using :auto-sitemap t, Org scans the project files and generates a sitemap document (defaulting to sitemap.org). The following configurations are available:

• :sitemap-filename: The name of the sitemap file
• :sitemap-title: Title of the sitemap page
• :sitemap-style: Either a list or a tree
• :sitemap-sort-files: Either alphabetically, chronologically (old to new), or anti-chronologically (new to old) or nil.
• :sitemap-sort-folders: Either first, last, ignore (no folders) or nil.
• :sitemap-ignore-case: Ignore case when sorting the sitemap.

When :auto-sitemap is t, two functions control entry generation and document structure, both of which you can replace with your own versions:

• :sitemap-format-entry: Formats the links inside the sitemap
• :sitemap-function: Assembles the sitemap document from the list of formatted entries.

In the last preparation step, Org scans the :base-directory for files matching :base-extension (by default “org”, or a regular expression matching file extensions).

Org mode includes the content of subdirectories when using :recursive t. The resulting matches are pruned using :exclude and extended using :include.

The previous article described how to read a template file to include a style sheet and favicon. But the head is missing some critical content, such as a canonical link, Open Graph, and Structured Data. Open Graph controls how social media presents links when somebody shares them. Structured Data gives search engines and generative AI crawlers semantic meaning to parse concepts, entities, and relationships. This content is dynamic and varies by file, so we need to use a function instead of a template.

Functions that generate head content take a single argument: a property list containing the publication project properties and the keywords from the exported file. The example function below extracts metadata from the published file and constructs partial Open Graph metadata.

You can go beyond the standard keywords and add special ones such as #+IMAGE: to provide the URL for a preview image. To do this, you need to instruct Org to include it in the export options by appending the org-export-options-alist.

(with-eval-after-load 'ox
  (add-to-list 'org-export-options-alist
               '(:image "IMAGE" nil nil t)))

(defun my-org-html-generate-open-graph (info)
  "Generate standard Open Graph tags using the INFO plist."
  (let* ((title (org-export-data (plist-get info :title) info))
         (description (plist-get info :description))
         (image (plist-get info :image))
         (concat
          (format "<meta property=\"og:title\" content=\"%s\" />\\n"
                  (or title ""))
          (when (org-string-nw-p description)
            (format "<meta property=\"og:description\" content=\"%s\" />\\n" description))
          (when (org-string-nw-p image)
            (format "<meta property=\"og:image\" content=\"%s\" />\\n" image))
          "<meta property=\"og:type\" content=\"article\" />"))))

Add the function name to the :html-head-extra declaration in your project to append it to the previously defined static part.

You can also use this approach to generate a bespoke preamble and postamble for your pages. However, for typical header and footer content, the provided content markers discussed in the previous article suffice.

When Org mode exports a document, it parses the file into an Abstract Syntax Tree (AST) using the org-element library. In the AST, every heading, paragraph, list item, link and drawer becomes a nested Lisp data structure which can be manipulated during the process.

The functions listed in :filter-parse-tree modify the AST for each file during the export process. Each filter is called with three arguments: the parse tree, as returned by org-element-parse-buffer, the export backend ('html in our case), and the project property list. Each filter returns the modified parse tree to transcode. If the filter returns nil, the process will crash!

Because the filter operates at the AST level before rendering, it allows you to modify or add content based on context dynamically. For this website, I use the AST filter to construct a subtitle based on author, authoring date, word count, and estimated reading time.

Org publish has many more such filtering functions in its arsenal to finetune the output, basically one for each syntax type. For example, this website uses the :filter-link option to remove the filenames from the URLs so that article.html becomes artcile/index.html.

The :filter-final-output option is a last opportunity to modify the main content. This filter provides the last hook in the Org mode export pipeline before the transcoded document string is committed to disk. While the syntax tree filter operates on the AST of Lisp elements, the final output filter operates on the complete, fully-rendered document string.

Every function listed in org-publish-after-publishing-hook in this hook will be called for each published file with two arguments: the name of the original file and the name of the exported file.

Org export calls a publishing function for each component to export files to the target format, such as org-html-publish-to-html for a website. Other backends extend the repertoire of output formats. When, for example, using org-gemini-publish-to-gemini the Org files create a Gemini capsule.

Publishing functions take three parameters:

• plist: property list for the given project (the export variables in org-publish-project-alist)
• filename: name of the source file
• pub-dir: directory where to store the resulting file

Besides a sitemap, you can also build an index for a website, just like in a book. Org export can generate an index page based on the #+INDEX: keywords in a file. Org’s indexing engine treats entries as hierarchical trees separated by exclamation marks: #+INDEX: TopLevel!SubEntry!SpecificItem.

Org creates a hierarchical index using these items. The final part becomes a hyperlink to the source. To make a subtree appear under more than one index, add more index lines in the front matter.

Index items relate to the whole file by placing them in the front matter, or they link directly to headings by adding the index keyword inside a subtree. You can sprinkle index lines throughout your writing, and Org will create an index page linking to the listed topics.

During the publishing process, Org creates the index.inc file, which contains a nested list of Org links to the relevant pages and headers. After the export process has completed, Org translates these to HTML links by referring to the section IDs in the exported file.

The process uses the theindex.org file, which is created during the export process if it does not yet exist. This file only contains a title and #+INCLUDE: “theindex.inc”. You can edit this file once it is generated to change the title or add any other additional content.

The index is a powerful alternative to category pages or sitemaps that adds heuristic value to a website. Unfortunately, Org mode’s indexation function offers no hooks for customisation beyond editing the Org file.

The :completion-function takes a list of one or more functions that run after publishing the project. While the preparation functions typically act on the source files, the completion functions act on the exported files. For this website, I use a completion function to generate the XML sitemap to guide crawlers.

When Org mode has finished processing all files, the updated timestamp cache is committed to disk for future reference, and the project closes.