Publishing the site

1 Overview

This site used to be built with Hakyll; we would generate the site, then commit all of the generated HTML and other static artifacts to a separate Git repo, and then host that on GitHub Pages.

However, we've since moved to SourceHut for hosting. SourceHut's model is slightly different because it just hosts a tarball of content, which is uploaded to their server. So the key difference is that the generated site is not a Git repo.

Another difference is that we use Lilac to generate the HTML. It is straightforward enough to use Lilac to write new content, but the Hakyll-era content still needs to be included in the tarball because there are old URLs out there that still reference it. See Old content.

2 Lilac configuration

🔗

🎯 .lilac.json

{
    "name": "Linus Arver's Website",
    "homepage": "https://listx.srht.site",
    "repo": "https://git.sr.ht/~listx/personal-website",
    "version": "",
    "nonTangledPathspecs": [
        ":(exclude,top).gitmodules",
        ":(exclude,top)*.org",
        ":(exclude,top)LICENSE",
        ":(exclude,top)art.md",
        ":(exclude,top)code/",
        ":(exclude,top)codex/",
        ":(exclude,top)favicon*",
        ":(exclude,top)file/",
        ":(exclude,top)img/",
        ":(exclude,top)legacy-generated-repo/",
        ":(exclude,top)post/2013-*.org",
        ":(exclude,top)post/2014-*.org",
        ":(exclude,top)post/2015-*.org",
        ":(exclude,top)post/2016-*.org",
        ":(exclude,top)post/2017-*.org",
        ":(exclude,top)post/2018-*.org",
        ":(exclude,top)post/2019-*.org",
        ":(exclude,top)post/2020-*.org",
        ":(exclude,top)post/2021-*.org",
        ":(exclude,top)post/2023-*.org",
        ":(exclude,top)post/2024-*.org",
        ":(exclude,top)post/2025-*.org",
        ":(exclude,top)rust-js/",
        ":(exclude,top)syscfg/",
        ":(exclude,top)vendor/"
    ]
}

3 Publish locally

🔗

🎯 Makefile

org_files := $(shell find . \
	\( -path "./syscfg" -o -path "./codex" \) -prune -o \
	-name "*.org" -exec grep -l "^:ID: " {} +)

obj_files := $(org_files:%.org=%.lobj)

archive_file := site.larc

all: tangle weave lint
.PHONY: all

$(obj_files) &: $(org_files)
	lilac compile $?
	touch $(obj_files)

$(archive_file): $(obj_files)
	lilac archive $^ --out-file $@

tangle: $(archive_file)
	lilac tangle $< --out-dir .
	touch tangle

weave: $(archive_file)
	lilac weave $< --out-dir ./site --write-css --write-js
	touch weave

lint: $(archive_file)
	lilac lint $<
	touch lint

Makefile:publish

gitconfig:
	git config diff.orderfile .git-orderfile
.PHONY: gitconfig

update:
	niv update nixpkgs --branch nixos-25.11
.PHONY: update

clean:
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard
	rm -rf \
		$(archive_file) \
		$(obj_files) \
		site.tar site.tar.gz \
		site/* \
.PHONY: clean

4 Publish to SourceHut

4.1 Old content

Until we migrate all old posts (pre-2026) to use Lilac, we will have a body of work which we'll need to still publish the old way. We don't want to maintain the Hakyll build dependencies, so instead we just pull in the old statically generated repo, then we overwrite it with any content generated by Lilac. Then we tar it up and deploy to SourceHut. That way the old content won't be broken (all of the old URLs will still work).

4.2 Makefile

🔗

Makefile:publish ⇑

site.tar.gz: $(archive_file) weave vendor/mathjax-modern-font vendor/google-fonts
	rm -f site.tar site.tar.gz
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard
	make -C syscfg/doc weave
	cd codex && git submodule update --init --recursive

	# 1
	fix-mathjax
	fix-jquery
	fix-google-fonts

	# 2
	tar cvf site.tar -C legacy-generated-repo --exclude='**/.*' .

	# 3
	tar rvf site.tar -C site .

	# 4
	find syscfg -type f -not -path '*/.*' \
		\( -name '*.html' \
		-o -path 'syscfg/css/*' \
		-o -path 'syscfg/js/*' \) \
		| tar rvf site.tar -T -

	# 5
	find -L codex -path 'codex/deps/elisp/lilac/deps' -prune -o \
		-type f \
		-not -path '*/.*' \
		\( -name '*.html' -o -name '*.css' -o -name '*.js' -o -name '*.svg' \) \
		| tar rvhf site.tar --hard-dereference --exclude='codex/deps/elisp/lilac/deps' -T -

	# 6
	tar rvf site.tar vendor
	gzip site.tar

	# 7
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard

# 8
vendor/mathjax-modern-font:
	./get-mathjax-fonts.sh

vendor/google-fonts:
	./get-google-fonts.sh

SRHT_TOKEN != cat .srht-pages-token 2>/dev/null || echo ""

publish: site.tar.gz syscfg/doc/weave
	@if [ -z "$(SRHT_TOKEN)" ]; then \
		echo "Error: .srht-pages-token is empty or missing." >&2; \
		exit 1; \
	fi
	@echo "Uploading site to SourceHut Pages..."
	@curl --oauth2-bearer "$(SRHT_TOKEN)" \
		-Fcontent=@site.tar.gz \
		https://pages.sr.ht/publish/funloop.org
	printf \
		'<meta http-equiv="refresh" content="0; url=https://funloop.org/">' > index.html && \
		tar -czf - index.html | \
		@curl --oauth2-bearer "$(SRHT_TOKEN)" \
		-F content=@- \
		https://pages.sr.ht/publish/www.funloop.org
	touch publish

2 bundles up the old legacy content into a tar file, after which we tar up Lilac's woven content into the same tar file.

3 publishes the main site; this comes after 2 just in case we want to overwrite any old pages.

4 publishes the https://git.sr.ht/~listx/syscfg as a syscfg subdirectory.

5 publishes the https://git.sr.ht/~listx/codex repo.

Publishing the site requires a personal access token from SourceHut, which we ignore from our repo (checking it into Git would be a disaster!). The actual command to publish is quite straightforward though, as we just need to invoke a single curl command.

8 cleans up the submodule directories, because fixing the MathJax results in modifying tracked files. This way we reduce the chance of accidentally commiting those changes to the main repo (because Git tracks whether a submodule is "dirty").

4.2.1 Fix MathJax

Because SourceHut does not allow pulling in MathJax over a CDN, we have to modify the HTML files to use a locally vendored version 1 via fix-mathjax, and also include MathJax dependencies in the tarball 6.

🔗

fix-mathjax ↑

find legacy-generated-repo -type f -name "*.html" \
	-exec sed -i 's|https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-AMS-MML_HTMLorMML|/vendor/MathJax/tex-mml-chtml.js|g' {} +
find site syscfg -type f -name "*.html" \
	-exec sed -i 's|/vendor/MathJax/tex-mml-chtml.js|/vendor/MathJax/tex-mml-chtml.js|g' {} +
find codex -path 'codex/deps' -prune -o \
	-type f -name "*.html" \
	-exec sed -i 's|https://cdn.jsdelivr.net/npm/mathjax@4.0.0-beta.4/tex-mml-chtml.js|/vendor/MathJax/tex-mml-chtml.js|g' {} +

# 9
find codex -path 'codex/deps' -prune -o -type f -name "*.html" -exec \
	sed-inject-mathjax-font-codex {} +

find legacy-generated-repo -type f -name "*.html" -exec \
	sed-inject-mathjax-font-legacy {} +

find site syscfg -type f -name "*.html" -exec \
	sed-inject-mathjax-font {} +

Apparently just including tex-mml-chtml.js is not enough, because it is missing fonts. We have to download the fonts from NPM and then include it as well.

🔗

get-mathjax-fonts.sh
🎯 get-mathjax-fonts.sh

set -euo pipefail

FONT_PACKAGE="mathjax-modern-font"
VERSION="4.1.2"

pushd vendor

if [[ -d "${FONT_PACKAGE}" ]]; then
	echo "${FONT_PACKAGE} directory already exists"
	exit
fi

curl -O "https://registry.npmjs.org/@mathjax/${FONT_PACKAGE}/-/${FONT_PACKAGE}-${VERSION}.tgz"
tar -xzvf "${FONT_PACKAGE}-${VERSION}.tgz"
mv package "${FONT_PACKAGE}"
rm "${FONT_PACKAGE}-${VERSION}.tgz"

We execute get-mathjax-fonts.sh at 8 to download the fonts and prepare it for inclusion into the vendor directory. The only thing remaining is to reference it in the HTML files in sed-inject-mathjax-font-codex 9.

Below we modify the HTML files depending on their "flavor". The critical piece in each of the code blocks below is telling MathJax to load the fonts from the local /vendor/mathjax-modern-font directory. If we don't do this, then MathJax still tries to load fonts from a CDN, which will break for SourceHut.

🔗

sed-inject-mathjax-font-codex ↑

sed -i 's|output:[[:space:]]*{|loader: { paths: { "mathjax-modern": "/vendor/mathjax-modern-font" } },\
output: {|'

🔗

sed-inject-mathjax-font-legacy ↑

sed -i '/<script type="text\/x-mathjax-config">/,/<\/script>/c\
<script>\
  window.MathJax = {\
	  loader: { paths: { "mathjax-modern": "/vendor/mathjax-modern-font" } },
    tex: {\
      ams: {\
        multlineWidth: '\''85%'\''\
      },\
      tags: '\''ams'\'',\
      tagSide: '\''right'\'',\
      tagIndent: '\''.8em'\''\
    },\
    chtml: {\
      scale: 1.0,\
      displayAlign: '\''center'\'',\
      displayIndent: '\''0em'\''\
    },\
    svg: {\
      scale: 1.0,\
      displayAlign: '\''center'\'',\
      displayIndent: '\''0em'\''\
    },\
    loader: { paths: { "mathjax-modern": "/vendor/mathjax-modern-font" } },\
    output: {\
      font: '\''mathjax-modern'\'',\
      displayOverflow: '\''overflow'\''\
    }\
  };\
</script>'

🔗

sed-inject-mathjax-font ↑

sed -i '/window\.MathJax[[:space:]]*=[[:space:]]*{/a\
  loader: { paths: { "mathjax-modern": "/vendor/mathjax-modern-font" } },'

4.2.2 Fix jQuery

Some of the pages pull in jQuery from upstream; we have to host these locally as well. They are vendored inside vendor, but we need to make some more sed replacements to make the relevant pages refer to the vendored version.

🔗

fix-jquery ↑

find codex -path 'codex/deps' -prune -o -type f -name "*.html" -exec \
	sed-inject-local-jquery-codex {} +
find legacy-generated-repo -type f -name "*.html" -exec \
	sed-inject-local-jquery-legacy {} +

🔗

sed-inject-local-jquery-codex ↑

sed -i 's|src="https://code.jquery.com/jquery-3.6.4.min.js"|src="/vendor/jquery-3.6.4.min.js"|g'

🔗

sed-inject-local-jquery-legacy ↑

sed -i 's|src="https://code.jquery.com/jquery-2.1.4.min.js"|src="/vendor/jquery-2.1.4.min.js"|g'

4.2.3 Fix Google fonts

We have to host the fonts directly as well, instead of pulling them over the network from Google web fonts.

The first thing to do is to download the fonts locally.

🔗

get-google-fonts.sh
🎯 get-google-fonts.sh

set -euo pipefail

if [[ -d vendor/google-fonts ]]; then
	echo "vendor/google-fonts directory already exists"
	exit
fi

mkdir -p vendor/google-fonts

curl -fsSL -A 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' \
'https://fonts.googleapis.com/css2?display=swap&family=Source+Code+Pro:ital,wght@0,400;0,700;1,400;1,700&family=Source+Sans+3:ital,wght@0,400;0,700;1,400;1,700&family=Source+Serif+4:ital,wght@0,400;0,700;1,400;1,700' \
-o vendor/google-fonts/google-fonts.css

# Filter specifically for links ending in .woff2
grep -o 'https://fonts.gstatic.com[^)]*\.woff2' \
	vendor/google-fonts/google-fonts.css |
	sort -u |
	while read -r url; do
		curl -fsSL "$url" -o "vendor/google-fonts/$(basename "$url")"
	done

sed -i -E \
's|https://fonts.gstatic.com/[^)]*/([^/)]+\.woff2)|/vendor/google-fonts/\1|g' \
vendor/google-fonts/google-fonts.css

Now make the HTML content refer to the local fonts. 10 makes the old blog posts use "Source Serif 4", not "Source Serif Pro" (because the former is what we download).

🔗

fix-google-fonts ↑

find site syscfg codex legacy-generated-repo -path 'codex/deps' -prune -o \
	-type f -name "*.html" -exec \
	sed-use-local-google-fonts {} +

# 10
sed -i 's|Source Serif Pro|Source Serif 4|g' legacy-generated-repo/css/base.css

🔗

sed-use-local-google-fonts ↑

sed -i \
's|href="https://fonts\.googleapis\.com[^"]*"|href="/vendor/google-fonts/google-fonts.css"|g'

5 Hakyll setup

Below is a record of how the old content used to be built with Hakyll. This is here mainly for historical interest. It will be removed after we've fully transitioned away to the new publishing step with Lilac.

🔗

Makefile:Hakyll

# This Makefile is expected to be run inside a nix-shell.

PROJ_ROOT := $(shell git rev-parse --show-toplevel)
BLOG_ADDR?=localhost
BLOG_PORT?=8020

all: sync
.PHONY: all

# Check for broken links.
check:
	cabal run -- blog check
.PHONY: check

gen-css:
	cabal build -- base >/dev/null
	cabal exec -- base
.PHONY: gen-css

# JavaScript generated from Rust.
gen-js:
	rustup default stable
	make -C rust-js build
.PHONY: gen-js

sync: build-site
	./sync.sh
.PHONY: sync

cabal-update:
	cabal update
.PHONY: cabal-update

build-binaries:
	cabal build
.PHONY: build-binaries

build-site: build-binaries
	cabal run -- blog rebuild
.PHONY: build-site

watch: build-binaries
	cabal run -- blog watch --host $(BLOG_ADDR) --port $(BLOG_PORT)
.PHONY: watch

nixpkgs_stable_channel := nixos-24.05
update-deps: nix/sources.json nix/sources.nix
	niv update nixpkgs --branch $(nixpkgs_stable_channel)
	niv update
	touch update-deps

The local listx.github.io folder used to have the Github repo which held the generated content. Hakyll would by default publish to _site, so the script below would sync these two with rsync.

🔗

dest='listx.github.io/'
targ='_site/'

if [[ ! -d $targ ]]; then
    echo "\`$targ' directory missing"
    exit 1
elif [[ ! -d $dest ]]; then
    echo "\`$dest' directory missing"
    exit 1
fi

rsync -ahP --no-whole-file --inplace --delete --exclude=.git\* $targ $dest

6 Miscellaneous

🔗

🎯 .gitignore

*.hi
*.html
*.o
*.lobj
*.larc
*.tar
*.tar.gz
**/__pycache__/
code/2017-04-02-calling-c-from-haskell/hs/ffi
code/2021-03-15-bresenham-circle-drawing-algorithm/golden.yaml
code/toy/binary-search-c
code/toy/binary-search-hs
.direnv
site
.srht-pages-token
update-deps
tangle
weave
lint
publish
vendor/

🔗

.gitattributes
🎯 .gitattributes

vendor/*.js binary

Page metrics