Linus Arver's Website / build.org

Publishing the site

1 Overview

This site used to be built with Hakyll; we would generate the site, then commit all of the generated HTML and other static artifacts to a separate Git repo, and then host that on GitHub Pages.

However, we've since moved to SourceHut for hosting. SourceHut's model is slightly different because it just hosts a tarball of content, which is uploaded to their server. So the key difference is that the generated site is not a Git repo.

Another difference is that we use Lilac to generate the HTML. It is straightforward enough to use Lilac to write new content, but the Hakyll-era content still needs to be included in the tarball because there are old URLs out there that still reference it. See Old content.

2 Lilac configuration

🔗
🎯 Lilac.toml
name = "Linus Arver's Website"
homepage = "https://funloop.org"
repo = "https://git.sr.ht/~listx/personal-website"
version = ""
nonTangledPathspecs = [
   ":(exclude,top).gitmodules",
   ":(exclude,top)*.org",
   ":(exclude,top)LICENSE",
   ":(exclude,top)art.md",
   ":(exclude,top)code/",
   ":(exclude,top)codex/",
   ":(exclude,top)favicon*",
   ":(exclude,top)file/",
   ":(exclude,top)img/",
   ":(exclude,top)legacy-generated-repo/",
   ":(exclude,top)post/2013-*.org",
   ":(exclude,top)post/2014-*.org",
   ":(exclude,top)post/2015-*.org",
   ":(exclude,top)post/2016-*.org",
   ":(exclude,top)post/2017-*.org",
   ":(exclude,top)post/2018-*.org",
   ":(exclude,top)post/2019-*.org",
   ":(exclude,top)post/2020-*.org",
   ":(exclude,top)post/2021-*.org",
   ":(exclude,top)post/2023-*.org",
   ":(exclude,top)post/2024-*.org",
   ":(exclude,top)post/2025-*.org",
   ":(exclude,top)rust-js/",
   ":(exclude,top)vendor/"
]

custom-html-head

3 Publish locally

🔗
🎯 Makefile
org_files := $(shell find . \
	\( -path "./codex" \) -prune -o \
	-name "*.org" -exec grep -l "^:ID: " {} +)

obj_files := $(org_files:%.org=%.lobj)

archive_file := site.larc

all: tangle weave lint
.PHONY: all

$(obj_files) &: $(org_files)
	lilac compile $?
	touch $(obj_files)

$(archive_file): $(obj_files)
	lilac archive $^ --out-file $@

tangle: $(archive_file)
	lilac tangle $< --out-dir .
	touch tangle

weave: $(archive_file)
	lilac weave $< --out-dir ./site --write-css --write-js
	touch weave

lint: $(archive_file)
	lilac lint $<
	touch lint

Makefile:publish

gitconfig:
	git config diff.orderfile .git-orderfile
.PHONY: gitconfig

update:
	niv update nixpkgs --branch nixos-25.11
.PHONY: update

clean:
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard
	rm -rf \
		$(archive_file) \
		$(obj_files) \
		site.tar site.tar.gz \
		site/* \
.PHONY: clean

4 Publish to SourceHut

4.1 Old content

Until we migrate all old posts (pre-2026) to use Lilac, we will have a body of work which we'll need to still publish the old way. We don't want to maintain the Hakyll build dependencies, so instead we just pull in the old statically generated repo, then we overwrite it with any content generated by Lilac. Then we tar it up and deploy to SourceHut. That way the old content won't be broken (all of the old URLs will still work).

4.2 Makefile

🔗
Makefile:publish ⇑
site.tar.gz: $(archive_file) weave vendor/mathjax vendor/mathjax-font-font vendor/source-fonts
	rm -f site.tar site.tar.gz
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard
	cd codex && git submodule update --init --recursive

	# 1
	fix-mathjax
	fix-jquery
	fix-google-fonts

	# 2
	tar cvf site.tar -C legacy-generated-repo --exclude='**/.*' .

	# 3
	tar rvf site.tar -C site .

	# 4
	find -L codex -path 'codex/deps/elisp/lilac/deps' -prune -o \
		-type f \
		-not -path '*/.*' \
		\( -name '*.html' -o -name '*.css' -o -name '*.js' -o -name '*.svg' \) \
		| tar rvhf site.tar --hard-dereference --exclude='codex/deps/elisp/lilac/deps' -T -

	# 5
	tar rvf site.tar vendor
	gzip site.tar

	# 6
	git -C legacy-generated-repo reset --hard
	git -C codex reset --hard

# 7
vendor/mathjax vendor/mathjax-newcm-font &:
	./vendor-mathjax.sh

vendor/source-fonts:
	./vendor-source-fonts.sh

SRHT_TOKEN != cat .srht-pages-token 2>/dev/null || echo ""

publish: site.tar.gz
	@if [ -z "$(SRHT_TOKEN)" ]; then \
		echo "Error: .srht-pages-token is empty or missing." >&2; \
		exit 1; \
	fi
	@echo "Uploading site to SourceHut Pages..."
	@curl --oauth2-bearer "$(SRHT_TOKEN)" \
		-Fcontent=@site.tar.gz \
		https://pages.sr.ht/publish/funloop.org
	@printf \
		'<meta http-equiv="refresh" content="0; url=https://funloop.org/">' > index.html && \
		tar -czf - index.html | \
		curl --oauth2-bearer "$(SRHT_TOKEN)" \
		-F content=@- \
		https://pages.sr.ht/publish/www.funloop.org
	touch publish

2 bundles up the old legacy content into a tar file, after which we tar up Lilac's woven content into the same tar file.

3 publishes the main site; this comes after 2 just in case we want to overwrite any old pages.

4 publishes the https://git.sr.ht/~listx/codex repo.

Publishing the site requires a personal access token from SourceHut, which we ignore from our repo (checking it into Git would be a disaster!). The actual command to publish is quite straightforward though, as we just need to invoke a single curl command.

6 cleans up the submodule directories, because fixing the MathJax (by modifying their HTML files to refer to our vendored versions) results in modifying tracked files. This way we reduce the chance of accidentally commiting those changes to the main repo (because Git tracks whether a submodule is "dirty").

4.2.1 Fix MathJax

Because SourceHut does not allow pulling in MathJax over a CDN, we have to modify the HTML files to use a locally vendored version 1 via fix-mathjax, and also include MathJax dependencies in the tarball 5.

First we download MathJax using vendor-mathjax.sh

🔗
vendor-mathjax.sh
🎯 vendor-mathjax.sh
set -euo pipefail

MAIN_PACKAGE="mathjax"
FONT_PACKAGE="mathjax-newcm-font"
VERSION="4.1.2"

mkdir -p vendor
pushd vendor

download_tarball()
{
	local package
	local version
	local url_prefix
	local tarball
	package="${1:-}"
	version="${2:-}"
	tarball="${package}-${version}.tgz"
	url_prefix="${3:-}"

	if [[ -d "${package}" ]]; then
		echo "${package} directory already exists"
		return 1
	fi

	curl -OL "${url_prefix}/-/${tarball}"
	tar xzvf "${tarball}"
	mv package "${package}"
	rm "${tarball}"
}

download_tarball \
	"${MAIN_PACKAGE}" \
	"${VERSION}" \
	"https://registry.npmjs.org/mathjax"
download_tarball \
	"${FONT_PACKAGE}" \
	"${VERSION}" \
	"https://registry.npmjs.org/@mathjax/${FONT_PACKAGE}"

Nowe we make the old content refer to this newer version in fix-mathjax.

🔗
fix-mathjax ↑
find legacy-generated-repo -type f -name "*.html" \
	-exec sed -i 's|https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-AMS-MML_HTMLorMML|/vendor/mathjax/tex-mml-chtml.js|g' {} +
find codex -path 'codex/deps' -prune -o \
	-type f -name "*.html" \
	-exec sed -i 's|https://cdn.jsdelivr.net/npm/mathjax@4.0.0-beta.4/tex-mml-chtml.js|/vendor/mathjax/tex-mml-chtml.js|g' {} +

find codex -path 'codex/deps' -prune -o -type f -name "*.html" -exec \
	sed-inject-mathjax-font-codex {} +

find legacy-generated-repo -type f -name "*.html" -exec \
	sed-inject-mathjax-font-legacy {} +

Below we modify the HTML files depending on their "flavor". The critical piece in each of the code blocks below is telling MathJax to load the fonts from the local /vendor/<<mathjax-font>>-font directory. If we don't do this, then MathJax still tries to load fonts from a CDN, which will break for SourceHut.

🔗
mathjax-font ↑ ↑ ↑
mathjax-newcm
🔗
sed-inject-mathjax-font-codex ↑
sed -i 's|output:[[:space:]]*{|loader: { paths: { "mathjax-font": "/vendor/mathjax-font-font" } },\
output: {|'
🔗
sed-inject-mathjax-font-legacy ↑
sed -i '/<script type="text\/x-mathjax-config">/,/<\/script>/c\
<script>\
  window.MathJax = {\
    tex: {\
      ams: {\
        multlineWidth: '\''85%'\''\
      },\
      tags: '\''ams'\'',\
      tagSide: '\''right'\'',\
      tagIndent: '\''.8em'\''\
    },\
    chtml: {\
      scale: 1.0,\
      displayAlign: '\''center'\'',\
      displayIndent: '\''0em'\''\
    },\
    svg: {\
      scale: 1.0,\
      displayAlign: '\''center'\'',\
      displayIndent: '\''0em'\''\
    },\
    loader: { paths: { "mathjax-font": "/vendor/mathjax-font-font" } },\
    output: {\
      font: '\''mathjax-font'\'',\
      displayOverflow: '\''overflow'\''\
    }\
  };\
</script>'

4.2.2 Fix jQuery

Some of the pages pull in jQuery from upstream; we have to host these locally as well. They are vendored inside vendor, but we need to make some more sed replacements to make the relevant pages refer to the vendored version.

🔗
fix-jquery ↑
find codex -path 'codex/deps' -prune -o -type f -name "*.html" -exec \
	sed-inject-local-jquery-codex {} +
find legacy-generated-repo -type f -name "*.html" -exec \
	sed-inject-local-jquery-legacy {} +
🔗
sed-inject-local-jquery-codex ↑
sed -i 's|src="https://code.jquery.com/jquery-3.6.4.min.js"|src="/vendor/jquery-3.6.4.min.js"|g'
🔗
sed-inject-local-jquery-legacy ↑
sed -i 's|src="https://code.jquery.com/jquery-2.1.4.min.js"|src="/vendor/jquery-2.1.4.min.js"|g'

4.2.3 Fix Google web fonts

We have to host the Source family of fonts directly as well, instead of pulling them over the network from Google web fonts.

🔗
vendor-source-fonts.sh
🎯 vendor-source-fonts.sh
set -euo pipefail

url_sans="https://raw.githubusercontent.com/adobe-fonts/source-sans/3.052R"
url_serif="https://raw.githubusercontent.com/adobe-fonts/source-serif/4.005R"
url_code="https://raw.githubusercontent.com/adobe-fonts/source-code-pro/2.042R-u%2F1.062R-i%2F1.026R-vf"

mkdir -p vendor/source-fonts
pushd vendor/source-fonts

curl -OL "${url_sans}/WOFF2/VF/SourceSans3VF-Italic.otf.woff2"
curl -OL "${url_sans}/WOFF2/VF/SourceSans3VF-Upright.otf.woff2"

curl -OL "${url_serif}/WOFF2/VAR/SourceSerif4Variable-Italic.otf.woff2"
curl -OL "${url_serif}/WOFF2/VAR/SourceSerif4Variable-Roman.otf.woff2"

curl -OL "${url_code}/WOFF2/VF/SourceCodeVF-Italic.otf.woff2"
curl -OL "${url_code}/WOFF2/VF/SourceCodeVF-Upright.otf.woff2"

cat << EOF > source-fonts.css
@font-face{
    font-family: 'Source Code Pro';
    font-weight: 200 900;
    font-style: normal;
    font-stretch: normal;
    src: url('SourceCodeVF-Upright.otf.woff2') format('woff2');
}

@font-face{
    font-family: 'Source Code Pro';
    font-weight: 200 900;
    font-style: italic;
    font-stretch: normal;
    src: url('SourceCodeVF-Italic.otf.woff2') format('woff2');
}

@font-face{
    font-family: 'Source Sans 3';
    font-weight: 200 900;
    font-style: normal;
    font-stretch: normal;
    src: url('SourceSans3VF-Upright.otf.woff2') format('woff2');
}

@font-face{
    font-family: 'Source Sans 3';
    font-weight: 200 900;
    font-style: italic;
    font-stretch: normal;
    src: url('SourceSans3VF-Italic.otf.woff2') format('woff2');
}

@font-face{
    font-family: 'Source Serif 4';
    font-weight: 200 900;
    font-style: normal;
    font-stretch: normal;
    src: url('SourceSerif4Variable-Roman.otf.woff2') format('woff2');
}

@font-face{
    font-family: 'Source Serif 4';
    font-weight: 200 900;
    font-style: italic;
    font-stretch: normal;
    src: url('SourceSerif4Variable-Italic.otf.woff2') format('woff2');
}
EOF

Now make the HTML content refer to the local fonts. 8 makes the old blog posts use "Source Serif 4", not "Source Serif Pro" (because the former is what we download).

🔗
fix-google-fonts ↑
find codex legacy-generated-repo -path 'codex/deps' -prune -o \
	-type f -name "*.html" -exec \
	sed-use-local-google-fonts {} +

# 8
sed -i 's|Source Serif Pro|Source Serif 4|g' legacy-generated-repo/css/base.css
🔗
sed-use-local-google-fonts ↑
sed -i \
's|href="https://fonts\.googleapis\.com[^"]*"|href="/vendor/source-fonts/source-fonts.css"|g'

4.2.4 Reference vendored assets

Finally we need to actually use these vendored assets by referring to them in every woven HTML file. That's what we do in custom-html-head.

🔗
custom-html-head ⇑
[htmlHead]
onlyLocalImports = true
injection = """
<link rel="stylesheet" href="/vendor/source-fonts/source-fonts.css">
<script>
  window.MathJax = {
      loader: { paths: { "mathjax-newcm": "/vendor/mathjax-newcm-font" } },
      output: { font: "mathjax-newcm" },
  tex: {
      tags: 'ams'
  },
  options: {
      ignoreHtmlClass: 'verbatim|code'
  }
};
</script>
<script id="MathJax-script" async="" src="/vendor/mathjax/tex-mml-chtml.js"></script>
"""

5 Hakyll setup

Below is a record of how the old content used to be built with Hakyll. This is here mainly for historical interest. It will be removed after we've fully transitioned away to the new publishing step with Lilac.

🔗
Makefile:Hakyll
# This Makefile is expected to be run inside a nix-shell.

PROJ_ROOT := $(shell git rev-parse --show-toplevel)
BLOG_ADDR?=localhost
BLOG_PORT?=8020

all: sync
.PHONY: all

# Check for broken links.
check:
	cabal run -- blog check
.PHONY: check

gen-css:
	cabal build -- base >/dev/null
	cabal exec -- base
.PHONY: gen-css

# JavaScript generated from Rust.
gen-js:
	rustup default stable
	make -C rust-js build
.PHONY: gen-js

sync: build-site
	./sync.sh
.PHONY: sync

cabal-update:
	cabal update
.PHONY: cabal-update

build-binaries:
	cabal build
.PHONY: build-binaries

build-site: build-binaries
	cabal run -- blog rebuild
.PHONY: build-site

watch: build-binaries
	cabal run -- blog watch --host $(BLOG_ADDR) --port $(BLOG_PORT)
.PHONY: watch

nixpkgs_stable_channel := nixos-24.05
update-deps: nix/sources.json nix/sources.nix
	niv update nixpkgs --branch $(nixpkgs_stable_channel)
	niv update
	touch update-deps

The local listx.github.io folder used to have the Github repo which held the generated content. Hakyll would by default publish to _site, so the script below would sync these two with rsync.

🔗
dest='listx.github.io/'
targ='_site/'

if [[ ! -d $targ ]]; then
    echo "\`$targ' directory missing"
    exit 1
elif [[ ! -d $dest ]]; then
    echo "\`$dest' directory missing"
    exit 1
fi

rsync -ahP --no-whole-file --inplace --delete --exclude=.git\* $targ $dest

6 Miscellaneous

🔗
🎯 .gitignore
*.hi
*.html
*.o
*.lobj
*.larc
*.tar
*.tar.gz
**/__pycache__/
code/2017-04-02-calling-c-from-haskell/hs/ffi
code/2021-03-15-bresenham-circle-drawing-algorithm/golden.yaml
code/toy/binary-search-c
code/toy/binary-search-hs
.direnv
site
.srht-pages-token
update-deps
tangle
weave
lint
publish
vendor/
🔗
.gitattributes
🎯 .gitattributes
vendor/*.js binary

Page metrics

Backlinks (1)

  1. index.org (1)
    1. ^ Top ← ¶ (Cell 68)

Tangled files (6)

  1. .gitattributes
  2. .gitignore
  3. Lilac.toml
  4. Makefile
  5. vendor-mathjax.sh
  6. vendor-source-fonts.sh

Named cells (15)

  1. .gitattributes
  2. Makefile:Hakyll
  3. Makefile:publish
  4. custom-html-head
  5. fix-google-fonts
  6. fix-jquery
  7. fix-mathjax
  8. mathjax-font
  9. sed-inject-local-jquery-codex
  10. sed-inject-local-jquery-legacy
  11. sed-inject-mathjax-font-codex
  12. sed-inject-mathjax-font-legacy
  13. sed-use-local-google-fonts
  14. vendor-mathjax.sh
  15. vendor-source-fonts.sh