, 3 min read

Performance Comparison Saaze vs. Hugo vs. Zola

Original post is here eklausmeier.goip.de/blog/2021/11-13-performance-comparison-saaze-vs-hugo-vs-zola.

Zola is a static site generator written in Rust. Hugo is a static site generator written in Go. Simplified Saaze, Saaze for short, is written in PHP.

Task: Compare run times of Saaze vs. Hugo vs Zola. I didn't find a direct comparison so far. There is a comparison of Eleventy vs. Gatsby vs. Hugo vs. Jekyll vs. Next vs. Nuxt.

Installation size on Arch Linux for x86 and Odroid/ARM.

Generator Size/MB x86 Size/MB ARM
Simplified Saaze 1.1 0.05 0.05
Hugo 0.88.1 61.01 49.17
Zola 0.14.1 21.33 16.89

Run times are all in seconds. Two different machines were used.

  1. x86 is Intel NUC i5-4250U, 4 cores, max 2.6 GHz running Arch Linux 5.14.14
  2. ARM is Cortex-A7, 8 core, big-little, max 1.5 GHz running Arch Linux 4.14.180-3

The 13 posts are the music posts in this blog, the 347 posts is the sum of all posts in this blog. 681 and 1349 are just copies of the 347 posts. Below table disadvantages the Simplified Saaze generator as neither PHP's OPCache nor JIT was enabled!

Generator #posts real x86 user x86 real ARM user ARM
S. Saaze 13 0.04 0.02 0.09 0.06
S. Saaze 347 0.15 0.09 0.37 0.25
S. Saaze 681 0.26 0.17 0.64 0.41
S. Saaze 1349 0.47 0.36 1.32 0.86
Hugo 13 0.22 0.31 0.76 1.47
Hugo 347 1.53 4.50 4.39 17.73
Hugo 681 2.43 7.34 6.76 29.88
Hugo 1349 3.87 12.41 11.83 52.08
Zola 13 0.08 0.06 0.27 0.23
Zola 347 0.36 0.31 1.16 1.10
Zola 681 0.60 0.58 2.05 1.99
Zola 1349 1.17 1.09 3.75 3.67

Initially one would think that static site generators written in Go and Rust are faster than generators written in PHP. This time, PHP beats them all.

User times in Hugo are way higher as Hugo uses threads to parallelize the task. But although it fully utilizes all the cores, it is way slower than Saaze and Zola. Neither Saaze nor Zola use threads. So if Saaze would use threads then the difference in run time would even get bigger. It would be quite easy to use threads in Saaze, i.e., so called entries and the chunks of collections could easily be processed in parallel.

Comparing the x86 real- and user-time:

Comparing the ARM real- and user-time:

Results for Hugo:

  1. Hugo is always the slowest, contrary to its constant bragging about "The world’s fastest framework for building websites"
  2. Hugo's templates are difficult to use, also see Migrating my blog to Zola
  3. Hugo is a CPU hog, also see Converting WordPress Export File to Hugo
  4. Hugo's Goldmark implementation does not honor plain, simple HTML
  5. In defense of Hugo: Hugo automatically uses syntax highlighting. This clearly slows down processing. I switched off syntax highlighting in Zola as well, otherwise Zola would be roughly two times slower.

So apparently it was a wise decision not to use Hugo for this blog, but go with Saaze instead, being way simpler to use and an order of magnitude faster.

Added 13-Jan-2022: Similar findings with regard to inferior performance of Hugo vs. Zola were found in Static site generator benchmarks. The numbers are:

Generator Time in ms
Blade 2.9
Zola 29.1
Hugo 45.7

Here Blade is another static site generator written by Maroš Grego. It is programmed in Rust and uses the mustache templating system.

The Nift static site generator also beats Hugo by a factor of four for real time. For user time, i.e., not accounting for multi-core CPUs, Hugo is more than 6-times slower. The results:

Generator Real User Sys
Nift 1.107 1.846 1.982
Hugo 4.222 11.644 1.276

Nift is written in C, C++ and Lua. Source code in GitHub. It was written by Nicholas Ham.

Added 11-Feb-2024: Below numbers are from Which generator builds Markdown the fastest? Numbers are in seconds and were measured on a MacBook Air (M1, 2020), macOS Monterey 12.5, 16 GB memory.

#Markdown files 250 500 1,000 2,000 4,000
Astro 1.0.1 2.270 3.172 5.098 9.791 22.907
Eleventy 1.0.1 0.584 0.683 0.914 1.250 1.938
Gatsby 4.19.0-cli 14.462 15.722 17.967 22.356 29.059
Hugo v0.101.0 0.071 0.110 0.171 0.352 0.684
Next.js 12.2.3 (JS routing) 6.552 6.932 8.034 9.582 13.409
Next.js 12.2.3 (File routing) 7.958 9.551 14.304 25.038 70.653
Remix 1.6.5 (File routing) 2.876 8.258 46.918 349.125 1800.000

According CPU Monkey Intel Core i5-4250U vs Apple M1 the M1 is roughly three to six times faster than the i5 used in above benchmarks.