still needed one more fix for xtree/tree_depend Signed-off-by: Leah Rowe <info@minifree.org> |
||
---|---|---|
canoe0.1test.sh | ||
canoetest.sh | ||
gnuboot0.1test.sh | ||
gnuboot0.2test.sh | ||
psdgtest.sh | ||
README.md |
Benchmarking coreboot distros
As Libreboot founder, I sometimes have fun working on anything other than Libreboot, so I took GNU Boot 0.1 RC3 and started fixing issues, sending patches to them for review. My purpose was to bring it in somewhat of a sync with Libreboot and Canoeboot, so that I could benchmark.
I will be maintaining this repository over time, comparing the build performance of various coreboot distros. I currently test Libreboot, Canoeboot and GNU Boot.
Current benchmark scores, on the scripts included:
Running ./build roms all
in lbmk/cbmk and ./build boot roms all
in
the gnu build system, we get the total build times:
- cbmk: 4736 seconds
- gnu0.1: 7135 seconds
- gnu0.2: 6190 seconds
- lbmk: 6639 seconds (psdgtest.sh)
The test machine was a ThinkPad T440p, running Libreboot 20231106, with a Intel i7-4940MX in it. PC3-12800S RAM. Running on a dedicated M2 SATA SSD (it saturates the bus and is pretty fast). It was done on a machine running Debian 12.4 (stable release at the time of writing), on 18 January 2024.
NOTE: make sure to always run the tests fresh, in a clean clone of this repository.
BUILD DEPENDENCIES: Just install the build dependencies for Libreboot, and then you will also have them for GNU Boot, which has more or less the same dependencies. Info about how to install build dependencies on various distros is here: https://libreboot.org/docs/build/#first-install-build-dependencies
Check each script in this repository. Each script has logic, tailored for build-testing each project.
It should be noted that lbmk builds roughly twice as many boards as gnuboot or canoeboot, yet is faster than gnuboot0.1 in the above test. The canoeboot (cbmk) test is faster than lbmk because it builds a lot fewer boards (about half as many boards), and doesn't have to extract files from vendor updates (e.g. ME neutering). During all of the second half of 2023, there were four major audits, bringing the code size down in Libreboot and Canoeboot build systems, optimising the build performance.
these test results are from benchmarking, to see just how much of a difference those optimisations made. NOTE: the 0.2 score is much lower(better) than the 0.1 score, because i added a few optimisations to that version of that build system, namely: don't use dd to insert ich9m ifd files and pike2008 fakeroms; instead, use pre-generated ich9m ifds from ich9gen, and declare pike2008 roms in coreboot menuconfig pointing to /dev/null. this is more efficient in terms of disk i/o because then it's done to 1 rom, and copied, rather than dd'd to multiple images.
About these tests
All of these build systems, in the above test results, support exactly the same boards, and all use the same upstream revisions, except for gnu0.1 which uses much older coreboot/grub/seabios/memtest86plus revisions for most x86 boards - but I added Dell Latitude E6400 support and gru_kevin and gru_bob chromebook boards from Libreboot, adding support for building the requisite U-Boot payload
files pre-downloaded
In each test, all files are pre-downloaded except crossgcc tarballs, because those are downloaded by coreboot's build system. Neither Canoeboot, Libreboot nor GNU Boot currently download crossgcc tarballs directly, but they are not a lot of data, and they download quickly from reliable servers - but the rest of it is a lot, and would skew the test due to varying internet speeds.
So, each test pre-downloads as much as possible. The purpose of the benchmark is to test build speeds, not measure your internet speed!
scripts
gnuboot0.1test.sh
the 0.1 gnu one is my patch series sent to fix build issues in gnuboot 0.1. i fixed build issues, made it compile on latest gentoo/arch/debian-sid as of 14 january 2024.
that patch series is here: https://git.disroot.org/vimuser/gnuboot/commits/branch/0.1-fix-build-v3
these patches were sent to the gnuboot mailing list for review. they enable gnuboot 0.1 to build on modern distros (gentoo, archlinux and debian sid were all tested, on around 14 january 2024)
gnuboot0.2test.sh
the 0.2 one is the same but updated to use same upstream revisions and patches as canoeboot. e.g. it uses grub 2.12 with argon2 support.
that patch series is here: https://git.disroot.org/vimuser/gnuboot/commits/branch/20240113-update-revs-3
these were also sent to the gnuboot mailing list, as a proof of concept, for them to cherry-pick whatever they want, if anything.
canoetest.sh
cbmk is canoeboot. find it here: https://canoeboot.org/
psdgtest.sh
psdg is libreboot. find it here: https://libreboot.org/
The times above will be updated, as and when the scripts are adapted to test new revisions.