The NetBSD OpenSSL PowerPC oopsie
Jun 7th, 2025 - scudoLast year, the NetBSD team has started to maintain an official port of the operating system to the Nintendo Wii, powered by a custom PowerPC processor by IBM.
Out of curiosity, I tried to run it on my console.
Everything went smooth, until when I tried running
pkg_add pkgin; the program would just hang immediately
and wouldn't respond.I then whipped out
gdb and ran the program through the debugger; I discovered that pkg_add
crashed during OpenSSL initialization. More specifically, it was crashing with an illegal instruction exception in
OPENSSL_ppc64_probe() from /usr/bin/libcrypto.so.15.
Wait, what?
PPC64? But the Wii only has a 32 bit CPU, no wonder it's crashing!I spent some more time disassembling parts of
libcrypto to figure out why an illegal instruction exception
was thrown in the first place. I ended up taking a look at the same function that crashed,
OPENSSL_ppc64_probe. This is what I found:
That
fcfid instruction is, indeed, present only on 64-bit PowerPC CPUs, as you can see from IBM's
website:
Testing on real hardware confirms this (again!): the GNU assembler (
as) indeed does not recognize the
instruction.
Digging out NetBSD's source code, I can indeed confirm that
libcrypto contains this function. Its scope appears to check if the CPU the OS is running on supports
64-bit
integer and floating point operations. The problem is that this code is part of the
powerpc source tree, which is the 32-bit flavour of the
architecture, so running this function will always raise an illegal instruction exception:
In the end, I gave up trying to figure this out - but this bug still persists in what was the latest PowerPC release when all of this happened (April 23, 2025).
