I suspect the recovery tool (which they haven’t published source code for) still follows an older SPI flash layout which reserved less space for the bootloaders.
In your shoes, since the SPI flash isn’t even touched at all when booting with the switches set to any other setting than SPI, such as UART (you’re using) or SD… I’d write a recent release’s sd image into an sd card and boot that.
Then I’d interrupt u-boot’s autoboot by pressing a key, and use this method to upgrade the SPI: