Is it possible that you have reset the board before the flash was fully written. There are two parts to it, the serial transfer into RAM which takes a while even at 115200bps, and then the contents of RAM being written to FLASH, which also takes a while because each sector/block of Flash need to be erased before new data can be written (typically each 4K/32K/64K takes 0.07/0.16/0.3 seconds respectively - individually insignificant, but it adds up fast).
(For VF2_v2.8.0)
filename: u-boot-spl.bin.normal.out
filesize: 130688 bytes
CRC-32: ca14da8e
MD5: c44036b3a07ec93a165b76ec3df0704e
SHA-1: 316c7be84eabbe8ab440dcd1ba4d4ac70d0ae817
filename: visionfive2_fw_payload.img
filesize: 2797189 bytes
CRC-32: 34d1ba5b
MD5: 21ae1a326e9ab5c52a7502da059d7bc5
SHA-1: 78b6a1526bec26b7d9d1d50c7c3001cf581003d8
It will take about ~22 times longer to write the much larger u-boot+sbi+dtb payload image fully to flash than the u-boot second program loader. And some of the error messages you are seeing, in my mind anyhow, might be consistent with the dtb’s being truncated. Also validate the file sizes and checksums, it might be possible that your download was truncated maybe ?