Frankly I don’t really understand these addresses. I suppose I should read about it. My thinking was to keep it below
ramdisk_addr_r=0x48100000
and
fdt_addr_r=0x48000000
defined in uEnv.txt
The size of the overlay.dtbo is about 1.6kB.
I don’t know. I await advice from somebody more knowledgeable.
Funny enough that the uEnv.txt changes initramfs and device tree addresses away from U-Boot defaults set by StarFive as well. A mystery why they do not simple adjust the values in U-Boot, if for some reason this is seen as the better defaults, but instead ship an environment file to change them 
. However, in this case, 0x47000000 works well (device tree then is at 1152 MiB, 16 MiB away), of course. I was hoping for a distro-wide valid value (when SPI bootloader is used), but whether the addresses are changed or not depends on which uEnv.txt is actually used (highly inconsistent: VisionFive2/conf at JH7110_VisionFive2_devel · starfive-tech/VisionFive2 · GitHub) and whether the partitioning is actually so that the uEnv.txt is read at all (must be on 3rd partition with FAT filesystem.
That is so odd to force the 3rd partition, up until then the Type-UUID (AKA GUID Type) is used for the rest of the boot up sequence, which I am guessing will be similar to this:
From: https://starfivetech.com/uploads/hifive-unleashed-getting-started-guide-v1p3.pdf
The Freedom U500 supports several boot methods. The general flow follows multiple stages:
ROM0 (0x1004): Decode MSEL and jump to the address found in 0x1100 + 8*MSEL. For MSEL > 4, the process proceeds to the next step.
ROM1 (0x1_0000): Decode MSEL to determine which media contains the First Stage Boot Loader (FSBL). The media is expected to be formatted with a GPT partition table (even on SPI flash). The entire contents of the first partition with GUID type 5B193300-FC78-40CD-8002-E86C45580B47 are downloaded into the L2 sideband memory and execution transfers there. ROM1 contains a hard-coded DTB block passed in a1 to the FSBL, which ignores it.
FSBL (0x800_0000): Initialize core PPLs, DDR, and Ethernet PHY. Decode MSEL to determine which media contains the Berkeley Boot Loader (BBL). The media is expected to contain a GPT-formatted partition table. The contents of the first partition with GUID type 2E54B353-1271-4842-806F-E436D6AF6985 are downloaded into DDR memory and execution transfers there. The FSBL contains an embedded DTB describing both the Freedom U500 SoC and HiFive Unleashed board. The DTB is updated with the board MAC address and DDR capacity and passed to BBL in a1.
(You can use “lsblk -f” or “fdisk -x” to see the GUID type)
$ /sbin/fdisk -x starfive-jh7110-VF2-SD-wayland.img
Disk starfive-jh7110-VF2-SD-wayland.img: 2.64 GiB, 2831155200 bytes, 5529600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5BDD0F8F-36BE-4C36-B1D9-1A904F6B5393
First LBA: 34
Last LBA: 5529566
Alternative LBA: 5529599
Partition entries LBA: 2
Allocated partition entries: 128
Device Start End Sectors Type-UUID UUID Name Attrs
starfive-jh7110-VF2-SD-wayland.img1 4096 8191 4096 2E54B353-1271-4842-806F-E436D6AF6985 E9C01DC7-C7E9-4E9D-8D2D-C19B30E73742
starfive-jh7110-VF2-SD-wayland.img2 8192 16383 8192 5B193300-FC78-40CD-8002-E86C45580B47 FBA0475F-9A11-4E24-B439-991F74EEAB7F
starfive-jh7110-VF2-SD-wayland.img3 16384 221183 204800 C12A7328-F81F-11D2-BA4B-00A0C93EC93B 0C10506E-2B34-40B5-9F06-92CF5745E874
starfive-jh7110-VF2-SD-wayland.img4 221184 5527518 5306335 0FC63DAF-8483-4772-8E79-3D69D8477DE4 B3A3CFE8-EAD8-41E0-9465-BE8EB26C0908 LegacyBIOSBootable
$ /sbin/fdisk -l starfive-jh7110-VF2-SD-wayland.img
Disk starfive-jh7110-VF2-SD-wayland.img: 2.64 GiB, 2831155200 bytes, 5529600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5BDD0F8F-36BE-4C36-B1D9-1A904F6B5393
Device Start End Sectors Size Type
starfive-jh7110-VF2-SD-wayland.img1 4096 8191 4096 2M HiFive Unleashed BBL
starfive-jh7110-VF2-SD-wayland.img2 8192 16383 8192 4M HiFive Unleashed FSBL
starfive-jh7110-VF2-SD-wayland.img3 16384 221183 204800 100M EFI System
starfive-jh7110-VF2-SD-wayland.img4 221184 5527518 5306335 2.5G Linux filesystem
$
And if you look here:
https://doc-en.rvspace.org/VisionFive2/Boot_UG/JH7110_SDK/bootrom.html
There is even the option for a second boot backup partition:
second boot"–typecode=1:2E54B353-1271-4842-806F-E436D6AF6985
second boot bak"–typecode=2:2E54B353-1271-4842-806F-E436D6AF6984
So I would have expected it to use the first “EFI System” (C12A7328-F81F-11D2-BA4B-00A0C93EC93B) Type-UUID partition next, which is really just FAT32 in disguise.
cwt12 image won’t boot
StarFive # boot
Can't set block device
## Warning: defaulting to text format
Can't set block device
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Retrieving file: /boot/extlinux/extlinux.conf
Can't set block device
Error reading config file
frustrating, there doesn’t seem to be a latest proper image to download and use from thefossguy either, the images dont seem to work using dd and generate an error about incorrect architecture
Did you set boot mode to SD? And what kind of power supply? Please try USB-PD and avoid any quick charge (USB-A).
My image booted on the board revision A and B, and they survived more than a month already with rebooting 3 - 4 times after some major packages updated. So I’m pretty sure that it is stable.
I looked in your archive and tried ArchLinux-VF2_515_v2.8.0-cwt6.1.img.xz and it worked, was a 15 gig expand but as long as it works it’s ok I don’t think I did anything wrong previously but who knows. See the difference with this image.
StarFive # boot
Can't set block device
Importing environment from mmc1 ...
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
385 bytes read in 3 ms (125 KiB/s)
## Warning: defaulting to text format
## Error: "boot2" not defined
47809 bytes read in 10 ms (4.6 MiB/s)
Retrieving file: /boot/extlinux/extlinux.conf
1758 bytes read in 7 ms (245.1 KiB/s)
U-Boot menu
1: Arch Linux 5.15.0-cwt6-server
2: Arch Linux 5.15.0-cwt6-server (Single User Mode)
3: Arch Linux 5.15.0-cwt6-desktop
4: Arch Linux 5.15.0-cwt6-lowlatency
Enter choice: 1: Arch Linux 5.15.0-cwt6-server
Retrieving file: /boot/initrd.img-5.15.0-cwt6-server
9167164 bytes read in 394 ms (22.2 MiB/s)
Retrieving file: /boot/vmlinuz-5.15.0-cwt6-server
9235419 bytes read in 395 ms (22.3 MiB/s)
append: root=/dev/mmcblk1p3 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 rootflags=defaults,discard=async,compress=lzo,user_subvol_rm_allowed,subvol=arch-minimal
Retrieving file: /boot/dtbs/starfive/jh7110-visionfive-v2.dtb
47809 bytes read in 10 ms (4.6 MiB/s)
Uncompressing Kernel Image
Interesting!
Thanks for this cwt!
I got my VF2 last week and this seems to be the best VF2 distro I’ve tried yet, although I’ve got a few issues / questions:
Could you please change the name of the root BTRFS subvolume to just @ (instead of @arch-minimal or whatever it is) in future releases? This would make it compatible with the default Ubuntu and Arch BTRFS layout and allow users to use timeshift to manage BTRFS snapshots.
Has anyone got Arch booting directly from NVMe on the VF2 using the latest VF2 u-boot and firmware? If so, could you please share the exact commands you run get this configured? I tried to do this yesterday but I was unable to correctly mount the VF2 Arch BTRFS partition to edit its fstab.
I was unable to get X or Wayland working yesterday after installing gnome, xorg, lightdm etc but I’ve not tried copying the Xorg.conf over from the Starfive Debian image, which might fix it? I’d prefer to be using the open source driver tho which brings me nicely onto…
Finally, has anyone produced PKGBUILDS for the open source mesa imagination GPU driver or ffmpeg (or ideally mpv) built with support for VDEC hardware video decoding?
Thanks
I just upload the cwt13 image, it’s based on StarFive release v3.0.4. I also add a section to the post on how to boot it directly on NVMe.
@danboid to use @ as subvolume instead of arch-minimal, please do the following command:
mount /mnt
cd /mnt
btrfs subvolume snapshot arch-minimal @
sed -i -e 's/arch-minimal/@/g' /boot/extlinux/extlinux.conf
sed -i -e 's/arch-minimal/@/g' /etc/fstab
If you can boot using the new subvolume successfully, then you may want to delete the old one:
mount /mnt
cd /mnt
btrfs subvolume delete arch-minimal
I have successfully booted cwt13 on my VF2 from TF and used it to upgrade to the latest firmware. I’ve not tried booting it from NVMe yet.
Please include mtd-utils by default in cwt14.
Thanks
cwt:
In your NVMe boot instuctions you have the user manually create the partitions then rsync the data frm the SD card to the NVMe. It should surely be possible and much easier to just burn the SD card image straight onto the NVMEe disk then just resize the 4th (BTRFS root) partition to fill the disk?
There is at least one mistake in the current instructions too, the line
mount /mnt
It seems this line should just be removed because the user should already have the root partition mounted.
You seem to be implying that Arch should auto boot from SSD after following your instructions (using latest firmware/u-boot and having set the DIP switch to QSPI mode) but that isn’t the case for me. Are there no u-boot config commands required?
I should be able to get it to boot using pretty much the same u-boot commands I used for Debian but I’d prefer to get extlinux.conf being used instead, which seems to be how you have it configured.
No, no, please don’t rsync from mounted root, mount /mnt first so you can rsync only the files from subvolume without interfearing of other mounted fs like /dev or /sys ontop of the mounted root.
Are there no u-boot config commands required?
I didn’t do any of that.
root@zbook:/home/dan# mount /mnt/
mount: /mnt/: can't find in /etc/fstab.
I’m not with my VF2 now, tomorrow I will check it. But if I remember correctly, my image shoud have /mnt in fstab.
Maybe I should try again with your exact method. I was trying by burning the SD card to NVMe then I ran the sed commands to adjust the extlinux.conf and fstab. There’s no reason why that method shouldn’t be able to work and it’d be much easier than creating the partitions and using rsync etc.
If its the subvolumes stopping my method working then maybe just exclude subvols from the sdcard image and let users add them in later if they want subvolumes. Having a simple NVMe install is more useful to more users than subvols.
I reset u-boot to the default settings (which I thought I’d already tried but…) and now it does start to boot to but it fails to mount the root partition.
I have run btrfs check on the root partition and it seems to be fine.
[rootfs ]# �
U-Boot SPL 2021.10 (May 31 2023 - 16:54:00 +0800)
DDR version: dc2e84f0.
Trying to boot from SPI
OpenSBI v1.2
____ _____ ____ _____
/ __ \ / ____| _ \_ _|
| | | |_ __ ___ _ __ | (___ | |_) || |
| | | | '_ \ / _ \ '_ \ \___ \| _ < | |
| |__| | |_) | __/ | | |____) | |_) || |_
\____/| .__/ \___|_| |_|_____/|___/_____|
| |
|_|
Platform Name : StarFive VisionFive V2
Platform Features : medeleg
Platform HART Count : 5
Platform IPI Device : aclint-mswi
Platform Timer Device : aclint-mtimer @ 4000000Hz
Platform Console Device : uart8250
Platform HSM Device : ---
Platform PMU Device : ---
Platform Reboot Device : pm-reset
Platform Shutdown Device : pm-reset
Platform Suspend Device : ---
Firmware Base : 0x40000000
Firmware Size : 392 KB
Firmware RW Offset : 0x40000
Runtime SBI Version : 1.0
Domain0 Name : root
Domain0 Boot HART : 1
Domain0 HARTs : 0*,1*,2*,3*,4*
Domain0 Region00 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region01 : 0x0000000040000000-0x000000004003ffff M: (R,X) S/U: ()
Domain0 Region02 : 0x0000000040040000-0x000000004007ffff M: (R,W) S/U: ()
Domain0 Region03 : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X)
Domain0 Next Address : 0x0000000040200000
Domain0 Next Arg1 : 0x0000000042200000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes
Domain0 SysSuspend : yes
Boot HART ID : 1
Boot HART Domain : root
Boot HART Priv Version : v1.11
Boot HART Base ISA : rv64imafdcbx
Boot HART ISA Extensions : none
Boot HART PMP Count : 8
Boot HART PMP Granularity : 4096
Boot HART PMP Address Bits: 34
Boot HART MHPM Count : 2
Boot HART MIDELEG : 0x0000000000000222
Boot HART MEDELEG : 0x000000000000b109
U-Boot 2021.10 (May 31 2023 - 16:54:00 +0800), Build: jenkins-github_visionfive2-14
CPU: rv64imacu_zba_zbb
Model: StarFive VisionFive V2
DRAM: 8 GiB
MMC: sdio0@16010000: 0, sdio1@16020000: 1
Loading Environment from SPIFlash... SF: Detected gd25lq128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
OK
StarFive EEPROM format v2
--------EEPROM INFO--------
Vendor : StarFive Technology Co., Ltd.
Product full SN: VF7110B1-2253-D008E000-00006423
data version: 0x2
PCB revision: 0xb2
BOM revision: A
Ethernet MAC0 address: 6c:cf:39:00:65:2a
Ethernet MAC1 address: 6c:cf:39:00:65:2d
--------EEPROM INFO--------
In: serial
Out: serial
Err: serial
Model: StarFive VisionFive V2
Net: eth0: ethernet@16030000, eth1: ethernet@16040000
starfive_pcie pcie@2C000000: Starfive PCIe bus probed.
PCI: Failed autoconfig bar 10
pci enum ...
IDE device 0: Vendor: 0x144d Rev: 5B2QGXA7 Prod: S5GYNX0TA78788V
Type: Hard Disk
Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
bootmode flash device 1/0
Hit any key to stop autoboot: 0
Failed to load 'vf2_nvme_uEnv.txt'
Importing environment from 1/0 ...
## Warning: Input data exceeds 1048576 bytes - truncated
## Info: input data size = 1048578 = 0x100002
## Error: "boot2" not defined
pci enum ...
IDE device 0: Vendor: 0x144d Rev: 5B2QGXA7 Prod: S5GYNX0TA78788V
Type: Hard Disk
Capacity: 476940.0 MB = 465.7 GB (976773168 x 512)
406 bytes read in 3 ms (131.8 KiB/s)
## Warning: defaulting to text format
50067 bytes read in 4 ms (11.9 MiB/s)
50067 bytes written in 1 ms (47.7 MiB/s)
Retrieving file: /extlinux/extlinux.conf
908 bytes read in 4 ms (221.7 KiB/s)
U-Boot menu
1: Arch Linux 5.15.2-cwt13
2: Arch Linux 5.15.2-cwt13 (Single User)
Enter choice: 1: Arch Linux 5.15.2-cwt13
Retrieving file: /initrd.img-5.15.2-cwt13
9810596 bytes read in 55 ms (170.1 MiB/s)
Retrieving file: /vmlinuz-5.15.2-cwt13
10553965 bytes read in 59 ms (170.6 MiB/s)
append: root=/dev/nvme0n1p4 rw console=tty1 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 rootflags=defaults,discard=async,compress=zstd,user_subvol_rm_allowed,subvol=arch
Retrieving file: /dtbs/starfive/jh7110-visionfive-v2.dtb
50067 bytes read in 5 ms (9.5 MiB/s)
Uncompressing Kernel Image
Moving Image from 0x44000000 to 0x40200000, end=41cb9000
## Flattened Device Tree blob at 48000000
Booting using the fdt blob at 0x48000000
Using Device Tree in place at 0000000048000000, end 000000004800f392
Starting kernel ...
clk u2_dw_i2c_clk_core already disabled
clk u2_dw_i2c_clk_apb already disabled
clk u5_dw_i2c_clk_core already disabled
clk u5_dw_i2c_clk_apb already disabled
[ 0.546402] L2CACHE: DataError @ 0x00000000.08040110
[ 0.551894] L2CACHE: DataFail @ 0x00000000.08040140
[ 1.258053] jh7110-sec 16000000.crypto: Unable to request sec_m dma channel in DMA channel
[ 1.267213] jh7110-sec 16000000.crypto: Cannot initial dma chan
[ 1.372578] debugfs: Directory '16008000.sec_dma' with parent 'dmaengine' already present!
[ 1.385166] seeed_panel 2-0045: Unknown Atmel firmware revision: 0x00
[ 1.425442] imx219 6-0010: failed to read chip id 219
[ 1.928071] usb usb2-port2: over-current condition
[ 2.064060] usb usb2-port4: over-current condition
[ 2.193039] mailbox_test soc:mailbox_client@0: invalid resource
[ 2.199085] mailbox_test soc:mailbox_client@0: invalid resource
[ 2.206224] starfive-i2s 120b0000.i2stx_4ch0: designware: i2s master mode supported
[ 2.214641] cdns-dsi 295d0000.mipi: starfive dsi bind end
:: running early hook [udev]
Starting systemd-udevd version 253.4-1-arch
:: running hook [udev]
:: Triggering uevents...
:: running hook [keymap]
:: Loading keymap...kbd_mode: KDSKBMODE: Inappropriate ioctl for device
done.
:: running hook [consolefont]
:: Loading console font...setfont: KDFONTOP: Inappropriate ioctl for device
done.
:: performing fsck on '/dev/nvme0n1p4'
:: mounting '/dev/nvme0n1p4' on real root
mount: /new_root: mount(2) system call failed: No such file or directory.
dmesg(1) may have more information after failed mount system call.
ERROR: Failed to mount '/dev/nvme0n1p4' on real root
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off
Has anyone got a display server working on this image yet? I tried for a bit but couldn’t get anything working.
I’ve got Arch booting directly from NVMe now, using the easy method of just burning the SD image to NVMe. I had to edit the extlinux.conf so that the subvol boot arguments ends with subvol=arch-minimal and not subvol=arch as is the default. Once you’ve done that it boots directly from NVMe with the default u-boot settings.
One of these latest updates (The VF2 3.0.4 firmware update?) has slightly increased my SSD read speed from 155 MB/s to 165 MB/s.
Not yet but now that I’ve got NVME boot working properly with Arch my next task is to create a mesa PKGBUILD with the experimental PowerVR support enabled, unless someone has beat me to it?