Experience VisionFive v2 day01: BOOTING FROM NVME

larryw3i · April 15, 2023, 1:30pm

Experience VisionFive v2 day01

All of the following may damage your device or delete your data. Please proceed with caution.

1. Download the `img` and flash it to Micro-SD Card.

Download the starfive-jh7110-VF2-SD-wayland.img.bz2 from debian.starfivetech.com and unzip it via bunzip2, copy it to Micro-SD Card:

larry@zx0:~/Downloads$ bunzip2 starfive-jh7110-VF2-SD-wayland.img.bz2
# Be careful! the `/dev/sdb` is my Micro-SD Card device, your may be different.
larry@zx0:~/Downloads$ sudo dd if=starfive-jh7110-VF2-SD-wayland.img  of=/dev/sdb     status=progress 
2827981312 bytes (2.8 GB, 2.6 GiB) copied, 684 s, 4.1 MB/s 
5529600+0 records in
5529600+0 records out
2831155200 bytes (2.8 GB, 2.6 GiB) copied, 684.487 s, 4.1 MB/s

and then, insert the Micro-SD Card to VisionFive v2 and boot it, the Boot Mode is set to:

2. Connect to the network.

Now, you could log in your VisionFive v2 via this tutorial for a few minutes:

Username: user
Password: starfive

And connect to the network. For me, I use the USB wifi dongle to connect the Hotspot:

user@starfive:~$ sudo nmcli device wifi connect your-wifi-name password your-wifi-password  
Device '******' successfully activated with '******'.

3. Extend the `/dev/mmcblk1`.

By default, the Debian Image 202303 doesn’t add /sbin to PATH for common users, you may add it to PATH by yourself or fdisk will NOT be found.

user@starfive:~$ export PATH=$PATH:/sbin
user@starfive:~$ which fdisk 
/sbin/fdisk

Visit this Manual and extend the /dev/mmcblk1:

user@starfive:~$ sudo fdisk /dev/mmcblk1

Welcome to fdisk (util-linux 2.38.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

GPT PMBR size mismatch (4505599 != 62929919) will be corrected by write.
This disk is currently in use - repartitioning is probably a bad idea.
It''s recommended to umount all file systems, and swapoff all swap
partitions on this disk. 
Command (m for help): d
Partition number (1-4, default 4): 4  

Partition 4 has been deleted.

Command (m for help): n
Partition number (4-128, default 4): 4
First sector (34-62929886, default 221184):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (221184-62929886, default 62928895):

Created a new partition 4 of type 'Linux filesystem' and of size 29.9 GiB.
Partition #4 contains a ext4 signature.

Do you want to remove the signature? [Y]es/[N]o: N

Command (m for help): w

The partition table has been altered.
Syncing disks.

user@starfive:~$:sudo resize2fs /dev/mmcblk1p4  
resize2fs 1.46.6[ 1080.736178] EXT4-fs (mmcblk1p4): resizing filesystem from 663291 to 6061747
2 blocks
-rc1 (12-Sep-2022)
Filesystem at /dev/mmcblk1p4 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 29
[ 1083.041985] EXT4-fs (mmcblk1p4): resized filesystem to 60617472
The filesystem on /dev/mmcblk1p4 is now 60617472 (4k) blocks long.

After these operations, the /dev/mmcblk1p4 was extended:

user@starfive:~$ df -mh
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G     0  3.7G   0% /dev
tmpfs           793M  3.2M  790M   1% /run
/dev/mmcblk1p4  228G  2.5G  226G   2% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M   12K  5.0M   1% /run/lock
/dev/mmcblk1p3  100M   23M   78M  23% /boot
tmpfs           793M   36K  793M   1% /run/user/110
tmpfs           793M   24K  793M   1% /run/user/1000

4. Now, let’s try booting from `NVME`.

Note: It seems that the NVME may not be detected if the power is not greater than 15 watts.
scp the starfive-jh7110-VF2-SD-wayland.img to VisionFive v2 and copy it to NVME:

# Pay attention to my host-names, they are different hosts.

user@starfive:~$ mkdir Downloads

larry@zx0:~/Downloads$ scp starfive-jh7110-VF2-SD-wayland.img  user@starfive.local:~/Downloads
The authenticity of host 'starfive.local (192.168.31.242)' can't be established.
ED25519 key fingerprint is SHA256:*******************.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'starfive.local' (ED25519) to the list of known hosts.
user@starfive.local's password: 
starfive-jh7110-VF2-SD-wayland.img                          100% 2700MB   3.2MB/s   13:54    

user@starfive:~/Downloads$ sudo dd if=starfive-jh7110-VF2-SD-wayland.img of=/dev/nvme0n1  status=progress
2816003072 bytes (2.8 GB, 2.6 GiB) copied, 226 s, 12.5 MB/s
5529600+0 records in
5529600+0 records out
2831155200 bytes (2.8 GB, 2.6 GiB) copied, 230.018 s, 12.3 MB/s

Now, let’s Change SD card’s rootfs UUID (Reference from https://jamesachambers.com/starfive-visionfive-2-debian-ssd-boot-guide/)：

user@starfive:~/Downloads$ sudo tune2fs -U random /dev/mmcblk1p4
tune2fs 1.46.6-rc1 (12-Sep-2022)
Setting the UUID on this filesystem could take some time.
Proceed anyway (or wait 5 seconds to proceed) ? (y,N) <proceeding>
The UUID may only be changed when the filesystem is unmounted.
If you only use kernels newer than v4.4, run 'tune2fs -O metadata_csum_seed' and re-run this command.

Modify the /boot/extlinux/extlinux.conf:

user@starfive:~/Downloads$ cat /boot/extlinux/extlinux.conf > extlinux.conf.202304152100.cp
user@starfive:~/Downloads$ sudo sed -i 's/mmcblk1p4/nvme0n1p4/g' /boot/extlinux/extlinux.conf
user@starfive:~/Downloads$ cat /boot/extlinux/extlinux.conf
## /extlinux/extlinux.conf
##
## IMPORTANT WARNING
##
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update

default l0
menu title U-Boot menu
prompt 0
timeout 50


label l0
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive


        fdtdir /dtbs
        append root=/dev/nvme0n1p4 root=/dev/nvme0n1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0

label l0r
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive (rescue target)
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive

        fdtdir /dtbs
        append root=/dev/nvme0n1p4 root=/dev/nvme0n1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single

OK, try to reboot your VisioinFive v2:

user@starfive:~/Downloads$ sudo reboot

If it could be booted normally, congratulations!

user@starfive:~$ df -mh
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G     0  3.7G   0% /dev
tmpfs           793M  3.2M  790M   1% /run
/dev/nvme0n1p4  2.5G  2.4G   28M  99% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M   12K  5.0M   1% /run/lock
/dev/mmcblk1p3  100M   23M   78M  23% /boot
tmpfs           793M   36K  793M   1% /run/user/110
tmpfs           793M   24K  793M   1% /run/user/1000

Now, just repeat Step2 and Step3 again. (The device you need to extend may should be /dev/nvme0n1p4.)

5. Use the `Debian-ports` mirror.

The debian-ports mirrors I found are following:

May it’s helpful for y’all. Modify the /etc/apt/sources.list:

deb https://mirrors.aliyun.com/debian-ports unstable main

202304161208

Add the apt-key(Packages authentication):

 wget -O - https://www.ports.debian.org/archive_2023.key | sudo apt-key add -

6 Upgrade firmware to `v3.0.4`

The latest firmware for May was released, and the /boot partition doesn’t need to be mounted at /dev/mmcblk1p3 now. Feel free to upgrade it:

wget -O upgrade-firmware.sh https://raw.githubusercontent.com/larryw3i/VisionFive2/JH7110_VisionFive2_devel/upgrade-firmware.sh && sudo bash upgrade-firmware.sh

and edit /etc/fstab:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# /dev/mmcblk1p3       /boot       vfat    defaults       0 2
/dev/nvme0n1p3       /boot       vfat    defaults       0 2
# /dev/mmcblk1p4       /       ext4    errors=remount-ro       0 1
/dev/nvme0n1p4       /       ext4    errors=remount-ro       0 1

(Of course, these are all operated on installed Debian system. )

mount /dev/nvme0n1p3 and modify extlinux.conf:

$ mkdir nvme0n1p3
$ sudo mount /dev/nvme0n1p3 nvme0n1p3/
# cp the extlinux.conf of mmcblk1p3 to nvme0n1p3
$ sudo cp /boot/extlinux/extlinux.conf nvme0n1p3/extlinux/ 
# or
# $ sudo sed -i 's/mmcblk1p4/nvme0n1p4/g' nvme0n1p3/boot/extlinux/extlinux.conf
$ sudo umount nvme0n1p3
$ rm -rf nvme0n1p3/

extlinux.conf (1.4 KB)

now unplug the Micro-SD card, and change the Boot Mode to 1-bit QSPI Nor Flash:

Trigger the Reset key and wait for a moment, VF2 will boot from NVMe directly and it works well.

7. Enjoy it.

LivingLinux · April 15, 2023, 6:25pm

Thanks for step 3. I was surprised I wasn’t able to run parted and resize2fs.

omendez · April 23, 2023, 12:02am

Hi @larryw3i , thanks for setting up this tutorial. Unfortunately, my VF2 did not come back after changing SD card’s rootfs UUID. Upon reboot, /dev/nvme0n1p4 cannot be found:

Is there a way to recover from this, or am I simply back to square (-)1?

Thanks for your reply.

larryw3i · April 23, 2023, 1:40am

Is it the Debian Image?

larryw3i · April 23, 2023, 2:23am

If you’re using the Debian Image, here is the content of /boot/extlinux/extlinux.conf in my Micro-SD Card:

larry@visionfive2:~/Downloads$ cat /boot/extlinux/extlinux.conf                                                                                                                               
## /extlinux/extlinux.conf
##       
## IMPORTANT WARNING                                                                           
##                                    
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update
                                               
default l0                                                                                                                                                                                    
menu title U-Boot menu
prompt 0
timeout 50
                                               
label l0
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive
        fdtdir /dtbs
        append root=/dev/nvme0n1p4 root=/dev/nvme0n1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0

label l0r
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive (rescue target)
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive
        fdtdir /dtbs
        append root=/dev/nvme0n1p4 root=/dev/nvme0n1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single


label l1
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive
        fdtdir /dtbs
        append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0

label l1r
        menu label Debian GNU/Linux bookworm/sid 5.15.0-starfive (rescue target)
        linux /vmlinuz-5.15.0-starfive
        initrd /initrd.img-5.15.0-starfive
        fdtdir /dtbs
        append root=/dev/mmcblk1p4 root=/dev/mmcblk1p4 rw console=tty0 console=ttyS0,115200 earlycon rootwait stmmaceth=chain_mode:1 selinux=0 single

Pull the Micro-SD Card out and connect it to your computer via an adapter, after mount the /dev/sdX3( replace /dev/sdX by the actual device name of your Micro-SD card ), you can change the default l0 to default l1 in the-dir-you-mount-sd/extlinux/extlinux.conf, and it changes the booting device of VisionFive2, and you back to the Micro-SD Card booting, then, you can fix something about the NVME booting now. (^_^)

larryw3i · April 23, 2023, 2:28am

I haven’t tried any other images yet. If you use other images, wait for your good news!

omendez · April 23, 2023, 7:56pm

HI @larryw3i,
I can confirm the thing went south somehow with the extlinux directory, which is not there. /home/user is there (I can see the .cp from the step where we back it up into the .cp file), but the extlinux is gone

Anyways, I’ll try and reuse your .conf file, and see if I manage to recover it, otherwise, will start from step 1.

Thanks!

dominik · April 24, 2023, 11:03am

I thinkg that title of this post is misleading, it’s not booting from nvme only, but from sd (or emmc) with root
\ mounted from nvme after it was initialized by kernel. Of course this makes sd card read-only (meaning it’s used only to read and boot kernel) so it will not worn out quickly, but still something more is required than nvme.

larryw3i · June 4, 2023, 3:42am

Topic has been updated.

philrandal · June 4, 2023, 10:13am

Only one problem. Mine still doesn’t boot without the SD card. Not sure what’s going on and haven’t the time to look at the u-boot messages at the moment.

larryw3i · June 4, 2023, 12:31pm

Have you copied the image to SSD?

philrandal · June 4, 2023, 1:47pm

Yes, did all that. It might be some sort of timing issue with nvme. I’ll do it all from scratch tomorrow, in case there’s something stupid going on.

larryw3i · June 4, 2023, 2:01pm

Wait for your good news.

philrandal · June 4, 2023, 4:31pm

It’s all working now. Thanks.

larryw3i · June 4, 2023, 5:09pm

@philrandal BTW, does its HDMI work?

philrandal · June 4, 2023, 8:50pm

Works fine on an HD monitor. Haven’t tried it on a 4K display (yet)

kengreen · June 5, 2023, 7:04pm

I got as far as Step 4 in larry’s guide but my NVMe drive is not recognized. I get the following message in dmesg | grep -i nvme nvme nvme0: Device not ready aborting CSTS=0x0, nvme nvme0 remove after probe failure stats -19. I am trying to use a SanDisk A400 NVMe 512 GB drive from a Dell computer. It supports PCIe 3.0 but perhaps not PCIe 2.0. Can’t find any detailed spec sheet on this OEM drive even tried Dell Support.

nvme-cli list shows only dashes for all the information. lsblk -f shows only mmcblk1 partitions and mtdblock 1 and 2.

larryw3i · June 6, 2023, 1:13am

Hey, my friend @kengreen ,does the power your VF2’s charger supplies is greater than 15 watts? I remember my SSD also couldn’t be detected if I used the 5V & 3A charger when I was writing this topic.

kengreen · June 6, 2023, 5:10pm

my VF2 charger is Samsung super fast charger that puts out 45W, 20V @2.25A.

kengreen · June 7, 2023, 12:21pm

I got a little further on step 4. couldn’t scp to user@starfive.local:~/Downloads but did sudo scp to user@starfive.local and was able to dd to /dev/nvme0n1. it showed the progress of the copy. I did the tune2fs and modified the /boot/extlinux/extlinux.conf as you indicated. However, when I tried to reboot, it failed with a notice the /dev/nvme0n1p4 did not exist. Boot mode was set as you indicated.

Experience VisionFive v2 day01: BOOTING FROM NVME

Experience VisionFive v2 day01

1. Download the img and flash it to Micro-SD Card.

2. Connect to the network.

3. Extend the /dev/mmcblk1.

4. Now, let’s try booting from NVME.

5. Use the Debian-ports mirror.

6 Upgrade firmware to v3.0.4