Debian - Move root partition to NVMe and keep boot partition

_eric · January 3, 2023, 12:39pm

Hey, is there a way to move the “/” ext4 partition to my NVMe?

That it boots from the microsd but uses the system on the much faster NVMe?

Thanks,

dtometzki · January 3, 2023, 1:00pm

I am unsure which boot setting for nvme is the correct one ?
And what i see in u-boot isnt NVME configured.

_eric · January 3, 2023, 1:51pm

@dtometzki Yes NVMe boot is currently not supported, I think. I plan to boot like @Manux described here: Debian NVME boot I’ll write again if I’m successful.

_eric · January 3, 2023, 1:52pm

It seems like the Debian Image doesn’t use GRUB. I’m not that much experienced with Linux and booting in general.

dtometzki · January 3, 2023, 3:25pm

Hello @_eric,

the board is using u-boot and opensbi.

Manux · January 3, 2023, 8:05pm

Hi eric,

It does use uboot. There sould be a config somwhere for it. Basicly its the similar to grub and does just name the root partition or UUID of it. If you change this to the nvme then it should work.

dtometzki · January 3, 2023, 8:43pm

Hello @_eric and @Manux,

the information you will find in /boot/extlinux/extlinux.conf
there are some issues with the boot partion all files located under /boot/boot.
I could resolve the issue with a new uboot patch.

I work for it to move the partion without boot partion.

Best regards
Damian

mzs · January 3, 2023, 9:28pm

Although not strictly necessary there is a tool for reading and writing U-boot environmental variables from within the Linux OS ( https://github.com/sbabic/libubootenv or “sudo apt install libubootenv-tool” for a binary package from 2021-12-14 ). But obviously this carries the risk of breaking your U-boot environment and if used too often it could rapidly consume the limited write lifetime of the 16 MiB flash chip which is soldered to the board.

It may need to be patched for the VF2 board, or a customised /etc/fw_env.config file created specifically for a VF2 board. I have not tested it (no board yet).

EDIT: Might be best to ignore my above suggestion for now, I’ve submitted an issue on github for something I noticed when trying to workout the details of the boot process.

Manux · January 6, 2023, 2:22pm

Hi Damian,

Thanks this would be very helpful i think to make things clearer.

As you posted a screenshot, i see that this is the correct configuration to have your root partiton in the nvm. Is this correct ?

I do not see the /boot/boot folder in action. Is this a pached version or did you just manually put the kernel into the /(boot) partition?

Thanks!

dtometzki · January 6, 2023, 2:53pm

Hi Manux,

i patched u-boot to resolve the issue with /boot/boot.
Second i moved the boot partition from vfat to ext4. because i couldnt install a new kernel with more wifi modules.
At the moment is my configuration so:
/boot → on SD Card
/ → on nvme

Manux · January 6, 2023, 2:58pm

Cool!
Could you make a shore writeup/turoial on how to patch and get this set up ? would you publish your patch ? What soure of uboot did you use the newest one or the one for the 55 image ?

Thanks a lot!
Best M

dtometzki · January 6, 2023, 3:43pm

Hello @Manux,

i will write an short tutorial what i did. Next week this weekend i have to work

mzs · January 6, 2023, 4:35pm

I see that as a move that may potentially need to be changed back later on.

There is a reason that the boot partition is typically FAT32 and that is because you want it to be accessible from the SPL-U-Boot which at that stage in the boot process is fully running inside the CPU’s 2MiB L2 cache which is being used as SRAM (part of the SPL’s function is to configure clocks and controllers for access to the main external 2/4/8 GiB DDR Memory - outside of the SoC). I’m not saying that ext4 can’t be used because it can, but if it is not used at that stage in the boot process then that can potentially be used to free up additional memory inside the L2 SRAM that might be used to access other hardware to then load the main U-Boot into main memory from a FAT32 partition. But if I look at everything that is in the SPL, there is probably other things that can be removed first.

dtometzki · January 6, 2023, 4:51pm

Hello mzs,

I have to move from vfat to ext4 because of installation of an new kernel wasnt possible.

_eric · January 6, 2023, 4:54pm

A tutorial would be great!

Manux · January 7, 2023, 12:23pm

im curios what did it not work with vfat ? What errors did you get ? Was the kernel to big ?

dtometzki · January 7, 2023, 1:37pm

Hi @Manux

that is my issue with vfat on boot partition:

dpkg: error processing archive /home/damian/data/starfive/new/linux-image-5.15.0-starfive_5.15.0-starfive-8_riscv64.deb (–unpack):
unable to make backup link of ‘./boot/config-5.15.0-starfive’ before installing new version: Operation not permitted
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

symlinks doesnt work with vfat

Damian

Manux · January 8, 2023, 11:34am

Hi Damian,

This is because vfat does not support symlinks. However you could just und the deb archiv by hand hand move the files to the boot partition. (Make a backup of the boot partition)
After copying the stuff into the dirs. Change the extlinux config to point to the new kernel.

dtometzki · January 8, 2023, 1:18pm

Hi Manux,

yes this works too. But for me is it an workaround and isnt the standard

cobalt · February 4, 2023, 4:45pm

Hi @_eric,

Yes, and it is quite straight forward, really. Mainly, all i did, was:

plugin, partition and format the NVMe. (mine is a WD Blue SN570, 500GB).
mount and debootstrap a Debian root file system onto it.
did a minimum on standard configuration (see below).
entered the new partition in /boot/boot/extlinux.conf.

For the configuration of the new file system:

chroot into it
set a root passwd so that you can log in later.
apt-get and configure locales and ntpsec.
apt-get openssh
apt-get install debian-keyring, debian-ports-archive-keyring
leave the chroot
copy and adjust /etc/fstab
copy and adjust /etc/network/interfaces
copy /root/.ssh to preserve the authorized_keys

Then reboot with a terminal attached to be able to select the new partition during the boot dialog. When everything comes out fine, set the default in extlinux.conf.

During that, I ran in to the following issues / made the following choices:

i use the stock debian-ports, not the snapshot.
one has some fun with the keyrings during debootstrap and later.
i found the interfaces renamed from ‘eth*’ to ‘end*’ and needed to activate the iface line in /etc/network/interfaces to get the dhcp requested.
/dev/ttyUSB0 and screen were sometimes dropping characters (wrong baud rate somewhere?). No issue once the ssh connection is going.

Later, i saw some NVMe I/O timeouts, but did not take care of them, yet.

Hmm, to mention a very silly problem, too, i had no suiting screw to fix the NVMe. The Quick Start Guide could be improved mentioning screw or thread spec. If you know the right diameter, please drop a line. For now, i fixed the device with a piece of tesa strip.

As good as i understand the boot process so far, the extlinux layer can be skipped, as the u-boot is placed in an mtd device, but might need some fixes (u-boot env, /boot/boot, …). So better preserve the SD until then. I don’t think, the two extra partitions in the beginning of the current image are actually really needed. Also no need to adjust the boot switches, which to my current understanding are mainly needed for other purposes.