Hi @tommythorn,
to the point of using the stock debian-ports, i think the commands were:
# mount /dev/nvme0n1p1 /mnt
# debootstrap --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg unstable /mnt https://deb.debian.org/debian-ports
As you see, just using debian-ports and the corresponding keyring does the trick.
The keyring is from the debian-ports-archive-keyring package. Hmm, you may need to fetch and install it from https://deb.debian.org/debian-ports, before. Please see e.g. this and the following posts to be aware of some minor troubles ahead.
Apparently the kernel is still fresh enough to run debian-unstable. I did not try to get the X-server going, only a command line debian, which does operate just as usual. You should have a terminal (screen) attached to the a serial line of the device when booting into the partition. This figure shows how to attach to the serial line.
Please feel free to ask if you get stuck somewhere.
I also managed to compile u-boot on the VF2 but did not yet dare to install it.