First you need to rebuild your kernel to have appropriate eGPU modules. One guy I know tested nouveau with an old 600 series Nvidia GPU, but you may also try radeon/amdgpu.
Install RISC-V toolchain on your build machine first (Or build directly on VF2, you should omit the CROSS_COMPILE variables mentioned here then)
Get kernel sources from here: GitHub - starfive-tech/linux at JH7110_VisionFive2_devel
Use exactly the branch I put in the link, it’s the 5.15 kernel. Other versions are not yet ready.
Get config-5.15.0-starfive from your /boot partition, rename into .config in the kernel source tree.
Run make menuconfig CROSS_COMPILE=riscv64-linux-gnu-
Go into Device Drivers → Graphics → Set [M] in boxes mentioning GPUs you want to use (Nvidia, Radeon).
Run make bindeb-pkg CROSS_COMPILE=riscv64-linux-gnu-
This should have dropped linux-image-??.deb files in upper directory after compiling. DO NOT INSTALL THEM! The current way the VF2 image is built it will brick your install. You should instead unpack the deb package with an archiver, then replace vmlinuz-5.15.0-starfive file on your /boot partition. After that, copy lib/modules from the package to be /lib/modules in root.
The tricky part is mostly over. You now need to install Mesa packages for userspace GPU drivers, connect your GPU using an M.2 → PCIe riser and it should work.
There are some issues concerning the preinstalled distro, namely the PVR GPU driver conflicts in one way or another, so the guy who tested my approach simply replaced the Debian userland with Arch. Any further research is welcome.