VisionFive2 hypervisor extension

What are all the RISC-V extensions, and versions, in the JH7110 SoC ?
U74-MC:
4x U74:
RV64IMAFDC_Zicsr_Zifencei_Zba_Zbb_Sscofpmf
or
rv64imafdczbb0p93_zba0p93_zicsr_zifencei_sscofpmf

1xS7:
RV64IMAC_Zicsr_Zifencei_Zba_Zbb_Sscofpmf
or
rv64imaczbb0p93_zba0p93_zicsr_zifencei_sscofpmf

1xE24:
RV32IMFC_???

Ah, yes, that’s familiar now. That seems to be regarded as an unimplemented feature the official kernel is sure to do Someday. It’s come and gone a few times in various branches, it seems.

Relevant (to this, not the OP) discussion:

3 Likes

But this would mean, the device tree has glitches in jh7110.dtsi:

cpu0: cpu@0 {
     compatible = "sifive,u74-mc", "riscv";
     riscv,isa = "rv64imac";   // << fd? extensions?
};

cpu1: cpu@1 {
     compatible = "sifive,u74-mc", "riscv";
     riscv,isa = "rv64imafdc"; // << extensions?
};
1 Like

U74-MC consists of:

cpu0 a S7  RV64IMAC
cpu1 a U74 RV64IMAFDC
cpu2 a U74 RV64IMAFDC
cpu3 a U74 RV64IMAFDC
cpu4 a U74 RV64IMAFDC

EDIT: The JH7110 has (S7->S76):

cpu0 a S76 RV64IMAFDC ???!!! instead of RV64IMAC ???!?!?!?
cpu1 a U74 RV64IMAFDC
cpu2 a U74 RV64IMAFDC
cpu3 a U74 RV64IMAFDC
cpu4 a U74 RV64IMAFDC

And accessed from the U74MC4+U76 across frontport0 axi4 @ 128bit 750MHz bus is a E24 RV32IMFC.

4 Likes
starfive:/home # uname -a
Linux starfive.lan 6.3.0-rc4-26-default #1 SMP Sun Mar 26 22:01:16 UTC 2023 (f77c350) riscv64 riscv64 riscv64 GNU/Linux
starfive:/home # cat /proc/cpuinfo
processor	: 0
hart		: 1
isa		: rv64imafdc_zbb
mmu		: sv39
uarch		: sifive,u74-mc
mvendorid	: 0x489
marchid		: 0x8000000000000007
mimpid		: 0x4210427

processor	: 1
hart		: 2
isa		: rv64imafdc_zbb
mmu		: sv39
uarch		: sifive,u74-mc
mvendorid	: 0x489
marchid		: 0x8000000000000007
mimpid		: 0x4210427

processor	: 2
hart		: 3
isa		: rv64imafdc_zbb
mmu		: sv39
uarch		: sifive,u74-mc
mvendorid	: 0x489
marchid		: 0x8000000000000007
mimpid		: 0x4210427

processor	: 3
hart		: 4
isa		: rv64imafdc_zbb
mmu		: sv39
uarch		: sifive,u74-mc
mvendorid	: 0x489
marchid		: 0x8000000000000007
mimpid		: 0x4210427

starfive:/home # dmesg | grep kvm
[  182.600501] kvm [1718]: hypervisor extension not available
1 Like

No zba, Just zbb? Or that means just zbb that is [currently] enabled in kernel 6.3?

2 Likes

Is this stock (ie. Linus’ upstream tree) or did you apply additional patches?

I really need this for perf, but the biggest problem is that OpenSBI needs the PMU entries in the device tree it sees (which isn’t the same that Linux gets alas)

1 Like

I would have expected a kernel to report back exactly whatever it was told to report back from the dts files (regardless if it is accurate or wrong).
For the visionfive2_upstream kernel I would have expected rv64imac_zba_zbb for the S7 hart (which might be a S76 hart) and rv64imafdc_zba_zbb for the 4 U74 hart’s.

And in the U-boot upstream patch it has rv64imacu for the S76(cpu@0) and rv64imafdcbsu for the U74 harts (cpu@1,cpu@2,cpu@3,cpu@4) all of which are part of the u74-mc.

I’m now wondering is it a cut S76 core with no floating point , i.e. with less features than a baseline S76 core (rv64imafdc_zba_zbb), with support for single and double precision floating point removed and bit Manipulation it would become rv64imac. A S76 core only has machine mode and by default a user mode, it has no supervisor mode, so it is different than the U74 cores anyhow.

I do not know if it is possible, but what would be fantastic is if the exact same features were listed in all the dts files for each of the six cores in the JH7110 SoC E24,U74-mc(S76+4xU74)

4 Likes

@mzs; Unfortunately I dont think we should expect anything more than a stock S7. :cry:

from this:
[v6,00/21] Basic clock, reset & device tree support for StarFive JH7110 RISC-V SoC - Patchwork

...
[Device tree]
Patch 6:
- Changed the label "S76_0" to "S7_0" and used compatible "sifive,s7"
  for core 0.
- Updated ISA of each cores. (by Conor)
...

Also note these patches from Starfive; they do not refer to it as a S76

[v6,18/21] dt-bindings: riscv: Add SiFive S7 compatible - Patchwork

[v6,19/21] riscv: dts: starfive: Add initial StarFive JH7110 device tree - Patchwork

1 Like

An S76 is technically a S7-series RISC-V core, just like a U74 is technically a U7-series core. And I see S76 mentioned in more and more places with regard to the U74-mc.

Travelling all the way back in time to 2018-08-22, there was no option to license a S7 Core from SiFive, but there was a S7 Series S76 Core.

So I do still think that it is really an S76 core, without any form of floating point support (FD), without support for Bit Manipulation specifically not Zba and not Zbb (B), with a smaller cache L1 I-Cache, with a smaller Data TIM instead of a Data cache and without a Instruction TIM (basically remove everything so that much less space on the silicon die, and less power, is required). And unlike the U74 (Machine,Supervisor,User mode) it does not have a Supervisory mode just like a S76 Core.

EDIT:
I found CSR MISA information about the 5 hart’s here:
Info : hart 0: XLEN=64, misa=0x8000000000901107
Info : hart 1: XLEN=64, misa=0x800000000094112f
Info : hart 2: XLEN=64, misa=0x800000000094112f
Info : hart 3: XLEN=64, misa=0x800000000094112f
Info : hart 4: XLEN=64, misa=0x800000000094112f

Converting to binary and consulting the “Encoding of Extensions field in misa” table in the riscv-privileged-20211203.pdf gives the following

    6666555555555544444444443333333333222222222211111111110000000000   
    3210987654321098765432109876543210987654321098765432109876543210
--------------------------------------------------------------------
S76 1000000000000000000000000000000000000000100100000001000100000111
U74 1000000000000000000000000000000000000000100101000001000100101111                                        
U74 1000000000000000000000000000000000000000100101000001000100101111                                        
U74 1000000000000000000000000000000000000000100101000001000100101111                                        
U74 1000000000000000000000000000000000000000100101000001000100101111
                                        
bit 63=1 ==> MXL (Machine XLEN) the native base integer ISA is 64-bit
bit 23=1 ==> X: Non-standard extensions present
bit 20=1 ==> U: User mode implemented
bit 18=1 ==> S: Supervisor mode implemented
bit 12=1 ==> M: Integer Multiply/Divide extension
bit 08=1 ==> I: RV32I/64I/128I base ISA
bit 05=1 ==> F: Single-precision floating-point extension
bit 03=1 ==> D: Double-precision floating-point extension
bit 02=1 ==> C: Compressed extension
bit 01=1 ==> B: Tentatively reserved for Bit-Manipulation extension
bit 00=1 ==> A: Atomic extension

From the above the letters are (but in a illegal format):
1x S76 is a RV64IABCMUX
4x U74 is a RV64IABCDFMSUX

So more precisely it is:
1x S76 rv64imacux_Zba_Zbb
4x U74 rv64imafdcsux_Zba_Zbb

And even more precisely:
F implies Zicsr, but it may be present even if F is not (e.g. The Core IP had optional F support)
G implies IMAFDZicsr_Zifencei
1x S76 rv64imacuxZba_Zbb_Zicsr
4x U74 rv64imafdcsuxZba_Zbb_Zifencei_Zicsr
EDIT: I now suspect this, but i could be wrong:
1x S76 rv64imacuxZicsr_Zifencei_Zba_Zbb_Sscofpmf
4× U74 rv64imafdcsuxZicsr_Zifencei_Zba_Zbb_Sscofpmf

and at a guess the E24 co-processor is probably:
1x E24 rv32imfcuxZicsr

I guess it is time to start trying to figure out how to get more details on the E24.

3 Likes

from the upstream tree.

1 Like

To be clear, are you refering to GitHub - starfive-tech/linux at JH7110_VisionFive2_upstream ?

u74-mc
S7_0 cpu@0 rv64imac_zba_zbb
U74_1 cpu@1 rv64imafdc_zba_zbb
U74_2 cpu@2 rv64imafdc_zba_zbb
U74_3 cpu@3 rv64imafdc_zba_zbb
U74_4 cpu@4 rv64imafdc_zba_zbb

E24 is no longer listed in JH7110_VisionFive2_upstream. It was removed from the dts files, but then again it is a difficult CPU to make good use of, it is 32-bit, asynchronous and without a MMU.

2 Likes

Humm, bummer. Actually it’s one of the things that attracted me to the board.

I want to run a real-time machine control application on the E24 that runs independently of the OS etc, My goal is to run three stepper motors and a laser.

Spoiler

… See the Pi3 in there (it has a 3.5’ screen)… Eventually the VF2 replaces that.

I was going to build another custom ESP32 controller for this (FluidNC) but thought it could be neat to drive the controller directly without needing the ESP32.

The E24 is nearly ideal for this, and no MMU is no problem, no serious maths to do. It was labelled as a ‘realtime processor’ in some of the JH7110 literature.

Hopefully it can still be used (or added to the dts as needed).

1 Like

AFAIK, only software emulated is there because hw lacks it. You might be interested in getting in touch: GitHub - dramforever/opensbi-h: WIP: A fork of OpenSBI, with software-emulated hypervisor extension support

3 Likes

https://starfivetech.com/uploads/u74mc_core_complex_manual_21G1.pdf has all you need to know. I just want to stress:

  1. No H extension on either S7 or U7
  2. Apart from lacking floating point in S7, the major difference is that S7 only has 2 modes, M/U. U7 has all 3 modes, M/S/U. Thus, S7 is not well suited for general Linux use. I’m think of using it for profiling other U7 cores through IPI, making it a little bit useful.
  3. For E24, it’s not even mentioned in dts. I had this post earlier about JH7100 (JH7100: Make use of E24?), and recently I asked starfive again about E24 on JH7110. In short, they are not ready to open source the Linux driver code that could load a ELF binary to E24, even though they have testing instructions: soft_3rdpart/e24_test.docx at JH7110_VisionFive2_devel · starfive-tech/soft_3rdpart · GitHub (In Chinese)
1 Like

After scanning JTAG chain in JH7110, I found 2 taps, one is u74-mc, another maybe e24, but connection to that would fail, maybe e24 needs to be enabled by register or something.

The problem might be that the E24 has no Boot Vector, the boot vector for the E24 is defined by the U74MC.

I don’t think so… JTAG debug has higher priority than program.

Reading page 134 of “SiFive E24 Core Complex Manual 21G1.01.00” (not sure if it is the right revision of document for the E24 in the JH7110), another possibility might be that the debugger might need to send a halt so that the E24 will wait running in a loop (entry_loop) for debugger instructions.

Table 98 “Debug Module Memory Map from the Perspective of the Core” from that page:

TL Address Name Attr. Description
0x800‑0xFFF ROM RO Debug interrupt or EBREAK enters at 0x800, saves s0,writes hartid to HALTED, then busy-waits for FLAGS[hartid] > 0. If FLAGS[hartid].go, write 0 to GOING, then jump to WHERETO. Else write hartid to RESUMING, then execute dret to return to user program. ROM Source Code: https://github.com/chipsalliance/rocket-chip/blob/master/scripts/debug_rom/debug_rom.S