Add support for qemu virtual machines using KVM#423
Conversation
IkerGalardi
force-pushed
the
kvm-support
branch
4 times, most recently
from
fe30316 to
f517242
Compare
|
@IkerGalardi you should include the full logs for the loader, it's not clear to me that the loader is working properly. There is most likely an issue with the initial virtual address space that is setup by the loader. I also would imagine that the |
|
Here are the full logs: The issue could be the page tables, but still, the kernel entry being at About the |
On non-HYP, the kernel uses the upper address range for its page table configured by |
IkerGalardi
force-pushed
the
kvm-support
branch
2 times, most recently
from
ed2145c to
c01bd3e
Compare
|
Makes sense. I've seen that the page tables used in the upper VA range are set in lvl0 descriptors are completelly set to 0, so it makes sense that the there is a instruction abort. (gdb) p /x boot_lvl0_upper
$5 = {0x0 <repeats 512 times>}
(gdb) p /x boot_lvl1_upper
$6 = {0x0, 0x7000a003, 0x0 <repeats 510 times>}
(gdb) p /x boot_lvl2_upper
$7 = {0x40000711, 0x40200711, 0x40400711, 0x40600711, 0x40800711, 0x40a00711, 0x40c00711, 0x40e00711, 0x41000711, 0x41200711, 0x41400711,
0x41600711, 0x41800711, 0x41a00711, 0x41c00711, 0x41e00711, 0x42000711, 0x42200711, 0x42400711, 0x42600711, 0x42800711, 0x42a00711,
0x42c00711, 0x42e00711, 0x43000711, 0x43200711, 0x43400711, 0x43600711, 0x43800711, 0x43a00711, 0x43c00711, 0x43e00711, 0x44000711,
0x44200711, 0x44400711, 0x44600711, 0x44800711, 0x44a00711, 0x44c00711, 0x44e00711, 0x45000711, 0x45200711, 0x45400711, 0x45600711,
0x45800711, 0x45a00711, 0x45c00711, 0x45e00711, 0x46000711, 0x46200711, 0x46400711, 0x46600711, 0x46800711, 0x46a00711, 0x46c00711,
0x46e00711, 0x47000711, 0x47200711, 0x47400711, 0x47600711, 0x47800711, 0x47a00711, 0x47c00711, 0x47e00711, 0x48000711, 0x48200711,
0x48400711, 0x48600711, 0x48800711, 0x48a00711, 0x48c00711, 0x48e00711, 0x49000711, 0x49200711, 0x49400711, 0x49600711, 0x49800711,
0x49a00711, 0x49c00711, 0x49e00711, 0x4a000711, 0x4a200711, 0x4a400711, 0x4a600711, 0x4a800711, 0x4aa00711, 0x4ac00711, 0x4ae00711,
0x4b000711, 0x4b200711, 0x4b400711, 0x4b600711, 0x4b800711, 0x4ba00711, 0x4bc00711, 0x4be00711, 0x4c000711, 0x4c200711, 0x4c400711,
0x4c600711, 0x4c800711, 0x4ca00711, 0x4cc00711, 0x4ce00711, 0x4d000711, 0x4d200711, 0x4d400711, 0x4d600711, 0x4d800711, 0x4da00711,
0x4dc00711, 0x4de00711, 0x4e000711, 0x4e200711, 0x4e400711, 0x4e600711, 0x4e800711, 0x4ea00711, 0x4ec00711, 0x4ee00711, 0x4f000711,
0x4f200711, 0x4f400711, 0x4f600711, 0x4f800711, 0x4fa00711, 0x4fc00711, 0x4fe00711, 0x50000711, 0x50200711, 0x50400711, 0x50600711,
0x50800711, 0x50a00711, 0x50c00711, 0x50e00711, 0x51000711, 0x51200711, 0x51400711, 0x51600711, 0x51800711, 0x51a00711, 0x51c00711,
0x51e00711, 0x52000711, 0x52200711, 0x52400711, 0x52600711, 0x52800711, 0x52a00711, 0x52c00711, 0x52e00711, 0x53000711, 0x53200711,
0x53400711, 0x53600711, 0x53800711, 0x53a00711, 0x53c00711, 0x53e00711, 0x54000711, 0x54200711, 0x54400711, 0x54600711, 0x54800711,
0x54a00711, 0x54c00711, 0x54e00711, 0x55000711, 0x55200711, 0x55400711, 0x55600711, 0x55800711, 0x55a00711, 0x55c00711, 0x55e00711,
0x56000711, 0x56200711, 0x56400711, 0x56600711, 0x56800711, 0x56a00711, 0x56c00711, 0x56e00711, 0x57000711, 0x57200711, 0x57400711,
0x57600711, 0x57800711, 0x57a00711, 0x57c00711, 0x57e00711, 0x58000711, 0x58200711, 0x58400711, 0x58600711, 0x58800711, 0x58a00711,
0x58c00711, 0x58e00711...}The kernel has its entry in the My test setup fails with a data fault on the serial transmit virtualizer, but this might be unrelated to the tool. I'll try to fix the table generator in the |
|
The EL1 page table generation works on my webserver example. But still faults on the Serial transmission virtualizer. Once I figure out whats going on there I'll mark this PR ready for review. |
This board differs from the standard qemu_virt_aarch64 board in that the entry point is in EL2 instead of EL1. This is necessary due to the lack of nested virtualization support on the KVM subsystem. Signed-off-by: IkerGalardi <contacto.ikergalardi@gmail.com>
Previously the defaults where applied ALWAYS, meaning that if some board tried to specialize a kernel building parameter that the default configuration did, it would get overwriten by the default configuration. This patch applies the specialization to the default configuration instead of doing it the other way around. Signed-off-by: IkerGalardi <contacto.ikergalardi@gmail.com>
Signed-off-by: IkerGalardi <contacto.ikergalardi@gmail.com>
|
I've tested this branch on the webserver and everything is working as expected. I've found an issue with virtio, but I assume its related to sDDF rather than microkit and the kvm platform. |
|
|
||
| for (uint32_t i = 0; i <= it_lines_number; i++) { | ||
| *((volatile uint32_t *)(GICD_BASE + 0x80 + (i * 4))) = 0xFFFFFFFF; | ||
| if (is_set(CONFIG_ARM_HYPERVISOR_SUPPORT)) { |
There was a problem hiding this comment.
Rather than this being done based on whether or not seL4 has hyp support, it should be done based on the current exception level, as that is what really matters.
There was a problem hiding this comment.
Also, for the sake of git blame: I'd rather do an early return, than to move this all into the if statement.
There was a problem hiding this comment.
Done!
I currently do not have any project to test it. But I guess that configuring the interrupt priority first does not break anything on systems running on EL2.
There was a problem hiding this comment.
Just tested it and it works.
There was a problem hiding this comment.
Hi: I'll look further tomorrow, but you'll want to reference the ARM GIC manual as to what needs to be done when, as in when can you write to certain registers. I do recall their are restrictions, but not what they are or what they apply to.
There was a problem hiding this comment.
Yeah, seL4 does already set the interrupt priority to 0x80.
The reason why this code did that is that we wanted to configure it while in EL3 (secure mode), which allows us to actually configure the GICC_PMR:
It will "work" but it also ignores it. If we're in EL1 the write to that won't do anything, and would later be done by seL4 later anyway.
So this should be after the if-check anyway.
There was a problem hiding this comment.
If I understand correctly, that code needs to run only if the loader is running in EL2, right? So the configure_gicv2 function is kind of useless in the KVM case.
I'll move the priority setting to the end of the function (as it was before), but I feel that this check could be done at compile time.
There was a problem hiding this comment.
It's more that on the ZCU** platforms where we start in EL3, and the QEMU platform where nobody starts before us, we are the onee who have to configure it. Most other platforms have UBoot configure it. When not in EL2 we don't configure it and hope it works.
The reason why this shouldn't be a compile time check is that if you want to run an EL1 seL4 image on the zcu** then you still need to do this set up before we drop to EL1. Note that configure_gicv2 is the absolute first thing we do.
IkerGalardi
force-pushed
the
kvm-support
branch
4 times, most recently
from
a619d3d to
4ca50bd
Compare
EL1 software can not access interrupt group registers on the GIC distributor, so the function does an early return if the loader is running in EL1. Signed-off-by: IkerGalardi <contacto.ikergalardi@gmail.com>
| }, | ||
| ), | ||
| BoardInfo( | ||
| name="qemu_virt_aarch64_kvm", |
There was a problem hiding this comment.
Ok, now that all the code changes are done: this should be called probably "_el1" or otherwise "non_hyp"; it's not for KVM more just a seeming limitation of KVM that it can't emulate EL2.
And you should add this board to the manual.md.
4 participants
This PR adds support for running microkit based operating systems using qemu with KVM enabled.
The main issue with the qemu platform with KVM enabled is that it drops the kernel in EL1 instead of the expected EL2. This PR builds a custom kernel for this platform with hypervisor support disabled.
The loader seems to work fine, but jumping to the kernel causes a instruction abort exception. The next is the log: