Re: failed to assign iommu device!


Geoffroy Van Cutsem
 

Do you see the ACRN shell prompt (ACRN:\>) if you press “Enter” at that point? I do on my setup and I can then switch to the Service VM console by typing ‘vm_console 0’ (press Ctrl+spacebar from there if you wish to the return to the ACRN shell).

 

I see the same error messages than you but ACRN is running. My Service VM kernel is not booting to completion though, it hangs at some point, then crashes and reboot. But I do see additional error messages from ACRN (related to iommu), specifically: “iommu: DMAR OP Timeout! @ dmar_issue_qi_request”

 

Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 2:53 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks Nanlin. This problem is resolved.

 

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

===

 

 

On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>

Join acrn-users@lists.projectacrn.org to automatically receive all group messages.