Date   

user VM Management from SOS VM

yogesh@...
 

Hi All,
    I have ubuntu 18.04 Service OS VM with ACRN hypervisor installed. I have generated Ubuntu UOS VM and have launched Ubuntu 20.04 UOS VM. 
Unfortunately at this time I dont have second monitor to assign to uos vm nor having serial cable to connect to console of uos. 
     I have tried to installed openssh-server into uos vm image and assigned ethernet controller in launcher script through Web interface(app.py). 
But unfortunately I could not able to see its IP address. 
# virsh net-list
# virsh net-info default
# virsh net-dhcp-leases default

sudo acrnctl list
[sudo] password for yogesh:
post_vm_id1             started

Logs: 
 bash ./launch_uos_id1_console.sh
cpu3 online=1
acrn-br0 bridge aleady exists, adding new tap device to it...
logger: name=console, level=4
logger: name=kmsg, level=3
logger: name=disk, level=5
SW_LOAD: get ovmf path /usr/share/acrn/bios/OVMF.fd, size 0x200000
vm_create: post_vm_id1
VHM api version 1.0
vm_setup_memory: size=0x100000000
open hugetlbfs file /run/hugepage/acrn/huge_lv1/post_vm_id1/D279543825D611E8864ECB7A18B34643
open hugetlbfs file /run/hugepage/acrn/huge_lv2/post_vm_id1/D279543825D611E8864ECB7A18B34643
level 0 free/need pages:0/1 page size:0x200000
level 1 free/need pages:12/4 page size:0x40000000
to reserve more free pages:
to reserve pages (+orig 0): echo 1 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
now enough free pages are reserved!
 
try to setup hugepage with:
        level 0 - lowmem 0x0, biosmem 0x200000, highmem 0x0
        level 1 - lowmem 0x80000000, biosmem 0x0, highmem 0x80000000
total_size 0x200000000
 
mmap ptr 0x0x7fe3d23f7000 -> baseaddr 0x0x7fe400000000
mmap 0x80000000@0x7fe400000000
touch 2 pages with pagesz 0x40000000
mmap 0x80000000@0x7fe540000000
touch 2 pages with pagesz 0x40000000
mmap 0x200000@0x7fe4ffe00000
touch 1 pages with pagesz 0x200000
 
really setup hugepage with:
        level 0 - lowmem 0x0, biosmem 0x200000, highmem 0x0
        level 1 - lowmem 0x80000000, biosmem 0x0, highmem 0x80000000
vm_init_vdevs
No pm notify channel given
Listening 0...
polling 37...
pci init passthru
pci init virtio-blk
pci init virtio-net
pci init igd-lpc
tpm: init_vtpm2:Invalid socket path!
 acrn_sw_load
 SW_LOAD: partition blob /usr/share/acrn/bios/OVMF.fd size 0x200000 copied to addr 0x7fe4ffe00000
 SW_LOAD: build e820 5 entries to addr: 0x7fe4000ef008
SW_LOAD: entry[0]: addr 0x0000000000000000, size 0x00000000000a0000,  type 0x1
SW_LOAD: entry[1]: addr 0x0000000000100000, size 0x000000007befc000,  type 0x1
SW_LOAD: entry[2]: addr 0x000000007bffc000, size 0x0000000004004000,  type 0x2
SW_LOAD: entry[3]: addr 0x00000000e0000000, size 0x0000000020000000,  type 0x2
SW_LOAD: entry[4]: addr 0x0000000140000000, size 0x0000000080000000,  type 0x1
SW_LOAD: ovmf_entry 0xfffffff0
 add_cpu
out instr on NMI port (0x61) not supported
out instr on NMI port (0x61) not supported
out instr on NMI port (0x61) not supported
Unhandled memory access to 0xfe001818
Failed to emulate instruction [mmio address 0xfe001818, size 4Unhandled ps2 mouse command 0xe1
Unhandled ps2 mouse command 0x0a
 Unhandled ps2 mouse command 0x01
 Unhandled ps2 mouse command 0x41
 Unhandled ps2 mouse command 0x88
 Connected:38
   Disconnect(38)!
 
Could you please let me know what could be the other ways by which we can know messages or status of uos VM. 

Thanks,
Yogesh


Invitation Reminder - ACRN Virtual Meet-Up Europe 2021

Geoffroy Van Cutsem
 

Dear ACRN community,

The ACRN team is hosting a virtual Meet-Up, brought to you by Intel. Please join us to learn more about ACRN, how far it has come over the last three years, and where it is headed. The Meet-Up is intended for attendees with a technical profile (but no prior knowledge on ACRN is required).

Dates: 25th and 26th of May 2021
Time: 9AM to 1PM (Central European Time)

Here is the link to register for this event: https://plan.seek.intel.com/ACRN-Meet-Up-Europe-REG

Agenda Day 1
===========
9:00 AM: Welcome
9:10 AM: ACRN Introduction and Architecture
9:50 AM: Real-Time Management and Performance Optimization
10:30 AM: Functional Safety Design and Certification Plan
11:10 AM: Break
11:30 AM: Installation and Configuration Introduction
12:10 PM: Enabling ACRN on new Hardware Platforms
12:50 PM: Conclusions
1:00 PM: End of Day 1

Agenda Day 2
===========
9:00 AM: Welcome
9:10 AM: Orchestration with Libvirt and Kata Container
9:50 AM: Boot process and secure boot
10:30 AM: Community & Development Model
11:10 AM: Break
11:30 AM: Debugging ACRN (inc. tracing & logging)
12:10 PM: Inter-VM communication Introduction
12:50 PM: Conclusions
1:00 PM: End

We look forward to seeing you there!

Best regards,
Geoffroy

Senior Technical Marketing Engineer
IAGS - System Software Products
Tel: +32 (0)3 450 0851

-----------------------------------------------
Intel Corporation NV/SA
Kings Square, Veldkant 31
2550 Kontich
RPM (Bruxelles) 0415.497.718.
Citibank, Brussels, account 570/1031255/09


Re: failed to assign iommu device!

Liu, Fuzhong
 

Hi Kishore

One question:

Do you change  any kernel command of HOST OS between OK and KO: Service VM is booting

Service VM is booting for me.

Very few times, will get following error:

ACRN:\>[2971433us][cpu=0][(null)][sev=2][seq=19]:Assertion failed in file arch/x86/vtd.c,line 256 : fatal error

 

Is  intel_iommu=on  in your HOST OS kernel cmdline for this Assert? If yes, please remove it.

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Wednesday, May 12, 2021 5:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Geoffroy,

    Thanks. This helps a lot.

    I saw ACRN shell prompt. But, did not bother to check VMs and boot service VM till I was your message.

    Service VM is booting for me.

 

    Very few times, when I select ACRN hypervisor in GRUB, I get following error message. When this error occurs, Service VM is not listed and thus, vm_console 0 gives invalid parameter error

 

Loading ACRN hypervisor with SDC scenario ... error: Can't get controller info.. Multiboot Bootloader: GRUB 2.02-2ubuntu8.23 [0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used! ACRN Hypervisor calibrate_tsc, tsc_khz=2591719 [2639579us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) ***@***.*** build by kishoreacrn, start time 2628205us [2640883us][cpu=0][(null)][sev=2][seq=3]:API version 1.0 [2641301us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz ACRN:\>[2971433us][cpu=0][(null)][sev=2][seq=19]:Assertion failed in file arch/x86/vtd.c,line 256 : fatal error [2972631us][cpu=0][(null)][sev=2][seq=20]: delta = (actual_load_address - CONFIG_HV_RAM_START) = 0x0 [2973735us][cpu=0][(null)][sev=2][seq=21]: Host Stack: CPU_ID = 0 [2974471us][cpu=0][(null)][sev=2][seq=22]:addr(0x1120aad0) 0x0000000000000000 0x00000000007d9bdc 0x000000001305d040 0x00000001ca882b8c [2975924us][cpu=0][(null)][sev=2][seq=23]:addr(0x1120aaf0) 0x000000001120ab30 0x000000001100c976 0x0000000011051160 0x000000001305d040 [2977368us][cpu=0][(null)][sev=2][seq=24]:addr(0x1120ab10) 0x0000000000000000 0x000000001100c880 0x000000001103d134 0x0000000000000000 [2978814us][cpu=0][(null)][sev=2][seq=25]:addr(0x1120ab30) 0x000000001120ab60 0x000000001100c362 0x000000001306d000 0x000000001306d000 [2980256us][cpu=0][(null)][sev=2][seq=26]:addr(0x1120ab50) 0x0000000011051160 0x000000001306d000 0x000000001120abf0 0x000000001101646e [2981701us][cpu=0][(null)][sev=2][seq=27]:addr(0x1120ab70) 0x0000000000000000 0x0000000011051160 0x000000000000000f 0x000000001120ac00 [2983148us][cpu=0][(null)][sev=2][seq=28]:addr(0x1120ab90) 0x000000000000000d 0x0000000000000000 0x000000017e300000 0x0000000000000000 [2984589us][cpu=0][(null)][sev=2][seq=29]:addr(0x1120abb0) 0x00000000110450c7 0x839118c584437d48 0x0000000000000000 0x0000000000000000 [2986034us][cpu=0][(null)][sev=2][seq=30]:addr(0x1120abd0) 0x0000000011051160 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2987358us][cpu=0][(null)][sev=2][seq=31]:addr(0x1120abf0) 0x000000001120ac30 0x000000001101701c 0x0000000000000000 0x839118c584437d48 [2988404us][cpu=0][(null)][sev=2][seq=32]:addr(0x1120ac10) 0x000000001120ac50 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2989448us][cpu=0][(null)][sev=2][seq=33]:addr(0x1120ac30) 0x000000001120ac50 0x0000000011017128 0x0000000000000000 0x0000000000000000 [2990490us][cpu=0][(null)][sev=2][seq=34]:addr(0x1120ac50) 0x000000001120ac70 0x00000000110081e8 0x00000000110010e2 0x0000000000000000 [2991538us][cpu=0][(null)][sev=2][seq=35]:addr(0x1120ac70) 0x00000000110548b0 0x00000000696e746c 0x0000000000000000 0x0000000000000000 [2992583us][cpu=0][(null)][sev=2][seq=36]:addr(0x1120ac90) 0x0000000100000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2993622us][cpu=0][(null)][sev=2][seq=37]:addr(0x1120acb0) 0x000000001120acb0 0x000000001120acb0 0x0000000000000000 0x0000000000000000 [2994663us][cpu=0][(null)][sev=2][seq=38]: [2995020us][cpu=0][(null)][sev=2][seq=39]:Host Call Trace: [2995677us][cpu=0][(null)][sev=2][seq=40]:----> 0x000000001100c976 [2996422us][cpu=0][(null)][sev=2][seq=41]:----> 0x000000001100c362 [2997173us][cpu=0][(null)][sev=2][seq=42]:----> 0x000000001101646e [2997922us][cpu=0][(null)][sev=2][seq=43]:----> 0x000000001101701c [2998665us][cpu=0][(null)][sev=2][seq=44]:----> 0x0000000011017128 [2999423us][cpu=0][(null)][sev=2][seq=45]:----> 0x00000000110081e8 [3000164us][cpu=0][(null)][sev=2][seq=46]:

 

Regards,

Kishore

 

On Wed, May 12, 2021 at 2:08 PM Geoffroy Van Cutsem <geoffroy.vancutsem@...> wrote:

Do you see the ACRN shell prompt (ACRN:\>) if you press “Enter” at that point? I do on my setup and I can then switch to the Service VM console by typing ‘vm_console 0’ (press Ctrl+spacebar from there if you wish to the return to the ACRN shell).

 

I see the same error messages than you but ACRN is running. My Service VM kernel is not booting to completion though, it hangs at some point, then crashes and reboot. But I do see additional error messages from ACRN (related to iommu), specifically: “iommu: DMAR OP Timeout! @ dmar_issue_qi_request”

 

Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 2:53 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks Nanlin. This problem is resolved.

 

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

===

 

 

On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: ACRN support for Dell 07KY25

yogesh@...
 

Hi Fuzhong,
    Thanks for your kind reply, Will follow steps to generate board and scenario xml. 
Regards,
Yogesh


Re: failed to assign iommu device!

Geoffroy Van Cutsem
 

Thanks Kishore. I see the same error message as reported below if I increase the loglevel for the console to 5 or 6. In that case, there is no Service VM started, ‘vm_list’ does not list any VM and ‘vm_console 0’ rightfully return that the parameter is invalid.

 

The difference I have is that I have never seen the Service VM booting to a logging prompt on my system (I’m using a different HW: Skylake, running Ubuntu 20.04)


Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Wednesday, May 12, 2021 11:54 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Geoffroy,

    Thanks. This helps a lot.

    I saw ACRN shell prompt. But, did not bother to check VMs and boot service VM till I was your message.

    Service VM is booting for me.

 

    Very few times, when I select ACRN hypervisor in GRUB, I get following error message. When this error occurs, Service VM is not listed and thus, vm_console 0 gives invalid parameter error

 

Loading ACRN hypervisor with SDC scenario ... error: Can't get controller info.. Multiboot Bootloader: GRUB 2.02-2ubuntu8.23 [0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used! ACRN Hypervisor calibrate_tsc, tsc_khz=2591719 [2639579us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) ***@***.*** build by kishoreacrn, start time 2628205us [2640883us][cpu=0][(null)][sev=2][seq=3]:API version 1.0 [2641301us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz ACRN:\>[2971433us][cpu=0][(null)][sev=2][seq=19]:Assertion failed in file arch/x86/vtd.c,line 256 : fatal error [2972631us][cpu=0][(null)][sev=2][seq=20]: delta = (actual_load_address - CONFIG_HV_RAM_START) = 0x0 [2973735us][cpu=0][(null)][sev=2][seq=21]: Host Stack: CPU_ID = 0 [2974471us][cpu=0][(null)][sev=2][seq=22]:addr(0x1120aad0) 0x0000000000000000 0x00000000007d9bdc 0x000000001305d040 0x00000001ca882b8c [2975924us][cpu=0][(null)][sev=2][seq=23]:addr(0x1120aaf0) 0x000000001120ab30 0x000000001100c976 0x0000000011051160 0x000000001305d040 [2977368us][cpu=0][(null)][sev=2][seq=24]:addr(0x1120ab10) 0x0000000000000000 0x000000001100c880 0x000000001103d134 0x0000000000000000 [2978814us][cpu=0][(null)][sev=2][seq=25]:addr(0x1120ab30) 0x000000001120ab60 0x000000001100c362 0x000000001306d000 0x000000001306d000 [2980256us][cpu=0][(null)][sev=2][seq=26]:addr(0x1120ab50) 0x0000000011051160 0x000000001306d000 0x000000001120abf0 0x000000001101646e [2981701us][cpu=0][(null)][sev=2][seq=27]:addr(0x1120ab70) 0x0000000000000000 0x0000000011051160 0x000000000000000f 0x000000001120ac00 [2983148us][cpu=0][(null)][sev=2][seq=28]:addr(0x1120ab90) 0x000000000000000d 0x0000000000000000 0x000000017e300000 0x0000000000000000 [2984589us][cpu=0][(null)][sev=2][seq=29]:addr(0x1120abb0) 0x00000000110450c7 0x839118c584437d48 0x0000000000000000 0x0000000000000000 [2986034us][cpu=0][(null)][sev=2][seq=30]:addr(0x1120abd0) 0x0000000011051160 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2987358us][cpu=0][(null)][sev=2][seq=31]:addr(0x1120abf0) 0x000000001120ac30 0x000000001101701c 0x0000000000000000 0x839118c584437d48 [2988404us][cpu=0][(null)][sev=2][seq=32]:addr(0x1120ac10) 0x000000001120ac50 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2989448us][cpu=0][(null)][sev=2][seq=33]:addr(0x1120ac30) 0x000000001120ac50 0x0000000011017128 0x0000000000000000 0x0000000000000000 [2990490us][cpu=0][(null)][sev=2][seq=34]:addr(0x1120ac50) 0x000000001120ac70 0x00000000110081e8 0x00000000110010e2 0x0000000000000000 [2991538us][cpu=0][(null)][sev=2][seq=35]:addr(0x1120ac70) 0x00000000110548b0 0x00000000696e746c 0x0000000000000000 0x0000000000000000 [2992583us][cpu=0][(null)][sev=2][seq=36]:addr(0x1120ac90) 0x0000000100000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2993622us][cpu=0][(null)][sev=2][seq=37]:addr(0x1120acb0) 0x000000001120acb0 0x000000001120acb0 0x0000000000000000 0x0000000000000000 [2994663us][cpu=0][(null)][sev=2][seq=38]: [2995020us][cpu=0][(null)][sev=2][seq=39]:Host Call Trace: [2995677us][cpu=0][(null)][sev=2][seq=40]:----> 0x000000001100c976 [2996422us][cpu=0][(null)][sev=2][seq=41]:----> 0x000000001100c362 [2997173us][cpu=0][(null)][sev=2][seq=42]:----> 0x000000001101646e [2997922us][cpu=0][(null)][sev=2][seq=43]:----> 0x000000001101701c [2998665us][cpu=0][(null)][sev=2][seq=44]:----> 0x0000000011017128 [2999423us][cpu=0][(null)][sev=2][seq=45]:----> 0x00000000110081e8 [3000164us][cpu=0][(null)][sev=2][seq=46]:

 

Regards,

Kishore

 

On Wed, May 12, 2021 at 2:08 PM Geoffroy Van Cutsem <geoffroy.vancutsem@...> wrote:

Do you see the ACRN shell prompt (ACRN:\>) if you press “Enter” at that point? I do on my setup and I can then switch to the Service VM console by typing ‘vm_console 0’ (press Ctrl+spacebar from there if you wish to the return to the ACRN shell).

 

I see the same error messages than you but ACRN is running. My Service VM kernel is not booting to completion though, it hangs at some point, then crashes and reboot. But I do see additional error messages from ACRN (related to iommu), specifically: “iommu: DMAR OP Timeout! @ dmar_issue_qi_request”

 

Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 2:53 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks Nanlin. This problem is resolved.

 

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

===

 

 

On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Kishore Kanala
 

Hi Geoffroy,
    Thanks. This helps a lot.
    I saw ACRN shell prompt. But, did not bother to check VMs and boot service VM till I was your message.
    Service VM is booting for me.

    Very few times, when I select ACRN hypervisor in GRUB, I get following error message. When this error occurs, Service VM is not listed and thus, vm_console 0 gives invalid parameter error

Loading ACRN hypervisor with SDC scenario ... error: Can't get controller info.. Multiboot Bootloader: GRUB 2.02-2ubuntu8.23 [0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used! ACRN Hypervisor calibrate_tsc, tsc_khz=2591719 [2639579us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) ***@***.*** build by kishoreacrn, start time 2628205us [2640883us][cpu=0][(null)][sev=2][seq=3]:API version 1.0 [2641301us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz ACRN:\>[2971433us][cpu=0][(null)][sev=2][seq=19]:Assertion failed in file arch/x86/vtd.c,line 256 : fatal error [2972631us][cpu=0][(null)][sev=2][seq=20]: delta = (actual_load_address - CONFIG_HV_RAM_START) = 0x0 [2973735us][cpu=0][(null)][sev=2][seq=21]: Host Stack: CPU_ID = 0 [2974471us][cpu=0][(null)][sev=2][seq=22]:addr(0x1120aad0) 0x0000000000000000 0x00000000007d9bdc 0x000000001305d040 0x00000001ca882b8c [2975924us][cpu=0][(null)][sev=2][seq=23]:addr(0x1120aaf0) 0x000000001120ab30 0x000000001100c976 0x0000000011051160 0x000000001305d040 [2977368us][cpu=0][(null)][sev=2][seq=24]:addr(0x1120ab10) 0x0000000000000000 0x000000001100c880 0x000000001103d134 0x0000000000000000 [2978814us][cpu=0][(null)][sev=2][seq=25]:addr(0x1120ab30) 0x000000001120ab60 0x000000001100c362 0x000000001306d000 0x000000001306d000 [2980256us][cpu=0][(null)][sev=2][seq=26]:addr(0x1120ab50) 0x0000000011051160 0x000000001306d000 0x000000001120abf0 0x000000001101646e [2981701us][cpu=0][(null)][sev=2][seq=27]:addr(0x1120ab70) 0x0000000000000000 0x0000000011051160 0x000000000000000f 0x000000001120ac00 [2983148us][cpu=0][(null)][sev=2][seq=28]:addr(0x1120ab90) 0x000000000000000d 0x0000000000000000 0x000000017e300000 0x0000000000000000 [2984589us][cpu=0][(null)][sev=2][seq=29]:addr(0x1120abb0) 0x00000000110450c7 0x839118c584437d48 0x0000000000000000 0x0000000000000000 [2986034us][cpu=0][(null)][sev=2][seq=30]:addr(0x1120abd0) 0x0000000011051160 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2987358us][cpu=0][(null)][sev=2][seq=31]:addr(0x1120abf0) 0x000000001120ac30 0x000000001101701c 0x0000000000000000 0x839118c584437d48 [2988404us][cpu=0][(null)][sev=2][seq=32]:addr(0x1120ac10) 0x000000001120ac50 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2989448us][cpu=0][(null)][sev=2][seq=33]:addr(0x1120ac30) 0x000000001120ac50 0x0000000011017128 0x0000000000000000 0x0000000000000000 [2990490us][cpu=0][(null)][sev=2][seq=34]:addr(0x1120ac50) 0x000000001120ac70 0x00000000110081e8 0x00000000110010e2 0x0000000000000000 [2991538us][cpu=0][(null)][sev=2][seq=35]:addr(0x1120ac70) 0x00000000110548b0 0x00000000696e746c 0x0000000000000000 0x0000000000000000 [2992583us][cpu=0][(null)][sev=2][seq=36]:addr(0x1120ac90) 0x0000000100000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 [2993622us][cpu=0][(null)][sev=2][seq=37]:addr(0x1120acb0) 0x000000001120acb0 0x000000001120acb0 0x0000000000000000 0x0000000000000000 [2994663us][cpu=0][(null)][sev=2][seq=38]: [2995020us][cpu=0][(null)][sev=2][seq=39]:Host Call Trace: [2995677us][cpu=0][(null)][sev=2][seq=40]:----> 0x000000001100c976 [2996422us][cpu=0][(null)][sev=2][seq=41]:----> 0x000000001100c362 [2997173us][cpu=0][(null)][sev=2][seq=42]:----> 0x000000001101646e [2997922us][cpu=0][(null)][sev=2][seq=43]:----> 0x000000001101701c [2998665us][cpu=0][(null)][sev=2][seq=44]:----> 0x0000000011017128 [2999423us][cpu=0][(null)][sev=2][seq=45]:----> 0x00000000110081e8 [3000164us][cpu=0][(null)][sev=2][seq=46]:

Regards,
Kishore

On Wed, May 12, 2021 at 2:08 PM Geoffroy Van Cutsem <geoffroy.vancutsem@...> wrote:

Do you see the ACRN shell prompt (ACRN:\>) if you press “Enter” at that point? I do on my setup and I can then switch to the Service VM console by typing ‘vm_console 0’ (press Ctrl+spacebar from there if you wish to the return to the ACRN shell).

 

I see the same error messages than you but ACRN is running. My Service VM kernel is not booting to completion though, it hangs at some point, then crashes and reboot. But I do see additional error messages from ACRN (related to iommu), specifically: “iommu: DMAR OP Timeout! @ dmar_issue_qi_request”

 

Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 2:53 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks Nanlin. This problem is resolved.

 

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

===

 

 

On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Geoffroy Van Cutsem
 

Do you see the ACRN shell prompt (ACRN:\>) if you press “Enter” at that point? I do on my setup and I can then switch to the Service VM console by typing ‘vm_console 0’ (press Ctrl+spacebar from there if you wish to the return to the ACRN shell).

 

I see the same error messages than you but ACRN is running. My Service VM kernel is not booting to completion though, it hangs at some point, then crashes and reboot. But I do see additional error messages from ACRN (related to iommu), specifically: “iommu: DMAR OP Timeout! @ dmar_issue_qi_request”

 

Thanks,

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 2:53 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks Nanlin. This problem is resolved.

 

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

===

 

 

On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: ACRN support for Dell 07KY25

Liu, Fuzhong
 

Hi Yogesh

From you shared info, the CPU is  i7-7700(Code Name Products formerly Kaby Lake)

 

     *-cpu

          description: CPU

          product: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz

          vendor: Intel Corp

 

Kaby Lake is in ACRN supported HW list, please generate your board.xml base on following link:

https://projectacrn.github.io/latest/tutorials/acrn_configuration_tool.html#using-acrn-configuration-toolset

 

Any more question, please raise git issue at:

https://github.com/projectacrn/acrn-hypervisor/issues

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of yogesh@...
Sent: Wednesday, May 12, 2021 12:36 AM
To: acrn-users@...
Subject: [acrn-users] ACRN support for Dell 07KY25

 

Hi,
   Could you please let me know if ACRN is having support for Dell 07KY25 Motherboard. 
I am looking for enabling Ubuntu service VM with couple of user vms(ubuntu/windows/debian).

I have added detailed information about the system in attachment. 

    description: Desktop Computer

    product: Inspiron 3668 (0763)

    vendor: Dell Inc.

    serial: 5PSRFK2

    width: 64 bits

    capabilities: smbios-3.0 dmi-3.0 smp vsyscall32

    configuration: boot=normal chassis=desktop family=Inspiron sku=0763 uuid=44454C4C-5000-1053-8052-B5C04F464B32

  *-core

       description: Motherboard

       product: 07KY25

       vendor: Dell Inc.

       physical id: 0

       version: A00

       serial: /5PSRFK2/CN7016373L069C/

Thanks,
Yogesh


ACRN support for Dell 07KY25

yogesh@...
 

Hi,
   Could you please let me know if ACRN is having support for Dell 07KY25 Motherboard. 
I am looking for enabling Ubuntu service VM with couple of user vms(ubuntu/windows/debian).

I have added detailed information about the system in attachment. 
    description: Desktop Computer
    product: Inspiron 3668 (0763)
    vendor: Dell Inc.
    serial: 5PSRFK2
    width: 64 bits
    capabilities: smbios-3.0 dmi-3.0 smp vsyscall32
    configuration: boot=normal chassis=desktop family=Inspiron sku=0763 uuid=44454C4C-5000-1053-8052-B5C04F464B32
  *-core
       description: Motherboard
       product: 07KY25
       vendor: Dell Inc.
       physical id: 0
       version: A00
       serial: /5PSRFK2/CN7016373L069C/

Thanks,
Yogesh


Re: failed to assign iommu device!

Kishore Kanala
 

Thanks Nanlin. This problem is resolved.

Next hurdle... check_vmx_ctrl is giving an error (hypervisor/arch/x86/guest/vmcs.c).

===
Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
ACRN Hypervisor
calibrate_tsc, tsc_khz=2592235
[2084832us][cpu=0][(null)][sev=2][seq=2]:HV version 2.4-2021-05-11 18:07:18- DBG (daily tag:) sdc@qemu build by kishoreacrn, start time 2072994us
[2086640us][cpu=0][(null)][sev=2][seq=3]:API version 1.0
[2087323us][cpu=0][(null)][sev=2][seq=4]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
ACRN:\>[2461933us][cpu=0][(null)][sev=2][seq=29]:Start VM id: 0 name: ACRN SOS VM
[2463120us][cpu=0][vm0:vcpu0][sev=3][seq=30]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[2464517us][cpu=0][vm0:vcpu0][sev=3][seq=31]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[3960186us][cpu=0][vm0:vcpu0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[0]...
[3961622us][cpu=1][vm0:vcpu1][sev=3][seq=35]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[3963651us][cpu=1][vm0:vcpu1][sev=3][seq=36]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4042481us][cpu=0][vm0:vcpu0][sev=3][seq=38]:vlapic: Start Secondary VCPU2 for VM[0]...
[4043858us][cpu=2][vm0:vcpu2][sev=3][seq=39]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4045112us][cpu=2][vm0:vcpu2][sev=3][seq=40]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab

[4121542us][cpu=0][vm0:vcpu0][sev=3][seq=42]:vlapic: Start Secondary VCPU3 for VM[0]...
[4123077us][cpu=3][vm0:vcpu3][sev=3][seq=43]:VMX ctrl 0x482 not fully enabled: request 0x92220088 but get 0x9621e1fa

[4125074us][cpu=3][vm0:vcpu3][sev=3][seq=44]:VMX ctrl 0x48b not fully enabled: request 0x4ab but get 0xab
===


On Tue, May 11, 2021 at 2:34 PM Xie, Nanlin <nanlin.xie@...> wrote:

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Xie, Nanlin
 

Hi, Kishore,

 

For v2.4 codebase; please change below and try again. Thanks!
MAXIMUM_PA_WIDTH from 39U to 40U

diff --git a/hypervisor/include/arch/x86/page.h b/hypervisor/include/arch/x86/page.h

index 1094fa30d..e7c042903 100644

--- a/hypervisor/include/arch/x86/page.h

+++ b/hypervisor/include/arch/x86/page.h

@@ -14,7 +14,7 @@

#define PAGE_SIZE      (1U << PAGE_SHIFT)

#define PAGE_MASK      0xFFFFFFFFFFFFF000UL

 

-#define MAXIMUM_PA_WIDTH       39U     /* maximum physical-address width */

+#define MAXIMUM_PA_WIDTH       40U     /* maximum physical-address width */

 

/* size of the low MMIO address space: 2GB */

#define PLATFORM_LO_MMIO_SIZE  0x80000000UL

 

 

Best wishes!

Nanlin

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Tuesday, May 11, 2021 4:54 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

 

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.

   Is there a configuration for physical-address width?

 

===

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!

===

 

Regards,

Kishore

 

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Kishore Kanala
 

Hi Fuzhong,

   Thanks for your response. You are correct. I had disabled this when I was using QEMU 2.11.1. With this, iommu error was seen.

   In the setup where this is enabled, I am getting the following error. I have tried with 2.4 and 2.0 versions.
   Is there a configuration for physical-address width?

===
Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.23
[0us][cpu=0][(null)][sev=3][seq=1]:vlapic: Basic apicv ops is used!
detect_hardware_support, physical-address width (40) over maximum physical-address width (39)
[0us][cpu=0][(null)][sev=1][seq=2]:PANIC: init_pcpu_pre line: 132

[0us][cpu=0][(null)][sev=1][seq=3]:hardware not support!
===

Regards,
Kishore

On Tue, May 11, 2021 at 6:02 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Liu, Fuzhong
 

Hi Kishore

Is aw-bits=48 included for your setup? If not, there will be the issue: failed to assign iommu device!

 

virt-install \

--connect qemu:///system \

--name ACRNSOS \

--machine q35 \

--cpu host-passthrough,+invtsc \

--ram 4096 \

--disk path=/var/lib/libvirt/images/acrnsos.img,size=32 \

--vcpus 4 \

--virt-type kvm \

--os-type linux \

--os-variant ubuntu18.04 \

--graphics none \

--clock offset=utc,tsc_present=yes,kvmclock_present=no \

--qemu-commandline="-machine kernel-irqchip=split -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48" \

--location 'http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/' \

--extra-args "console=tty0 console=ttyS0,115200n8"

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 9:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Thanks for response.

I get same error even with QEMU 4.2.1

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

 

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7

 

On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Geoffroy Van Cutsem
 

Hi Kishore,

 

You also need the gnu-efi package installed to compile the 2.0 version – see https://projectacrn.github.io/2.0/getting-started/building-from-source.html#install-build-tools-and-dependencies (this is the documentation for the v2.0 release). The latest version of that doc dropped that package as we longer use/need it.

 

Thanks,

Geoffroy

 

 

From: acrn-users@... <acrn-users@...> On Behalf Of Kishore Kanala
Sent: Monday, May 10, 2021 3:47 PM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Fuzhong,

 

     I am getting the following compilation error with 2.0. Trying to include acrn-kernel/include/kernel

 

====

boot.c:34:10: fatal error: efi.h: No such file or directory
 #include <efi.h>
          ^~~~~~~
compilation terminated.
Makefile:125: recipe for target '/home/kishore/2.0-acrn-hypervisor/acrn-hypervisor-2.0/build/hypervisor/misc/efi-stub/boot.o' failed

====

 

Regards,

Kishore

 

On Fri, May 7, 2021 at 4:17 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Kishore Kanala
 

Hi Fuzhong,

     I am getting the following compilation error with 2.0. Trying to include acrn-kernel/include/kernel

====
boot.c:34:10: fatal error: efi.h: No such file or directory
 #include <efi.h>
          ^~~~~~~
compilation terminated.
Makefile:125: recipe for target '/home/kishore/2.0-acrn-hypervisor/acrn-hypervisor-2.0/build/hypervisor/misc/efi-stub/boot.o' failed
====

Regards,
Kishore


On Fri, May 7, 2021 at 4:17 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Kishore Kanala
 

Thanks for response.
I get same error even with QEMU 4.2.1
kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ /usr/bin/kvm-spice  --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

kishore@kishore-OMEN-by-HP-Laptop-15-dh0xxx:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-72-generic root=UUID=27bc3288-2660-433b-93f2-be9ff3ed9e47 ro quiet splash vt.handoff=7


On Fri, May 7, 2021 at 8:01 AM Liu, Fuzhong <fuzhong.liu@...> wrote:

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Liu, Fuzhong
 

Hi Krishna Kishore

Could you please share your qemu version?

QEMU emulator version 4.2.0 is verified in this guide: https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

 

You should use qemu version >= 2.11.2 with below commit:

commit 0b250250b7e7298e0001faf76015da4b03b6b1b3

Author: Prasad Singamsetty <prasad.singamsetty@...>

Date:   Tue Nov 14 18:13:50 2017 -0500

 

    intel-iommu: Extend address width to 48 bits

 

    The current implementation of Intel IOMMU code only supports 39 bits

    iova address width. This patch provides a new parameter (x-aw-bits)

    for intel-iommu to extend its address width to 48 bits but keeping the

    default the same (39 bits). The reason for not changing the default

    is to avoid potential compatibility problems with live migration of

    intel-iommu enabled QEMU guest. The only valid values for 'x-aw-bits'

    parameter are 39 and 48.

 

    After enabling larger address width (48), we should be able to map

    larger iova addresses in the guest. For example, a QEMU guest that

    is configured with large memory ( >=1TB ). To check whether 48 bits

    aw is enabled, we can grep in the guest dmesg output with line:

    "DMAR: Host address width 48".

 

    Signed-off-by: Prasad Singamsetty <prasad.singamsety@...>

    Reviewed-by: Peter Xu <peterx@...>

    Reviewed-by: Michael S. Tsirkin <mst@...>

    Signed-off-by: Michael S. Tsirkin <mst@...>

    (cherry picked from commit 37f51384ae05bd50f83308339dbffa3e78404874)

    Signed-off-by: Michael Roth <mdroth@...>

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Friday, May 7, 2021 6:47 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Liu, Fuzhong
 

Hi Krishna Kishore

Please try v2.0 for ACRN on qemu.

If still fail, please share cat /proc/cmdline of your host OS.

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Friday, May 7, 2021 5:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] failed to assign iommu device!

 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


Re: failed to assign iommu device!

Geoffroy Van Cutsem
 

Hi!

 

Which version of ACRN are you using?

 

The error looks a bit different but be aware that the latest version of ACRN has a regression on QEMU: https://github.com/projectacrn/acrn-hypervisor/issues/5938

 

Thanks,
Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Krishna Kishore
Sent: Thursday, May 6, 2021 7:22 PM
To: acrn-users@...
Subject: [acrn-users] failed to assign iommu device!

 

[Edited Message Follows]

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>


failed to assign iommu device!

Kishore Kanala
 
Edited

Can someone let me know how to overcome this problem? I am trying this https://projectacrn.github.io/latest/tutorials/acrn_on_qemu.html

Loading ACRN hypervisor with SDC scenario ...
error: Can't get controller info..
Multiboot Bootloader: GRUB 2.02-2ubuntu8.21
ACRN Hypervisor
calibrate_tsc, tsc_khz=2591442
[4589978us][cpu=0][(null)][sev=2][seq=1]:HV version 2.5-unstable-2021-05-06 20:06:44-7c92ac425 DBG (daily tag:acrn-2021w18.5-180000p) sdc@qemu build by kishore, start time 4579269us
[4591202us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[4591568us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
[4908926us][cpu=0][(null)][sev=1][seq=18]:PANIC: assign_vdev_pt_iommu_domain line: 296

[4909856us][cpu=0][(null)][sev=1][seq=19]:failed to assign iommu device!
ACRN:\>