Issue on latest Clear Linux and ACRN on UP2 board


Tomasz Bursztyka
 

Hi,

I cannot get ACRN hypervisor running on my setup:
- UP² board (Celeron N3350)
- Clear Linux, fresh install, version 32910

I followed the documentation here:
- https://projectacrn.github.io/latest/tutorials/up2.html
- https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html


However, once I rebooted I did not get the kernel message
[ 0.000000] Hypervisor detected: ACRN

I only get these:
[ 5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[ 5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!

So the hypervisor is not running.
There is no acrn related devices in /dev, I cannot start any VM
obviously etc..

Here is the kernel cmd line:

[ 0.000000] Command line: initrd=\EFI\org.clearlinux\freestanding-
00-intel-ucode.cpio initrd=\EFI\org.clearlinux\freestanding-i915-
firmware.cpio.xz root=PARTUUID=05e27678-99d6-4bed-bbbe-d28cb026566a
quiet console=tty0 console=ttyS0,115200n8 consoleblank=0
cryptomgr.notests hvlog=2M@0x1FE00000
i915.avail_planes_per_pipe=0x01010F
i915.domain_plane_owners=0x011111110000 i915.enable_guc=0
i915.enable_gvt=1 i915.nuclear_pageflip=1 ignore_loglevel
init=/usr/lib/systemd/systemd-bootchart intel_iommu=igfx_off
memmap=2M$0x1FE00000 no_timer_check no_timer_check noreplace-smp
rcu_nocbs=0-64 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs
rootwait tsc=reliable rw


The EFI configuration seems legit, however:

# efibootmgr -v
BootCurrent: 0003
Timeout: 1 seconds
BootOrder: 0003,0005,0001
Boot0001 UEFI: Built-in EFI Shell VenMedia(5023b95c-db26-429b-
a648-bd47664c8012)..BO
Boot0003* UEFI OS HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0005* ACRN Hypervisor HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\ACRN\ACRN.EFI)b.o.o.t.l.o.a.d.e.r
.=.\.E.F.I.\.o.r.g...c.l.e.a.r.l.i.n.u.x.\.b.o.o.t.l.o.a.d.e.r.x.6.4...
e.f.i. .u.a.r.t.=.b.d.f.@.0.:.1.8...1.


I switched the order to 0005,0003,0001 but it made no difference, and
once I did: the bootorder had automaticaly changed back to
0003,0005,0001. Is this nominal?

The acrn.efi is from acrn_quick_setup.sh, which requested to do:

cp /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi /usr/lib/acrn/acrn.up-
apl01.industry.efi

So I did. (and installation seemed to go well.)


Any idea where the problem could be?


Yin, Fengwei <fengwei.yin@...>
 


On 4/28/20 3:13 PM, Tomasz Bursztyka wrote:
Hi,

I cannot get ACRN hypervisor running on my setup: 
- UP² board (Celeron N3350)
- Clear Linux, fresh install, version 32910

I followed the documentation here: 
- https://projectacrn.github.io/latest/tutorials/up2.html
- https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html


However, once I rebooted I did not get the kernel message
[    0.000000] Hypervisor detected: ACRN

I only get these:
[    5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[    5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!

This means you boot sos kernel directly without ACRN.


Regards

Yin, Fengwei



So the hypervisor is not running.
There is no acrn related devices in /dev, I cannot start any VM
obviously etc..

Here is the kernel cmd line:

[    0.000000] Command line: initrd=\EFI\org.clearlinux\freestanding-
00-intel-ucode.cpio initrd=\EFI\org.clearlinux\freestanding-i915-
firmware.cpio.xz root=PARTUUID=05e27678-99d6-4bed-bbbe-d28cb026566a
quiet console=tty0 console=ttyS0,115200n8 consoleblank=0
cryptomgr.notests hvlog=2M@0x1FE00000
i915.avail_planes_per_pipe=0x01010F
i915.domain_plane_owners=0x011111110000 i915.enable_guc=0
i915.enable_gvt=1 i915.nuclear_pageflip=1 ignore_loglevel
init=/usr/lib/systemd/systemd-bootchart intel_iommu=igfx_off
memmap=2M$0x1FE00000 no_timer_check no_timer_check noreplace-smp
rcu_nocbs=0-64 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs
rootwait tsc=reliable rw


The EFI configuration seems legit, however:

 # efibootmgr -v
BootCurrent: 0003
Timeout: 1 seconds
BootOrder: 0003,0005,0001
Boot0001  UEFI: Built-in EFI Shell	VenMedia(5023b95c-db26-429b-
a648-bd47664c8012)..BO
Boot0003* UEFI OS	HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0005* ACRN Hypervisor	HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\ACRN\ACRN.EFI)b.o.o.t.l.o.a.d.e.r
.=.\.E.F.I.\.o.r.g...c.l.e.a.r.l.i.n.u.x.\.b.o.o.t.l.o.a.d.e.r.x.6.4...
e.f.i. .u.a.r.t.=.b.d.f.@.0.:.1.8...1.


I switched the order to 0005,0003,0001 but it made no difference, and
once I did: the bootorder had automaticaly changed back to
0003,0005,0001. Is this nominal? 

The acrn.efi is from acrn_quick_setup.sh, which requested to do:

cp /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi /usr/lib/acrn/acrn.up-
apl01.industry.efi

So I did. (and installation seemed to go well.)


Any idea where the problem could be?





Tomasz Bursztyka
 

Hi,


However, once I rebooted I did not get the kernel message
[ 0.000000] Hypervisor detected: ACRN

I only get these:
[ 5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[ 5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!


This means you boot sos kernel directly without ACRN.
That I figured out already yes.

But why? (installer did not show any issue, the only thing which seems
suspicious is the EFI bootorder as I described)
And how to fix this?

Thanks,

Tomasz


Yin, Fengwei <fengwei.yin@...>
 

Hi Tomasz,

On 4/28/20 3:40 PM, Tomasz Bursztyka wrote:
Hi,


However, once I rebooted I did not get the kernel message
[    0.000000] Hypervisor detected: ACRN

I only get these:
[    5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[    5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!

    
    
This means you boot sos kernel directly without ACRN.
That I figured out already yes.

But why? (installer did not show any issue, the only thing which seems
suspicious is the EFI bootorder as I described)
And how to fix this?

Not sure whether you are familiar with EFI shell. If yes, can you

try to load acrn.efi from EFI shell? That make sure the ACRN is

loaded.


Regards

Yin, Fengwei



Thanks,

Tomasz





Tomasz Bursztyka
 

Hi,

suspicious is the EFI bootorder as I described)
And how to fix this?
Not sure whether you are familiar with EFI shell. If yes, can you

try to load acrn.efi from EFI shell? That make sure the ACRN is

loaded.
Thanks for the hint.

I had no re-enable the UEFI shell boot entry. There I could try to load
acrn.efi: and it loaded well.

So that gave me an idea: I disabled the boot 0003 entirely (namely UEFI
OS, I guess it's the one installed by clear linux). Reordered to get
the entry 0005 to be the first and now it works:

[ 0.000000] Hypervisor detected: ACRN
[ 2.052220] ACRNTrace: Initialized acrn trace module with 2 cpu
[ 2.053849] ACRN HVLog: Failed to init last hvlog devs, errno -19
[ 2.054223] ACRN HVLog: Initialized hvlog module with 2 cpu


Thanks,

Tomasz


Liu, Fuzhong
 

Hi Tomasz,

After UP2 boot up with native, Could you please try following command with uart disabled first?

sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0-p 1 -L "ACRN Hypervisor UP2" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=disabled "

 

Thanks!

 

BR.

Fuzhong

 

From: acrn-users@... <acrn-users@...> On Behalf Of Yin, Fengwei
Sent: Tuesday, April 28, 2020 3:45 PM
To: acrn-users@...
Subject: Re: [acrn-users] Issue on latest Clear Linux and ACRN on UP2 board

 

Hi Tomasz,

On 4/28/20 3:40 PM, Tomasz Bursztyka wrote:

Hi,
 
 
However, once I rebooted I did not get the kernel message
[    0.000000] Hypervisor detected: ACRN
 
I only get these:
[    5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[    5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!
 
    
 
    
This means you boot sos kernel directly without ACRN.
 
That I figured out already yes.
 
But why? (installer did not show any issue, the only thing which seems
suspicious is the EFI bootorder as I described)
And how to fix this?

Not sure whether you are familiar with EFI shell. If yes, can you

try to load acrn.efi from EFI shell? That make sure the ACRN is

loaded.

 

Regards

Yin, Fengwei

 

 
 
Thanks,
 
Tomasz
 
 
 
 


Tzeng, Tonny <tonny.tzeng@...>
 

Hi Tomasz,

Your `efibootmgr -v` output indicates the #3 entry will be picked up first, which is the fall back boot entry for Systemd-boot. Please change the order with `efibootmgr -o` option. You should also enter the UP2 BIOS to make sure the ACRN boot entry is the first option.

Regards,
Tonny

-----Original Message-----
From: acrn-users@... <acrn-users@...> On Behalf Of Tomasz Bursztyka
Sent: Tuesday, April 28, 2020 3:14 PM
To: acrn-users@...
Subject: [acrn-users] Issue on latest Clear Linux and ACRN on UP2 board

Hi,

I cannot get ACRN hypervisor running on my setup:
- UP² board (Celeron N3350)
- Clear Linux, fresh install, version 32910

I followed the documentation here:
- https://projectacrn.github.io/latest/tutorials/up2.html
- https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html


However, once I rebooted I did not get the kernel message
[ 0.000000] Hypervisor detected: ACRN

I only get these:
[ 5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[ 5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!

So the hypervisor is not running.
There is no acrn related devices in /dev, I cannot start any VM obviously etc..

Here is the kernel cmd line:

[ 0.000000] Command line: initrd=\EFI\org.clearlinux\freestanding-
00-intel-ucode.cpio initrd=\EFI\org.clearlinux\freestanding-i915-
firmware.cpio.xz root=PARTUUID=05e27678-99d6-4bed-bbbe-d28cb026566a
quiet console=tty0 console=ttyS0,115200n8 consoleblank=0 cryptomgr.notests hvlog=2M@0x1FE00000 i915.avail_planes_per_pipe=0x01010F
i915.domain_plane_owners=0x011111110000 i915.enable_guc=0
i915.enable_gvt=1 i915.nuclear_pageflip=1 ignore_loglevel init=/usr/lib/systemd/systemd-bootchart intel_iommu=igfx_off
memmap=2M$0x1FE00000 no_timer_check no_timer_check noreplace-smp
rcu_nocbs=0-64 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs rootwait tsc=reliable rw


The EFI configuration seems legit, however:

# efibootmgr -v
BootCurrent: 0003
Timeout: 1 seconds
BootOrder: 0003,0005,0001
Boot0001 UEFI: Built-in EFI Shell VenMedia(5023b95c-db26-429b-
a648-bd47664c8012)..BO
Boot0003* UEFI OS HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0005* ACRN Hypervisor HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\ACRN\ACRN.EFI)b.o.o.t.l.o.a.d.e.r
.=.\.E.F.I.\.o.r.g...c.l.e.a.r.l.i.n.u.x.\.b.o.o.t.l.o.a.d.e.r.x.6.4...
e.f.i. .u.a.r.t.=.b.d.f.@.0.:.1.8...1.


I switched the order to 0005,0003,0001 but it made no difference, and once I did: the bootorder had automaticaly changed back to 0003,0005,0001. Is this nominal?

The acrn.efi is from acrn_quick_setup.sh, which requested to do:

cp /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi /usr/lib/acrn/acrn.up- apl01.industry.efi

So I did. (and installation seemed to go well.)


Any idea where the problem could be?


Geoffroy Van Cutsem
 

I think this has to do with the UP2 UEFI firmware, it gives me problem on my UP2 board. I could never figure out what sequence of commands would reliably set the boot order correctly but I also very often see that setting a new entry with efibootmgr looks alright, until you reboot and the boot order was changed (I suspect by the firmware itself). It seems that I have had better luck with adding a new boot entry *and* changing the boot order (efibootmgr -o).

Net net, your installation is most likely just fine but you're dealing with a capricious EFI firmware.

Geoffroy

-----Original Message-----
From: acrn-users@... <acrn-users@...>
On Behalf Of Tzeng, Tonny
Sent: Tuesday, April 28, 2020 11:34 AM
To: acrn-users@...
Subject: Re: [acrn-users] Issue on latest Clear Linux and ACRN on UP2 board

Hi Tomasz,

Your `efibootmgr -v` output indicates the #3 entry will be picked up first,
which is the fall back boot entry for Systemd-boot. Please change the order
with `efibootmgr -o` option. You should also enter the UP2 BIOS to make
sure the ACRN boot entry is the first option.

Regards,
Tonny

-----Original Message-----
From: acrn-users@... <acrn-users@...>
On Behalf Of Tomasz Bursztyka
Sent: Tuesday, April 28, 2020 3:14 PM
To: acrn-users@...
Subject: [acrn-users] Issue on latest Clear Linux and ACRN on UP2 board

Hi,

I cannot get ACRN hypervisor running on my setup:
- UP² board (Celeron N3350)
- Clear Linux, fresh install, version 32910

I followed the documentation here:
- https://projectacrn.github.io/latest/tutorials/up2.html
- https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html


However, once I rebooted I did not get the kernel message
[ 0.000000] Hypervisor detected: ACRN

I only get these:
[ 5.759483] ACRNTrace: acrn_trace: not support acrn hypervisor!
[ 5.766132] ACRN HVLog: acrn_hvlog: not running under acrn
hypervisor!

So the hypervisor is not running.
There is no acrn related devices in /dev, I cannot start any VM obviously etc..

Here is the kernel cmd line:

[ 0.000000] Command line: initrd=\EFI\org.clearlinux\freestanding-
00-intel-ucode.cpio initrd=\EFI\org.clearlinux\freestanding-i915-
firmware.cpio.xz root=PARTUUID=05e27678-99d6-4bed-bbbe-d28cb026566a
quiet console=tty0 console=ttyS0,115200n8 consoleblank=0
cryptomgr.notests hvlog=2M@0x1FE00000
i915.avail_planes_per_pipe=0x01010F
i915.domain_plane_owners=0x011111110000 i915.enable_guc=0
i915.enable_gvt=1 i915.nuclear_pageflip=1 ignore_loglevel
init=/usr/lib/systemd/systemd-bootchart intel_iommu=igfx_off
memmap=2M$0x1FE00000 no_timer_check no_timer_check noreplace-smp
rcu_nocbs=0-64 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs
rootwait tsc=reliable rw


The EFI configuration seems legit, however:

# efibootmgr -v
BootCurrent: 0003
Timeout: 1 seconds
BootOrder: 0003,0005,0001
Boot0001 UEFI: Built-in EFI Shell VenMedia(5023b95c-db26-429b-
a648-bd47664c8012)..BO
Boot0003* UEFI OS HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0005* ACRN Hypervisor HD(1,GPT,f2d8aada-01a9-4c9c-86fe-
61ccb217e678,0x800,0x47000)/File(\EFI\ACRN\ACRN.EFI)b.o.o.t.l.o.a.d.e.r
.=.\.E.F.I.\.o.r.g...c.l.e.a.r.l.i.n.u.x.\.b.o.o.t.l.o.a.d.e.r.x.6.4...
e.f.i. .u.a.r.t.=.b.d.f.@.0.:.1.8...1.


I switched the order to 0005,0003,0001 but it made no difference, and once I
did: the bootorder had automaticaly changed back to 0003,0005,0001. Is this
nominal?

The acrn.efi is from acrn_quick_setup.sh, which requested to do:

cp /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi /usr/lib/acrn/acrn.up-
apl01.industry.efi

So I did. (and installation seemed to go well.)


Any idea where the problem could be?






Tomasz Bursztyka
 

Hi Tonny,


Your `efibootmgr -v` output indicates the #3 entry will be picked up
first, which is the fall back boot entry for Systemd-boot. Please
change the order with `efibootmgr -o` option. You should also enter
the UP2 BIOS to make sure the ACRN boot entry is the first option.
I did that already and did not work (see my original post), the working
trick was to disable the boot entry 0003.

Tomasz


Tomasz Bursztyka
 

Hi,

I think this has to do with the UP2 UEFI firmware, it gives me
problem on my UP2 board. I could never figure out what sequence of
commands would reliably set the boot order correctly but I also very
often see that setting a new entry with efibootmgr looks alright,
until you reboot and the boot order was changed (I suspect by the
firmware itself). It seems that I have had better luck with adding a
new boot entry *and* changing the boot order (efibootmgr -o).

Net net, your installation is most likely just fine but you're
dealing with a capricious EFI firmware.
Good to know I am not the only one who got this issue.

Maybe it would be worth mentioning it in the up2 doc section of acrn
then.

Tomasz


Geoffroy Van Cutsem
 

-----Original Message-----
From: acrn-users@... <acrn-users@...>
On Behalf Of Tomasz Bursztyka
Sent: Tuesday, April 28, 2020 1:33 PM
To: acrn-users@...
Subject: Re: [acrn-users] Issue on latest Clear Linux and ACRN on UP2 board

Hi,

I think this has to do with the UP2 UEFI firmware, it gives me problem
on my UP2 board. I could never figure out what sequence of commands
would reliably set the boot order correctly but I also very often see
that setting a new entry with efibootmgr looks alright, until you
reboot and the boot order was changed (I suspect by the firmware
itself). It seems that I have had better luck with adding a new boot
entry *and* changing the boot order (efibootmgr -o).

Net net, your installation is most likely just fine but you're dealing
with a capricious EFI firmware.
Good to know I am not the only one who got this issue.

Maybe it would be worth mentioning it in the up2 doc section of acrn then.
I agree, I thought it was mainly my board up until now but I have had similar reports from others since then. I've added a note (PR still pending): https://github.com/projectacrn/acrn-hypervisor/pull/4747

Geoffroy


Tomasz