Questions about ACRN's dependency and Other Linux Distro Supports


Shin, Jun-Sik
 

Hi,
I would like to ask several questions about ACRN.

1. Hardware/Software Dependency
I am trying to follow the tutorial "Using Ubuntu as Service OS" with a hardware that is not included in the supported hardware list.
Specifically, I used SuperMicro E200-8D having Intel Xeon processor D1518, but enabling ACRN hypervisor was failed.
After selecting ACRN hypervisor from Grub menu, the hardware just freezes with a black screen.
So I would like to ask you about specific ACRN's dependencies on hardware/software components such as CPU and BIOS version.

2. ACRN's roadmap for supporting other Linux distros as guest OS
According to the nice presentation by Yu Wang in LinuxCon 2018, ACRN already has a plan for adding various OS distros like Windows and Android, as guest OS.
But, it seems that ACRN doesn't consider supporting popular Linux distros such as Ubuntu, CentOS, and Fedora in near future.
Since ACRN provides great features, If ACRN can support such distros, then, I think, users who are familiar with them can easily utilize it.
So, I would like to ask this question.

Best Regards,
Jun-Sik Shin


Geoffroy Van Cutsem
 

Hi Jun-Sik Shim,

 

Please find my answers in-line. Note that it’s Golden week in China this week (where many of the ACRN developers are located), you may be getting additional details/feedback from them next week.

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of jsshin@...
Sent: Tuesday, October 2, 2018 4:32 PM
To: acrn-users@...
Subject: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi,
I would like to ask several questions about ACRN.

1. Hardware/Software Dependency
I am trying to follow the tutorial "Using Ubuntu as Service OS" with a hardware that is not included in the supported hardware list.
Specifically, I used SuperMicro E200-8D having Intel Xeon processor D1518, but enabling ACRN hypervisor was failed.
After selecting ACRN hypervisor from Grub menu, the hardware just freezes with a black screen.
So I would like to ask you about specific ACRN's dependencies on hardware/software components such as CPU and BIOS version.

[Geoffroy] We have not tested this platform (and I don’t have one available to me). If you see the Grub menu, it means the ACRN hypervisor started successfully and loaded Grub. My first suspect at this stage is the video initialization, can you try to remove the “load_video” line in the /etc/grub.d/40_custom file and re-run ‘sudo update-grub’?

 

Have you tried using Clear Linux as the Service OS, if so, what was the result of that experiment?


2. ACRN's roadmap for supporting other Linux distros as guest OS
According to the nice presentation by Yu Wang in LinuxCon 2018, ACRN already has a plan for adding various OS distros like Windows and Android, as guest OS.
But, it seems that ACRN doesn't consider supporting popular Linux distros such as Ubuntu, CentOS, and Fedora in near future.
Since ACRN provides great features, If ACRN can support such distros, then, I think, users who are familiar with them can easily utilize it.
So, I would like to ask this question.

[Geoffroy] We support Clear Linux as a reference User OS (Guest OS). We’ve successfully used Ubuntu as a Guest OS as an experiment, it was not too hard provided you configure the kernel correctly. The area that may need the most attention is graphics (that is, if you care/want graphics sharing, aka GVT-g). We are not planning to validate or provide recipes for those at the moment but we would very much like to know if you do try, and if you are successful, I would also encourage you to share your experience (perhaps create a wiki page about it?).


Best Regards,
Jun-Sik Shin


Shin, Jun-Sik
 

Hi Geoffroy,

According to your comment, I tried removing a line "load_video" from "/etc/grub.d/40_custom", run "sudo update-grub", and reboot. But, my hardware still showed a black screen.
As another try, I tried installing Clear Linux as the Service OS. Clear Linux as host OS was working without any problems. But, booting ACRN hypervisor showed the same problem. ACRN hypervisor appeared in grub list, but after selecting it, my hardware didn't show anything.
If I can get detailed log messages regarding this issue, then those are very helpful to locate the cause of this problem. Are there any methods to get log messages or debugging messages in this situation?


Miguel Bernal Marin
 

According to your comment, I tried removing a line "load_video" from "/etc/grub.d/40_custom", run "sudo update-grub", and reboot. But, my hardware still showed a black screen.
As another try, I tried installing Clear Linux as the Service OS. Clear Linux as host OS was working without any problems. But, booting ACRN hypervisor showed the same problem. ACRN hypervisor appeared in grub list, but after selecting it, my hardware didn't show anything.
If I can get detailed log messages regarding this issue, then those are very helpful to locate the cause of this problem. Are there any methods to get log messages or debugging messages in this situation?
Looks like a video issue, try removing

i915.enable_initial_modeset=1 i915.avail_planes_per_pipe=0x000C00

from kernel cmdline.

--
Regards,

Miguel Bernal Marin Open Source Technology Center
https://clearlinux.org Intel Corporation


Shin, Jun-Sik
 

ClearLinux SOS and Ubuntu SOS still have the same issue, even though I removed "i915.enable_initial_modeset=1 i915.avail_planes_per_pipe=0x000C00" from grub entry configuration file. (acrn.conf, 40_custom)
At below, I attached the grub configuration files used for my test.

/etc/grub.d/40_custom (Ubuntu as SOS)
menuentry 'ACRN ubuntu SOS' {
        recordfail
        insmod gzio
        insmod part_gpt
        insmod ext2
        linux  /boot/acrn/org.clearlinux.pk414-sos.4.14.71-110 pci_devices_ignore=(0:18:1)  console=tty0 console=ttyS0 i915.nuclear_pageflip=1 root=PARTUUID=a3bd9401-910c-4256-9150-6da0687d6a89 rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.tsd_init=7 i915.tsd_delay=2000 i915.domain_plane_owners=0x011111110000 i915.enable_guc_loading=0 i915.enable_guc_submission=0 i915.enable_preemption=1 i915.context_priority_mode=2 i915.enable_gvt=1 hvlog=2M@0x1FE00000
}
 
/boot/loader/entries/acrn.conf (ClearLinux as SOS)
title The ACRN Service OS
linux   /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.71-110
options pci_devices_ignore=(0:18:1) console=tty0 console=ttyS0 i915.nuclear_pageflip=1 root=/dev/sda3 rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.tsd_init=7 i915.tsd_delay=2000 i915.domain_plane_owners=0x011111110000 i915.enable_guc_loading=0 i915.enable_guc_submission=0 i915.enable_preemption=1 i915.context_priority_mode=2 i915.enable_gvt=1 i915.enable_guc=0 hvlog=2M@0x1FE00000 hugepagesz=1G hugepages=2


tk.kim@...
 

Hello. This is Taekyoung Kim. 
I have same problem with booting both Clear Linux SOS and Ubuntu SOS. 
My machine shows black screen and reboots automatically.

I have a question regarding this issue.
- Is there H/W dependency in the grub command? What are they? How can I get rid of dependency to boot SOS?

The environment I am using is below (Ubuntu as SOS)
H/W : NUC6i5SYH 
Hypervisor : ACRN Hypervisor v0.3
SOS Kernel : org.clearlinux.pk414-sos.4.14.74-115

/etc/grub.d/40_custom 
menuentry 'ACRN ubuntu SOS' {
        recordfail
        insmod gzio
        insmod part_gpt
        insmod ext2
        linux  /boot/acrn/org.clearlinux.pk414-sos.4.14.74-115 pci_devices_ignore=(0:18:1)  console=tty0 console=ttyS0 i915.nuclear_pageflip=1 root=PARTUUID=b668057b-46ff-44fb-8ba0-624550abad2b rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.tsd_init=7 i915.tsd_delay=2000 i915.domain_plane_owners=0x011111110000 i915.enable_guc_loading=0 i915.enable_guc_submission=0 i915.enable_preemption=1 i915.context_priority_mode=2 i915.enable_gvt=1 hvlog=2M@0x1FE00000
}

Any comments and thoughts would be helpful.

Thanks
Taekyoung Kim


Yang, Ailin
 

Showing “Black screen” only is hard to know your issue exactly.

Also NUC6i5SYH is not official supported hardware,  we didn’t run test with it.

 

But can you check below 3 points to narrow down your issue first:

1,  is your acrn.efi loaded by EFI firmware correctly?

     You can run “ sudo efibootmgr –v” to check if the “ACRN Hypervisor” is first boot option,

    Also you can enter the EFI firmware at boot (using F10) to select “ACRN Hypervisor”

2, check you SOS booted up successfully with right kernel and kernel command line?

You should able to login from remote ssh Client if the OS boot up successfully

3, check if ACRN hypervisor is running properly with “$ dmesg | grep ACRN”

     The right output is:

     $ dmesg | grep ACRN

   [    0.000000] Hypervisor detected: ACRN

   [    1.687128] ACRNTrace: acrn_trace_init, cpu_num 4

   [    1.693129] ACRN HVLog: acrn_hvlog_init

 

 

Thanks

ailin

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Wednesday, November 21, 2018 2:51 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hello. This is Taekyoung Kim. 
I have same problem with booting both Clear Linux SOS and Ubuntu SOS. 
My machine shows black screen and reboots automatically.

I have a question regarding this issue.
- Is there H/W dependency in the grub command? What are they? How can I get rid of dependency to boot SOS?

The environment I am using is below (Ubuntu as SOS)
H/W : NUC6i5SYH 
Hypervisor : ACRN Hypervisor v0.3
SOS Kernel : org.clearlinux.pk414-sos.4.14.74-115

/etc/grub.d/40_custom 

menuentry 'ACRN ubuntu SOS' {

        recordfail

        insmod gzio

        insmod part_gpt

        insmod ext2

        linux  /boot/acrn/org.clearlinux.pk414-sos.4.14.74-115 pci_devices_ignore=(0:18:1)  console=tty0 console=ttyS0 i915.nuclear_pageflip=1 root=PARTUUID=b668057b-46ff-44fb-8ba0-624550abad2b rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.tsd_init=7 i915.tsd_delay=2000 i915.domain_plane_owners=0x011111110000 i915.enable_guc_loading=0 i915.enable_guc_submission=0 i915.enable_preemption=1 i915.context_priority_mode=2 i915.enable_gvt=1 hvlog=2M@0x1FE00000

}

Any comments and thoughts would be helpful.

Thanks
Taekyoung Kim


tk.kim@...
 

Hello, Yang.
Thank you for the reply.

1,  is your acrn.efi loaded by EFI firmware correctly?

     You can run “ sudo efibootmgr –v” to check if the “ACRN Hypervisor” is first boot option,

    Also you can enter the EFI firmware at boot (using F10) to select “ACRN Hypervisor”

  -> [TK] ACRN ubuntu SOS appeared in boot list, but after selecting it, my hardware didn't show anything and rebooted in few seconds. 

    because of that I couldn't go further step 2 or 3 as you mentioned. 

I was wondering if the SOS kernel that I'm using(org.clearlinux.pk414-sos.4.14.74-115) has device dependency (might be CPU, graphic card and so on)

I think knowing which component or options are dependent on device would be helpful to porting ACRN hypervisor and Service OS


Yang, Ailin
 

The normal boot flow is:

EFI BIOS-> acrn.efi->grub->SOS kernel -> user land

ACRN ubuntu SOS appeared in boot list” doesn’t mean the grub must be loaded by acrn.efi.

This is why I asked you to check #1 point to make sure you first boot stage is correct.

 

By the way, you need to change org.clearlinux.pk414-sos.4.14.74-115 to org.clearlinux.iot-lts2018-sos.4.19.0-19

We have update kernel to 4.19 already if you are using ACRN v0.3 or latest.

 

Thanks

Ailin

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Wednesday, November 21, 2018 4:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hello, Yang.
Thank you for the reply.

1,  is your acrn.efi loaded by EFI firmware correctly?

     You can run “ sudo efibootmgr –v” to check if the “ACRN Hypervisor” is first boot option,

    Also you can enter the EFI firmware at boot (using F10) to select “ACRN Hypervisor”

  -> [TK] ACRN ubuntu SOS appeared in boot list, but after selecting it, my hardware didn't show anything and rebooted in few seconds. 

    because of that I couldn't go further step 2 or 3 as you mentioned. 

I was wondering if the SOS kernel that I'm using(org.clearlinux.pk414-sos.4.14.74-115) has device dependency (might be CPU, graphic card and so on)

I think knowing which component or options are dependent on device would be helpful to porting ACRN hypervisor and Service OS


tk.kim@...
 

Hi. 
I had changed SOS kernel image to org.clearlinux.iot-lts2018-sos.4.19.0-19.
and checked ACRN hypervisor is first boot option with command 'sudo efibootmgr –v'.
Also, I did 
select “ACRN Hypervisor” using F10 key.  Then  select 'ACRN ubuntu SOS' in the grub menu. 
However, the same issue happens. (black screen and reboot) 

Is there anything I'm missing something or doing wrong, please let me know.

Thanks 

Taekyoung Kim 


Geoffroy Van Cutsem
 

Are you sure you have the kernel in the right place, your grub entry indicated: /boot/acrn/org.clearlinux.pk414-sos.4.14.74-115 but the kernel is usually placed in a different folder (and under a slightly different name), e.g. /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.57-69

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Wednesday, November 21, 2018 10:55 AM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi. 
I had changed SOS kernel image to org.clearlinux.iot-lts2018-sos.4.19.0-19.
and checked ACRN hypervisor is first boot option with command '
sudo efibootmgr –v'.
Also, I did select “ACRN Hypervisor” using F10 key.  Then  select 'ACRN ubuntu SOS' in the grub menu. 

However, the same issue happens. (black screen and reboot) 

Is there anything I'm missing something or doing wrong, please let me know.

Thanks 

Taekyoung Kim 


tk.kim@...
 
Edited

Hi Geoffroy. 
I'm little bit confused now.

Two things need to be clear. 

1. SOS kernel image file 
  - What kernel image should I use? with ACRN Hypervisor v0.3
  - org.clearlinux.pk414-sos.4.14.57-69 or org.clearlinux.iot-lts2018-sos.4.19.0-19 ?

2. SOS kernel image location 
  - According to the guide page(https://projectacrn.github.io/latest/tutorials/using_ubuntu_as_sos.html)
The SOS kernel copies to '/boot/acrn/',
sudo cp ~/kernel-build/usr/lib/kernel/org.clearlinux.pk414-sos.4.14.57-69 /boot/acrn/

but in the '/etc/grub.d/40_cutom file says
'linux  /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.57-69'
  - Is this ok? For me, it looks like two different partitions, '/boot/acrn/' is /dev/sda3, and '/EFI/org.clearlinux/' is /dev/sda1.

Thanks
Taekyoung Kim


Geoffroy Van Cutsem
 

Hi Taekyoung,

 

With the latest release (v0.3), you should be using org.clearlinux.iot-lts2018-sos.4.19.0-19, we’re in the process of updating the Getting Started Guide to reflect these changes – see https://github.com/projectacrn/acrn-hypervisor/pull/1859. We will also need to update our tutorial on how to use Ubuntu as the Service OS as it is, as you’ve noted, out of date in a few places. I have filed a ticket to track that task: https://github.com/projectacrn/acrn-hypervisor/issues/1878

 

You’re absolutely right about the discrepancy about the kernel location. Ultimately, you can put it pretty much where you want on the ESP (/dev/sda1, the EFI System Partition), the key being to use the right path in your grub conf file. IMHO, the most logical location for that kernel would be under /EFI/Ubuntu

 

Thanks,

Geoffroy

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Thursday, November 22, 2018 11:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

[Edited Message Follows]

Hi Geoffroy. 
I'm little bit confused now.

Two things need to be clear. 

1. SOS kernel image file 
  - What kernel image should I use? with ACRN Hypervisor v0.3
  - org.clearlinux.pk414-sos.4.14.57-69 or org.clearlinux.iot-lts2018-sos.4.19.0-19 ?

2. SOS kernel image location 
  - According to the guide page(https://projectacrn.github.io/latest/tutorials/using_ubuntu_as_sos.html)
The SOS kernel copies to '/boot/acrn/',
sudo cp ~/kernel-build/usr/lib/kernel/org.clearlinux.pk414-sos.4.14.57-69 /boot/acrn/

but in the '/etc/grub.d/40_cutom file says
'linux  /EFI/org.clearlinux/kernel-org.clearlinux.pk414-sos.4.14.57-69'
  - Is this ok? For me, it looks like two different partitions, '/boot/acrn/' is /dev/sda3, and '/EFI/org.clearlinux/' is /dev/sda1.

Thanks
Taekyoung Kim


tk.kim@...
 

Hi Geoffroy.
Thank you for the clarification.
What I'm trying to do is porting ACRN hypervisor and SOS to my own device. (NUC6i5SYH)
I hope to find some hints from links that you had mentioned. 

Thanks :)
Taekyoung Kim. 


Geoffroy Van Cutsem
 

Sounds great, good luck and do let us know if you are successful!

 

If possible, I would advice that you set up ACRN on another platform that is supported, to make sure you are very familiar and comfortable with the process. Please note that it will be hard to debug problems if you don’t have a serial port (as it’s the only way you can get debug info from the hypervisor).

 

Thanks,

Geoffroy

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Friday, November 23, 2018 2:00 AM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi Geoffroy.
Thank you for the clarification.
What I'm trying to do is porting ACRN hypervisor and SOS to my own device. (NUC6i5SYH)
I hope to find some hints from links that you had mentioned. 

Thanks :)
Taekyoung Kim. 


tk.kim@...
 
Edited

Hi Geoffroy. 

In your tested device (NUC6CAYH), how to set a serial port to debug the hypervisor? 
Can you show me an example? 

Did you put the serial port address in the efibootmgr command? (uart=xxxx)
efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/sda -p 1 -L "ACRN NUC Hypervisor" \
      -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=disabled"

Thank you.
Taekyoung Kim


tk.kim@...
 

Hi 
I'm using a mmio for the uart setting and I show shell prompt.  
However, I'm not sure this is the SOS or the host clear linux. 

dmesg | grep ACRN  gives me below logs...

ACRNTrace : acrn_trace : not support acrn hypervisor!
ACRN HVLog : acrn_hvlog_init
ACRN HVLog : Failed to setup acrn_hvlog_cur_0, errno -19

What is error number 19, and do you have any idea to clarify root cause?
(I couldn't find this messages from the hypervisor source codes)

Thanks 
Taekyoung Kim


Yang, Ailin
 

Check kernel command line to see which kernel loaded?

cat /proc/cmdline

Below log shows ACRN hypervisor isn’t launched successfully, and no hv log

 

 

 

Thanks

Ailin

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Tuesday, November 27, 2018 4:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi 
I'm using a mmio for the uart setting and I show shell prompt.  
However, I'm not sure this is the SOS or the host clear linux. 

dmesg | grep ACRN  gives me below logs...

ACRNTrace : acrn_trace : not support acrn hypervisor!
ACRN HVLog : acrn_hvlog_init
ACRN HVLog : Failed to setup acrn_hvlog_cur_0, errno -19

What is error number 19, and do you have any idea to clarify root cause?
(I couldn't find this messages from the hypervisor source codes)

Thanks 
Taekyoung Kim


Yang, Ailin
 

drivers/acrn/hvlog.c

174         if (x86_hyper_type != X86_HYPER_ACRN)

175                 return -ENODEV;

 

Thanks

Ailin

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Yang, Ailin
Sent: Tuesday, November 27, 2018 5:10 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Check kernel command line to see which kernel loaded?

cat /proc/cmdline

Below log shows ACRN hypervisor isn’t launched successfully, and no hv log

 

 

 

Thanks

Ailin

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Tuesday, November 27, 2018 4:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi 
I'm using a mmio for the uart setting and I show shell prompt.  
However, I'm not sure this is the SOS or the host clear linux. 

dmesg | grep ACRN  gives me below logs...

ACRNTrace : acrn_trace : not support acrn hypervisor!
ACRN HVLog : acrn_hvlog_init
ACRN HVLog : Failed to setup acrn_hvlog_cur_0, errno -19

What is error number 19, and do you have any idea to clarify root cause?
(I couldn't find this messages from the hypervisor source codes)

Thanks 
Taekyoung Kim


Geoffroy Van Cutsem
 

Ailin is correct the ACRN hypervisor is not running (for some reason).

 

There are three possibilities for the ‘uart=’ parameter:

- uart=disabled

- uart=mmio@0x9141e000

- uart=port@0x3F8

https://github.com/projectacrn/acrn-hypervisor/blob/master/hypervisor/bsp/uefi/cmdline.c#L18-L20

 

The best is to boot a standard OS off a USB stick (Clear, Ubuntu, …), check from dmesg the serial port settings: “dmesg | grep ttyS”, and use those.

 

HTH,

Geoffroy

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Yang, Ailin
Sent: Tuesday, November 27, 2018 10:15 AM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

drivers/acrn/hvlog.c

174         if (x86_hyper_type != X86_HYPER_ACRN)

175                 return -ENODEV;

 

Thanks

Ailin

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Yang, Ailin
Sent: Tuesday, November 27, 2018 5:10 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Check kernel command line to see which kernel loaded?

cat /proc/cmdline

Below log shows ACRN hypervisor isn’t launched successfully, and no hv log

 

 

 

Thanks

Ailin

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of tk.kim@...
Sent: Tuesday, November 27, 2018 4:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] Questions about ACRN's dependency and Other Linux Distro Supports

 

Hi 
I'm using a mmio for the uart setting and I show shell prompt.  
However, I'm not sure this is the SOS or the host clear linux. 

dmesg | grep ACRN  gives me below logs...

ACRNTrace : acrn_trace : not support acrn hypervisor!
ACRN HVLog : acrn_hvlog_init
ACRN HVLog : Failed to setup acrn_hvlog_cur_0, errno -19

What is error number 19, and do you have any idea to clarify root cause?
(I couldn't find this messages from the hypervisor source codes)

Thanks 
Taekyoung Kim