Date   

Re: Setting up ACRN on a new board with Apollo Lake

Yin, Fengwei <fengwei.yin@...>
 

Hi Dubravko,

On 2019/9/28 上午12:58, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi everyone,
We've been busy bringing up some new hardware so it took me some time to respond. We are very grateful for all the assistance.
I have uploaded the requested new logs to https://www.amazon.com/clouddrive/share/fu8tJayVkkSkFZdfbVj8v7t32SZwsC7Cw9KNjLVMzdO
Our board has an Intel I211 network controller. lspci_hv log is with ACRN running, lspci_nohv is without ACRN.
Associated kernel and interrupt logs are included too.
We could download, configure and build our own version of the Clear Linux kernel to add the network controller, just I was unable to find instructions what specifically we need to set so it works with ACRN? It looks to me that it uses a different kernel than the plain Clear Linux.
For SOS kernel, I suppose you could build it based on our acrn-kernel
with igb enabled.

Regarding serial port interrupts, I must admit I'm confused with all the port and interrupt settings; anyway I've followed the instructions and enabled console=ttyS2 (acrn log), and requested device 0.18.2/HSUART2 (efibootmgr log) because it's the only serial port on our board that "just works" (others need setting up some GPIOs first). With these settings, Clear Linux runs noticeably slow, something is obviously wrong.
If it simplifies things, we don't strictly need debug printouts from ACRN, we just want it to work.
It looks like the serial port is a little bit mess.
From dmesg_nohv.log, we could see following serial port:

[ 7.097307] printk: console [ttyS0] disabled
[ 7.097347] dw-apb-uart.5: ttyS0 at MMIO 0x91328000 (irq = 4, base_baud = 115200) is a 16550A
[ 7.097379] printk: console [ttyS0] enabled
[ 7.099613] probe of dw-apb-uart.5 returned 1 after 6946 usecs
[ 7.111867] dw-apb-uart.6: ttyS1 at MMIO 0x91326000 (irq = 5, base_baud = 115200) is a 16550A
[ 7.112334] probe of dw-apb-uart.6 returned 1 after 1479 usecs
[ 7.126345] dw-apb-uart.7: ttyS2 at MMIO 0x91324000 (irq = 6, base_baud = 115200) is a 16550A

We could see ttyS0 is used as kernel console. This could be confirmed
by interrupt_nohv.log as well:
4: 117 0 0 0 IR-IO-APIC 4-fasteoi idma64.5, ttyS0

But, you said the ttyS2 is only serial port on the board. Kind of
confuse here. So are you using ttyS2 or ttyS0 as kernel console without
HV?

With HV, from efibootmgr.txt, we could see you are 0.18.2 as HV console
which should be mapped to ttyS2. So I suppose you could get workable HV
console (let me know if I am wrong).

But you have wrong console defined for SOS. Let me step back and explain
more. With HV, the ttyS2 is hold in HV and HV emulates a virtual serial
port to SOS. From dmesg_hv.log, we could see:
[ 0.985360] serial8250: ttyS0 at I/O 0x3f8 (irq = 6, base_baud = 115200) is a 16550A


This is emulated serial port IO base = 0x3f8 and irq = 6. So I suppose
you should change the SOS kernel cmd line to use ttyS0 instead of ttyS2
for SOS. Again from dmesg_hv.log, we could see you still use ttyS2 as
sos kernel console (which is NOT correct):
[ 0.000000] Command line: console=tty0 console=ttyS2 root=PARTUUID=1bbb889b-e284-4d16-a1d7-79ef02ed69a3 rw rootwait ignore_loglevel no_timer_check consoleblank=0 i915.nuclear_pageflip=1 i915.avail_planes_per_pipe=0x01010F i915.domain_plane_owners=0x011111110000 i915.enable_gvt=1 i915.enable_guc=0


So could you please change the SOS console from ttyS2 to ttyS0? Suppose
you should change acrn.conf:
console=ttyS2 -> console=ttyS0


Another strange thing is:
From dmesg_hv.log, we could see following message:
[ 1.063019] dw-apb-uart.5: ttyS1 at MMIO 0x91328000 (irq = 4, base_baud = 115200) is a 16550A
[ 1.065287] idma64 idma64.6: Found Intel integrated DMA 64-bit
[ 1.066333] console [ttyS2] disabled
[ 1.066670] dw-apb-uart.6: ttyS2 at MMIO 0x91326000 (irq = 5, base_baud = 115200) is a 16550A
[ 1.314086] mmcblk1: p1 p2 p3
[ 1.666493] console [ttyS2] enabled
[ 1.770323] rpmb_mux:rpmb_key_get: Failed to get boot_params from the command line!

[ 2.036999] dw-apb-uart.7: ttyS3 at MMIO 0x91324000 (irq = 6, base_baud = 115200) is a 16550A

This ttyS3 should match the ttyS2 without HV (ttyS0 of SOS is emulated
serial port). Suppose if this device is hold by HV, we should hide it
from SOS. I will check how this happen.

In summary, could you please try:
1. enable igb driver in your sos kernel to see whether the network could
work or not.
2. Change the sos kernel console from ttyS2 to ttyS0


I will check why the ttyS3 is shown in SOS.

Regards
Yin, Fengwei

Other configuration files I've mentioned earlier are Clear-linux-iot...conf and Clear-linux-native...conf. They must have been provided by Clear Linux and ACRN installations. I don't have a problem with them, just I've wanted to be sure, since they are so different from acrn.conf, is acrn.conf missing something?
Regarding IPv6 stuff, I've included UP2 ifconfig log. This is with ACRN running; network and everything else works fine here.
All the tests/logs, except for the UP2, were made yesterday and today using newest Clear Linux and newest ACRN, rebuilt in release mode, with SOS_COM1_IRQ changed to 6 as instructed. All our issues are still present.
Best regards,
Dubravko
------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
*Sent:* Monday, September 16, 2019 7:42 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Dubravko,
The UP2 board uses realtek network controller. I believe the reason
that your network can't work is the network controller is not enabled
in SOS kernel. Could you please find out the network controller for
your board? Or share the output of "lspci -k" with/without hv. Thanks.
Regards
Yin, Fengwei
On 2019/9/16 上午10:48, Yin, Fengwei wrote:
Hi Dubravko,
On 2019/9/13 上午1:01, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Geoffroy,

I have already tried with uart=disabled, and repeated the test now; everything remains the same, all the issues are present.

SSH works when not running in the hypervisor.

I'm using the default "clear" user from projectacrn.github.io tutorial.

When network doesn't work, I'm not even getting the IP address (we have a DHCP server in our LAN, it's otherwise working fine).
I only see following message from your sos kernel log:
[    1.200862] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[    1.201695] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
And there is no interrupt for network chip in interrupt dump.
On my MRB (I don't have UP2 env setup. Will check UP2 info later):
I could see following interrupt info:
   126:         40          0          0          0   PCI-MSI 245760-edge        mei_me
   127:          0          0          0          0   PCI-MSI 294912-edge        ahci[0000:00:12.0]
   128:          0          1          0          0   PCI-MSI 1048576-edge      ens4
   129:          0          0      38006          0   PCI-MSI 1048577-edge      ens4-TxRx-0
   130:          0          0          0      17499   PCI-MSI 1048578-edge      ens4-TxRx-1
   131:      15357          0          0          0   PCI-MSI 1048579-edge      ens4-TxRx-2
   132:          0      32620          0          0   PCI-MSI 1048580-edge      ens4-TxRx-3
   133:          0          0          0          0   PCI-MSI 344064-edge        xhci_hcd
   135:          0          0          0          4   PCI-MSI 49152-edge        intel-ipu4
And following info in kernel log:
[    0.789202] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
[    0.789203] e100: Copyright(c) 1999-2006 Intel Corporation
[    0.789237] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    0.789238] igb: Copyright (c) 2007-2014 Intel Corporation.
[    0.818600] pps pps0: new PPS source ptp0
[    0.818607] igb 0000:02:00.0: added PHC on eth0
[    0.818609] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
[    0.818612] igb 0000:02:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 98:4f:ee:13:ce:43
[    0.818653] igb 0000:02:00.0: eth0: PBA No: 000300-000
[    0.818655] igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[    0.818711] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[    0.818713] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
Can you also share the kernel/interrupt dump when there is no HV?
Thanks.
Regards
Yin, Fengwei


I have uploaded new dmesg and ifconfig logs to https://www.amazon.com/clouddrive/share/3wZ8oRCroPr9wpfcl60XAJov69Do8Hg5RiX1GRRpYRI,
76 kB RAR archive:

  * log_hyper.log files are with hypervisor running (resulting in all
    the problems, including no network)
  * 'hyper kernel no hyper' files are with regular non-hypervisor boot,
    plus the same kernel that hypervisor is using
    (lts2018-sos-4.19.64-76) - less problems e.g. terminal works, but
    still no network
  * 'native kernel' files are with regular non-hypervisor boot, and
    native-something kernel - no problems and network works; this is the
    combination I'm using when SSH works

The date is always off by three days, it doesn't seem to affect anything: with hypervisor there are problems, without hypervisor there aren't. I'll fix it, just in case.

It looks like maybe the hypervisor is actually working fine all the time, but the kernel it is loading is somehow misconfigured so it doesn't initialize network interface(s) and user account(s)?

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Geoffroy Van Cutsem via Lists.Projectacrn.Org <geoffroy.vancutsem=intel.com@...>
*Sent:* Thursday, September 12, 2019 5:46 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko,

*From:*acrn-users@... <acrn-users@...> *On Behalf Of *Dubravko Moravski | Exor Embedded S.r.l.
*Sent:* Thursday, September 12, 2019 5:15 PM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi everyone,

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

[Geoffroy] A few questions and one suggestion:

* Have you tried to start the ACRN hypervisor with “uart=disabled”? (see https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition).
A quick test if you have an EFI shell on your platform is to run “acrn.efi uart=disabled” from it.

* Does SSH work when you are not running in the hypervisor?

* Which user are you using when trying to connect over SSH? (note that root is blocked by default)

* When you say “networking is not working in general”, does that mean you are not even getting an IP address?

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo
(ignore the date, I don't know why it's three days in the past).

[Geoffroy] Is the date always off by three days (regardless of whether you’re running in the hypervisor or not)? I don’t know if that is the cause but dates in the past can cause problems with certificates (and hence something to look at as a potential issue related to SSH).

I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

These all work when not running with the hypervisor.

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3,
35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@... <mailto:acrn-users@...>
<acrn-users@... <mailto:acrn-users@...>> on behalf of Yin, Fengwei
via Lists.Projectacrn.Org <fengwei.yin=intel.com@... <mailto:fengwei.yin=intel.com@...>>
*Sent:* Thursday, September 12, 2019 3:21 PM
*To:* acrn-users@... <mailto:acrn-users@...>
<acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko,


*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:



+38 512455659 <tel:+38%20512455659>  m: +38 5915402413 <tel:+38%205915402413>

a:



Slavonska avenija, 50, Zagreb, Croatia, 10000

w:



exorint.com <https://exorint.com/>



  Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>




  Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,

We haven't observed any issues on the UP2 either, except one, screen updates been erratic and mouse pointer jumping around.

We've just followed instructions from https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
automated setup with some manual fine tuning.
I guess since we are using the same software and UP2 works (mostly) fine, the causes of our problems must be either different hardware or BIOS. But the trouble is figuring out what exactly.

All mentioned issues are for the Service OS. We can't even install/launch Guest OS because the Terminal no longer works after we restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei


Now that the memory log issue is fixed, could we use that to get additional debug information? If yes, how can I save that information for later? (We managed to create some scripts so we can save data like dmesg even when terminal can't be invoked.)

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...>> on behalf of Liu, Fuzhong via
Lists.Projectacrn.Org <fuzhong.liu=intel.com@...
<mailto:fuzhong.liu=intel.com@...>>
*Sent:* Thursday, September 12, 2019 11:47 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
<acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

No observed the issue on the UP2 board in our lab.

Need detail steps to reproduce.

BR.

Fuzhong

*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van
Cutsem
*Sent:* Thursday, September 12, 2019 4:42 PM
*To:* acrn-users@... <mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Have you observed the same problems on the UP2 board?

Thanks,

Geoffroy

*From:*acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...%20%0b>>
<mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
*Sent:* Thursday, September 12, 2019 7:15 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue

BR.

Fuzhong

*From:*acrn-users@... <mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Dubravko
Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 8:01 PM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Yin Fenghwei,

Thank you for replying.

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17


kB zip file, containing 4 text files). The zip archive contains:

   * dmesg output from SOS
   * /proc/interrupts from SOS
   * loader.conf from EFI partition
   * acrn.conf (here I've verified that root UUID is correct, and I've
     had to delete settings related to hypervisor memory log because
     otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@... <mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>>
<mailto:acrn-users@...>> on behalf of Yin, Fengwei via
Lists.Projectacrn.Org <fengwei.yin=intel.com@...
<mailto:fengwei.yin=intel.com@...%20%0b>>
<mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>>
<mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,

I work for Exor Embedded and we are developing a new board with an Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed, following the instructions on https://projectacrn.github.io/latest/index.htmlI've managed to get it
running, dmesg among other things says:

Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.

However, it's not really usable. We have following issues:

   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual

When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same kernel with and without hypervisor, but so far we were unable to figure out if any of these differences is significant.

We would appreciate if you have any hints for us, either what is exactly our problem, or at least how we should continue to debug the system.
Could you share:
     1. ACRN last git commit hash
     2. dmesg output in your SOS
     3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei


Best regards,
Dubravko Moravski



*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659
<tel:+38%20512455659%20%3ctel:+38%20512455659>>>  m:+38 5915402413 <tel:+38
<tel:+38%20%0b>> 5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>



   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:


+38 512455659 <tel:+38%20512455659>  m: +38 5915402413
<tel:+38%205915402413>

a:


Slavonska avenija, 50, Zagreb, Croatia, 10000

w:


exorint.com <https://exorint.com/>


imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg


imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima

di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>



   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>



Re: Setting up ACRN on a new board with Apollo Lake

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi everyone,

We've been busy bringing up some new hardware so it took me some time to respond. We are very grateful for all the assistance.


Our board has an Intel I211 network controller. lspci_hv log is with ACRN running, lspci_nohv is without ACRN.
Associated kernel and interrupt logs are included too.
We could download, configure and build our own version of the Clear Linux kernel to add the network controller, just I was unable to find instructions what specifically we need to set so it works with ACRN? It looks to me that it uses a different kernel than the plain Clear Linux.

Regarding serial port interrupts, I must admit I'm confused with all the port and interrupt settings; anyway I've followed the instructions and enabled console=ttyS2 (acrn log), and requested device 0.18.2/HSUART2 (efibootmgr log) because it's the only serial port on our board that "just works" (others need setting up some GPIOs first). With these settings, Clear Linux runs noticeably slow, something is obviously wrong.
If it simplifies things, we don't strictly need debug printouts from ACRN, we just want it to work.

Other configuration files I've mentioned earlier are Clear-linux-iot...conf and Clear-linux-native...conf. They must have been provided by Clear Linux and ACRN installations. I don't have a problem with them, just I've wanted to be sure, since they are so different from acrn.conf, is acrn.conf missing something?

Regarding IPv6 stuff, I've included UP2 ifconfig log. This is with ACRN running; network and everything else works fine here.

All the tests/logs, except for the UP2, were made yesterday and today using newest Clear Linux and newest ACRN, rebuilt in release mode, with SOS_COM1_IRQ changed to 6 as instructed. All our issues are still present.

Best regards,
Dubravko


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Monday, September 16, 2019 7:42 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
 
Hi Dubravko,
The UP2 board uses realtek network controller. I believe the reason
that your network can't work is the network controller is not enabled
in SOS kernel. Could you please find out the network controller for
your board? Or share the output of "lspci -k" with/without hv. Thanks.

Regards
Yin, Fengwei

On 2019/9/16 上午10:48, Yin, Fengwei wrote:
> Hi Dubravko,
>
> On 2019/9/13 上午1:01, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hi Geoffroy,
>>
>> I have already tried with uart=disabled, and repeated the test now;
>> everything remains the same, all the issues are present.
>>
>> SSH works when not running in the hypervisor.
>>
>> I'm using the default "clear" user from projectacrn.github.io tutorial.
>>
>> When network doesn't work, I'm not even getting the IP address (we
>> have a DHCP server in our LAN, it's otherwise working fine).
> I only see following message from your sos kernel log:
> [    1.200862] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function
> Network Driver - version 4.1.0-k
> [    1.201695] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
>
> And there is no interrupt for network chip in interrupt dump.
>
> On my MRB (I don't have UP2 env setup. Will check UP2 info later):
> I could see following interrupt info:
>   126:         40          0          0          0   PCI-MSI 245760-edge
>       mei_me
>   127:          0          0          0          0   PCI-MSI 294912-edge
>       ahci[0000:00:12.0]
>   128:          0          1          0          0   PCI-MSI
> 1048576-edge      ens4
>   129:          0          0      38006          0   PCI-MSI
> 1048577-edge      ens4-TxRx-0
>   130:          0          0          0      17499   PCI-MSI
> 1048578-edge      ens4-TxRx-1
>   131:      15357          0          0          0   PCI-MSI
> 1048579-edge      ens4-TxRx-2
>   132:          0      32620          0          0   PCI-MSI
> 1048580-edge      ens4-TxRx-3
>   133:          0          0          0          0   PCI-MSI 344064-edge
>       xhci_hcd
>   135:          0          0          0          4   PCI-MSI 49152-edge
>       intel-ipu4
>
> And following info in kernel log:
> [    0.789202] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
> [    0.789203] e100: Copyright(c) 1999-2006 Intel Corporation
> [    0.789237] igb: Intel(R) Gigabit Ethernet Network Driver - version
> 5.4.0-k
> [    0.789238] igb: Copyright (c) 2007-2014 Intel Corporation.
> [    0.818600] pps pps0: new PPS source ptp0
> [    0.818607] igb 0000:02:00.0: added PHC on eth0
> [    0.818609] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network
> Connection
> [    0.818612] igb 0000:02:00.0: eth0: (PCIe:2.5Gb/s:Width x1)
> 98:4f:ee:13:ce:43
> [    0.818653] igb 0000:02:00.0: eth0: PBA No: 000300-000
> [    0.818655] igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s),
> 4 tx queue(s)
> [    0.818711] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function
> Network Driver - version 4.1.0-k
> [    0.818713] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
>
> Can you also share the kernel/interrupt dump when there is no HV?
> Thanks.
>
> Regards
> Yin, Fengwei
>
>>
>> I have uploaded new dmesg and ifconfig logs to
>> https://www.amazon.com/clouddrive/share/3wZ8oRCroPr9wpfcl60XAJov69Do8Hg5RiX1GRRpYRI,
>> 76 kB RAR archive:
>>
>>   * log_hyper.log files are with hypervisor running (resulting in all
>>     the problems, including no network)
>>   * 'hyper kernel no hyper' files are with regular non-hypervisor boot,
>>     plus the same kernel that hypervisor is using
>>     (lts2018-sos-4.19.64-76) - less problems e.g. terminal works, but
>>     still no network
>>   * 'native kernel' files are with regular non-hypervisor boot, and
>>     native-something kernel - no problems and network works; this is the
>>     combination I'm using when SSH works
>>
>> The date is always off by three days, it doesn't seem to affect
>> anything: with hypervisor there are problems, without hypervisor there
>> aren't. I'll fix it, just in case.
>>
>> It looks like maybe the hypervisor is actually working fine all the
>> time, but the kernel it is loading is somehow misconfigured so it
>> doesn't initialize network interface(s) and user account(s)?
>>
>> Best regards,
>> Dubravko
>>
>>
>> ------------------------------------------------------------------------
>> *From:* acrn-users@...
>> <acrn-users@...> on behalf of Geoffroy Van Cutsem
>> via Lists.Projectacrn.Org
>> <geoffroy.vancutsem=intel.com@...>
>> *Sent:* Thursday, September 12, 2019 5:46 PM
>> *To:* acrn-users@... <acrn-users@...>
>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo
>> Lake
>>
>> Hi Dubravko,
>>
>> *From:*acrn-users@...
>> <acrn-users@...> *On Behalf Of *Dubravko Moravski |
>> Exor Embedded S.r.l.
>> *Sent:* Thursday, September 12, 2019 5:15 PM
>> *To:* acrn-users@...
>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo
>> Lake
>>
>> Hi everyone,
>>
>> SOS console/terminal can't be opened at all; the window actually opens
>> but it says "There was an error creating a child process for this
>> terminal. Failed to open PTY: Permission denied" and we can't do
>> anything. It is the same if we use launcher on the left side on the
>> screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in
>> general) so we can't connect remotely either.
>>
>> [Geoffroy] A few questions and one suggestion:
>>
>> * Have you tried to start the ACRN hypervisor with “uart=disabled”?
>> (see
>> https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition).
>> A quick test if you have an EFI shell on your platform is to run
>> “acrn.efi uart=disabled” from it.
>>
>> * Does SSH work when you are not running in the hypervisor?
>>
>> * Which user are you using when trying to connect over SSH? (note that
>> root is blocked by default)
>>
>> * When you say “networking is not working in general”, does that mean
>> you are not even getting an IP address?
>>
>> 200K JPG image uploaded to
>> https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo
>> (ignore the date, I don't know why it's three days in the past).
>>
>> [Geoffroy] Is the date always off by three days (regardless of whether
>> you’re running in the hypervisor or not)? I don’t know if that is the
>> cause but dates in the past can cause problems with certificates (and
>> hence something to look at as a potential issue related to SSH).
>>
>> I don't see why the console would have anything to do with the
>> hypervisor, but simply it doesn't work with it, and works without it,
>> so it's a symptom of some problem.
>>
>> These all work when not running with the hypervisor.
>>
>> Regarding ACPI and LPC, unfortunately we are not that familiar with
>> ACPI, I have uploaded our disassembled ACPI dump to
>> https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3,
>> 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.
>>
>> Best regards,
>>
>> Dubravko
>>
>> ------------------------------------------------------------------------
>>
>> *From:*acrn-users@...
>> <mailto:acrn-users@...>
>> <acrn-users@...
>> <mailto:acrn-users@...>> on behalf of Yin, Fengwei
>> via Lists.Projectacrn.Org <fengwei.yin=intel.com@...
>> <mailto:fengwei.yin=intel.com@...>>
>> *Sent:* Thursday, September 12, 2019 3:21 PM
>> *To:* acrn-users@...
>> <mailto:acrn-users@...>
>> <acrn-users@...
>> <mailto:acrn-users@...>>
>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo
>> Lake
>>
>> Hi Dubravko,
>>
>>
>> *Dubravko Moravski*
>>
>> /SW engineering/
>>
>> *Exor Embedded S.r.l.*
>>
>> p:
>>
>>
>>
>> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
>> <tel:+38%205915402413>
>>
>> a:
>>
>>
>>
>> Slavonska avenija, 50, Zagreb, Croatia, 10000
>>
>> w:
>>
>>
>>
>> exorint.com <https://exorint.com/>
>>
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38
>> 5915402413>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>

Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

>> On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>>> Hi,
>>>
>>> We haven't observed any issues on the UP2 either, except one, screen
>>> updates been erratic and mouse pointer jumping around.
>>>
>>> We've just followed instructions from
>>> https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
>>> automated setup with some manual fine tuning.
>>> I guess since we are using the same software and UP2 works (mostly)
>>> fine, the causes of our problems must be either different hardware or
>>> BIOS. But the trouble is figuring out what exactly.
>>>
>>> All mentioned issues are for the Service OS. We can't even
>>> install/launch Guest OS because the Terminal no longer works after we
>>> restart the board with ACRN.
>> We checked the log you shared and didn't see anything unusual. We had
>> one issue which could make serial console use polling mode. I suspected
>> you hit that issue. But from the interrupt info, everything is fine (
>> there will be irq flooding if we hit that issue).We need more info.
>>
>> When you say "Terminal no longer work", do you mean the SOS console
>> can't work any more? Or it could work for a while and then can't work?
>>
>> Regards
>> Yin, Fengwei
>>
>>>
>>> Now that the memory log issue is fixed, could we use that to get
>>> additional debug information? If yes, how can I save that information
>>> for later? (We managed to create some scripts so we can save data
>>> like dmesg even when terminal can't be invoked.)
>>>
>>> Best regards,
>>> Dubravko
>>>
>>>
>>> ------------------------------------------------------------------------
>>> *From:* acrn-users@...
>>> <mailto:acrn-users@...>
>>> <acrn-users@...
>> <mailto:acrn-users@...>> on behalf of Liu, Fuzhong via
>>> Lists.Projectacrn.Org <fuzhong.liu=intel.com@...
>> <mailto:fuzhong.liu=intel.com@...>>
>>> *Sent:* Thursday, September 12, 2019 11:47 AM
>>> *To:* acrn-users@...
>> <mailto:acrn-users@...>
>> <acrn-users@...
>> <mailto:acrn-users@...>>
>>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with
>>> Apollo Lake
>>>
>>> No observed the issue on the UP2 board in our lab.
>>>
>>> Need detail steps to reproduce.
>>>
>>> BR.
>>>
>>> Fuzhong
>>>
>>> *From:*acrn-users@...
>>> [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van
>>> Cutsem
>>> *Sent:* Thursday, September 12, 2019 4:42 PM
>>> *To:* acrn-users@...
>>> <mailto:acrn-users@...>
>>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with
>>> Apollo Lake
>>>
>>> Have you observed the same problems on the UP2 board?
>>>
>>> Thanks,
>>>
>>> Geoffroy
>>>
>>> *From:*acrn-users@...
>>> <mailto:acrn-users@...>
>>> <acrn-users@...
>> <mailto:acrn-users@...%20%0b>>
>> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
>>> *Sent:* Thursday, September 12, 2019 7:15 AM
>>> *To:* acrn-users@...
>>> <mailto:acrn-users@...>
>>> <mailto:acrn-users@...>
>>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with
>>> Apollo Lake
>>>
>>> Hi Dubravko
>>>
>>> All issues you mentioned  are for Service OS, right?
>>>
>>> Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
>>>
>>> BR.
>>>
>>> Fuzhong
>>>
>>> *From:*acrn-users@...
>>> <mailto:acrn-users@...>
>>> [mailto:acrn-users@...] *On Behalf Of *Dubravko
>>> Moravski | Exor Embedded S.r.l.
>>> *Sent:* Wednesday, September 11, 2019 8:01 PM
>>> *To:* acrn-users@...
>>> <mailto:acrn-users@...>
>>> <mailto:acrn-users@...>
>>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with
>>> Apollo Lake
>>>
>>> Hi Yin Fenghwei,
>>>
>>> Thank you for replying.
>>>
>>> ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I
>>> am getting identical behavior with ACRN that I've compiled, as well
>>> as with ACRN installed by Clear Linux (version id 30770).
>>>
>>> I'm not sure if this mailing lists allows attachments, so I've
>>> uploaded the requested files to
>>> https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
>>
>>
>>> kB zip file, containing 4 text files). The zip archive contains:
>>>
>>>    * dmesg output from SOS
>>>    * /proc/interrupts from SOS
>>>    * loader.conf from EFI partition
>>>    * acrn.conf (here I've verified that root UUID is correct, and I've
>>>      had to delete settings related to hypervisor memory log because
>>>      otherwise it crashed)
>>>
>>> Our board has 8 GB of memory and 64 GB MMC, and is similar to Up
>>> Squared and Leaf Hill CRB.
>>>
>>> Best regards,
>>>
>>> Dubravko
>>>
>>> ------------------------------------------------------------------------
>>>
>>> *From:*acrn-users@...
>>> <mailto:acrn-users@...><acrn-users@...
>>>
>> <mailto:acrn-users@...%20%0b>>
>> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
>>> Lists.Projectacrn.Org <fengwei.yin=intel.com@...
>> <mailto:fengwei.yin=intel.com@...%20%0b>>
>> <mailto:fengwei.yin=intel.com@...>>
>>> *Sent:* Wednesday, September 11, 2019 4:41 AM
>>> *To:* acrn-users@...
>>> <mailto:acrn-users@...>
>>> <mailto:acrn-users@...><acrn-users@...
>>>
>> <mailto:acrn-users@...%20%0b>>
>> <mailto:acrn-users@...>>
>>> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with
>>> Apollo Lake
>>>
>>> Hi,
>>>
>>> On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>>>> Hello everyone,
>>>>
>>>> I work for Exor Embedded and we are developing a new board with an
>>>> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
>>>> following the instructions on
>>>> https://projectacrn.github.io/latest/index.htmlI've managed to get it
>>>> running, dmesg among other things says:
>>>>
>>>> Hypervisor detected: ACRN
>>>> ...
>>>> ACRNTrace: Initialized acrn trace module with 4 cpu
>>>> ...
>>>> systemd[1]: Detected virtualization acrn.
>>>>
>>>> However, it's not really usable. We have following issues:
>>>>
>>>>    * mouse cursor jumps around in a weird way, like there is some
>>>> problem
>>>>      with switching video buffers
>>>>    * we can't open the Terminal in any way, there's always a message:
>>>>      "There was an error creating a child process for this terminal.
>>>>      Failed to open PTY: Permission denied"
>>>>    * network doesn't work
>>>>    * everything works much slower than usual
>>>>
>>>> When not running with hypervisor, everything works.
>>>> We've noticed some differences between dmesgs when running the same
>>>> kernel with and without hypervisor, but so far we were unable to
>>>> figure out if any of these differences is significant.
>>>>
>>>> We would appreciate if you have any hints for us, either what is
>>>> exactly our problem, or at least how we should continue to debug the
>>>> system.
>>> Could you share:
>>>      1. ACRN last git commit hash
>>>      2. dmesg output in your SOS
>>>      3. the output of cat /proc/interrupt in your SOS
>>>
>>> Regards
>>> Yin, Fengwei
>>>
>>>>
>>>> Best regards,
>>>> Dubravko Moravski
>>>>
>>>>
>>>>
>>>> *Dubravko Moravski*
>>>> /SW engineering/
>>>> *Exor Embedded S.r.l.*
>>>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659
>> <tel:+38%20512455659%20%3ctel:+38%20512455659>>>  m:+38 5915402413
>> <tel:+38
>> <tel:+38%20%0b>> 5915402413 <tel:+38%205915402413>>
>>>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>>>> w:     exorint.com <https://exorint.com/>
>>>>
>>>>
>>>>
>>>>    Prima di stampare pensa ai costi ambientali. Please consider the
>>>> environment before printing this email.
>>>>
>>>> Privacy <https://www.exorint.com/it/privacy>
>>>>
>>>
>>> *Dubravko Moravski*
>>>
>>> /SW engineering/
>>>
>>> *Exor Embedded S.r.l.*
>>>
>>> p:
>>>
>>>
>>> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
>>> <tel:+38%205915402413>
>>>
>>> a:
>>>
>>>
>>> Slavonska avenija, 50, Zagreb, Croatia, 10000
>>>
>>> w:
>>>
>>>
>>> exorint.com <https://exorint.com/>
>>>
>>>
>>> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
>>>
>>>
>>> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
>>>
>>> di stampare pensa ai costi ambientali. Please consider the
>>> environment before printing this email.
>>>
>>> Privacy <https://www.exorint.com/it/privacy>
>>>
>>>
>>> *Dubravko Moravski*
>>> /SW engineering/
>>> *Exor Embedded S.r.l.*
>>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38
>>> 5915402413 <tel:+38
>> 5915402413 <tel:+38%205915402413>>
>>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>>> w:     exorint.com <https://exorint.com/>
>>>
>>>
>>>
>>>    Prima di stampare pensa ai costi ambientali. Please consider the
>>> environment before printing this email.
>>>
>>> Privacy <https://www.exorint.com/it/privacy>
>>>
>>
>>
>>
>>
>
>
>




QNX / Page-Fault question

devin.acrn@...
 

Hello,

I'm completely new to ACRN (only having learned about it yesterday). Let me know if this is not the proper group to post this question to.

Should ACRN support any OS or does the OS require explicit support in order to work with ACRN? Or similarly, does ACRN require specific support for certain OSes?

As an example, should I be able to run QNX on the ACRN hypervisor, or would it/ACRN need some sort of support built-in?

I'm basing my experiment on the information from the following pages and trying to apply it to boot QNX:
https://projectacrn.github.io/latest/getting-started/apl-nuc.html
https://projectacrn.github.io/latest/tutorials/using_vxworks_as_uos.html

I have an EFI file named QNX.efi that I can boot on a NUC7i3DNB when booting on bare metal just fine. However when I try to boot this on the ACRN hypervisor I get a page fault error (which I'll copy below).

My launch file is based on the launch_vxworks.sh file but has a few changes, including the following for acrn-dm parameters. Is there anything obviously wrong with it?
acrn-dm \
  -A \
  -m $mem_size \
  -c $2 \
  -s 0:0,hostbridge \
  -s 1:0,lpc \
  -s 2,pci-gvt \
  -G "10 128 6" \
  -l com1,stdio \
  -s 5,virtio-console,@pty:pty_port \
  -s 6,virtio-hyper_dmabuf \
  -s 3,virtio-blk,./qnx.img \
  --virtio_poll 1000000 \
  --ovmf ./OVMF.fd \
  --lapic_pt \
  $vm_name

I can change the QNX.efi to a shell.efi, as an sanity check, and the EFI shell boots up just fine. If I try to launch the QNX.efi from within the EFI shell then I get the same page fault error.

The error that I'm getting is copied below. Where would this error be coming from? Would it be OVMF.fd, the ACRN hypervisor, or something else (like the QNX bootloader)?

!!!! X64 Exception Type - 0E(#PF - Page-Fault)  CPU Apic ID - 00000000 !!!!
ExceptionData - 0000000000000000  I:0 R:0 U:0 W:0 P:0 PK:0 S:0
RIP  - 0000000001400B64, CS  - 0000000000000038, RFLAGS - 0000000000011006
RAX  - 0000000000000000, RCX - 0000000000000001, RDX - 0000000000000800
RBX  - 0000000000000063, RSP - 00000000014009A0, RBP - 000000000141E5A0
RSI  - 0000000001400A40, RDI - 000000000141E5A0
R8   - 000000007FFA0000, R9  - 0000000000000000, R10 - 0000000000019000
R11  - 0000000001F94618, R12 - 00000000FFFFFFFC, R13 - 0000000001400A40
R14  - 00000000014009EF, R15 - 00000000FEE00000
DS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030
GS   - 0000000000000030, SS  - 0000000000000030
CR0  - 0000000080010033, CR2 - FFFFFFFFCBE02874, CR3 - 000000007FC01000
CR4  - 00000000000006A8, CR8 - 0000000000000000
DR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3  - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000007FBEEA98 0000000000000047, LDTR - 0000000000000000
IDTR - 000000007F914018 0000000000000FFF,   TR - 0000000000000000
FXSAVE_STATE - 0000000001400600
!!!! Can't find image information. !!!!

Any help/advice would be appreciated. Is what I'm attempting reasonable or should I not expect an OS like QNX to run on the ACRN hypervisor?

As a side question, are there build instructions for the ACRN fork of edk2 (https://github.com/projectacrn/acrn-edk2)? When I try building it I end up with an OVMF.fd file that's exactly the same size as when I build the regular edk2, which seems unlikely so I imagine that I'm doing something wrong. I found the following build instructions for edk2 (I'm using Ubuntu to build it): https://wiki.ubuntu.com/UEFI/EDK2. Would those instructions need any changes to build OVMF.fd from acrn-edk2?
 

Many thanks,
Devin


ACRN Project Technical Community Meeting (2019/7~2019/12): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

Notice: Join ACRN’s TCM meeting with WebEx till Nov. 2019
Due to Zoom’s connection issue in PRC mainland recently, we have to use WebEx to replace Zoom for ACRN TCM meeting temporarily. We’ll monitor Zoom’s availability closely and switch back to Zoom as it comes back to normal. Sorry for the inconvenience!
Cisco WebEx: https://intel.webex.com/intel/j.php?MTID=m682b81bf58c7ec875c32c469ff22116f  (More details is at the bottom of the meeting request)
 
 
Date: 9/25
Topic:  Implementation of GOP driver with AcrnGT
Description: This is the follow up presentation of “Design of GOP driver for GVT-g “ (https://drive.google.com/drive/folders/17M08UYe5Bsi2jVBd6RGVb-laao31nfwT ). The implementation is a solution to show Guest OS content by local display before Intel GPU driver loaded in Guest OS. The purpose of GOP driver is to show Guest Windows boot GUI based on framebuffer driver. We’ll talk implementation details in OVMF GOP driver  and the related changes in ACRN  and AcrnGT.
 
WW Topic Presenter Status
WW02 TPM2.0 virtualization in ACRN DENG, Wei 1/9
WW03 Polling mode Virtio and its advantage for RT VM DENG, Jie 1/16
WW04 Buffer sharing from UOS to SOS, HyperDMA usage LIU, Xinyun 1/23
WW05 USB HUB Virtualization WU, Xiaoguang 1/30
WW07 ACRN Device Model QoS Design LIU, Long 2/13
WW08 ACRN Debug Tips CHEN, Jason 2/20
WW09 GVT-g debug trace tool GONG, Zhipeng 2/27
WW10 Kata Container Architecture: First Steps with ACRN Dhanraj, Vijay  3/6
WW11 One ACRN hypervisor to support multi-platform WU, Xiangyang 3/13
WW12 Power button key mediator design in ACRN LIU, Yuan 3/20
WW13 Local APIC Virtualization Enhancement for Intel KBL platform LI, Fei 3/27
WW14 Safety VM Support YIN, FengWei 4/3
WW15 How to customize GPIO in ACRN LIU, Yuan 4/10
WW16 ACRN Cache QoS support based on CAT TAO, Yuhong 4/17
WW17 ACRN Real-Time measurement Methodology LI, Wilson 4/24
WW19 I2C Virtualization CHEN, Conghui 5/8
WW20 SGX Virtualization in ACRN WU, Binbin 5/15
WW21 Logger Improvement on acrn-dm CAO, Minggui 5/22
WW22 AcrnGT Virtual Display Deep Dive HE, Min 5/29
WW23 Local APIC Emulation and Pass-through Grandhi, Sainath 6/5
WW25
 
ACRN Functional Safety: Understanding and Mitigating Inter-VM Interference MAO, Junjie 6/19
WW26 Enable VwWorks as RTVM on ACRN FU, Kaige 6/26
WW27 ACRN 2.0 New Architecture Sharing - 1/2 REN, Jack 7/3
WW28 ACRN 2.0 New Architecture Sharing- 2/2 REN, Jack 7/10
WW29 How to enable open vSwitch on ACRN LIU, Yuan 7/17
WW30 ACRN Configuration Design Tool SUN, Victor 7/24
WW31 Design of GOP Driver for GVT-g HE, Min 7/31
WW32 On Dynamic Resource Allocation Mao, Junjie 8/7
WW33 ACRN virtual secure boot and key enrollment Wang, Kai 8/14
WW34 Split ACRN’s Device Model CHEN, Jason CJ 8/21
WW35 ACRN Continuous Integration System Introduction ZHANG, Wenling 8/28
WW36 BKC for ACRN RT and vmexit analysis YAN, Like 9/4
WW37 ACRN Windows As A Guest (WaaG) Overview CHEN, Jianjun 9/11
WW38 ACRN Fuzzing Test HUANG, Yonghua 9/18
WW39 Implementation of GOP driver with AcrnGT LIU, Xinyun 9/25
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
  • -- Do not delete or change any of the following text. --  
When it's time, join your Webex meeting here.
 
 
Meeting number (access code): 590 576 334
 
 
Join
 
 
 
 
Join by phone 
Tap to call in from a mobile device (attendees only) 
+1-210-795-1110 US Toll 
+1-866-662-9987 US Toll Free 
Global call-in numbers  |  Toll-free calling restrictions  
 
Join from a video system or application
Dial 590576334@... 
You can also dial 173.243.2.68 and enter your meeting number.  
 
 
 
  • Meeting Notes:
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 


Re: Setting up ACRN on a new board with Apollo Lake

Yin, Fengwei <fengwei.yin@...>
 

Hi Dubravko,
The UP2 board uses realtek network controller. I believe the reason
that your network can't work is the network controller is not enabled
in SOS kernel. Could you please find out the network controller for
your board? Or share the output of "lspci -k" with/without hv. Thanks.

Regards
Yin, Fengwei

On 2019/9/16 上午10:48, Yin, Fengwei wrote:
Hi Dubravko,
On 2019/9/13 上午1:01, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Geoffroy,

I have already tried with uart=disabled, and repeated the test now; everything remains the same, all the issues are present.

SSH works when not running in the hypervisor.

I'm using the default "clear" user from projectacrn.github.io tutorial.

When network doesn't work, I'm not even getting the IP address (we have a DHCP server in our LAN, it's otherwise working fine).
I only see following message from your sos kernel log:
[    1.200862] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[    1.201695] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
And there is no interrupt for network chip in interrupt dump.
On my MRB (I don't have UP2 env setup. Will check UP2 info later):
I could see following interrupt info:
 126:         40          0          0          0   PCI-MSI 245760-edge      mei_me
 127:          0          0          0          0   PCI-MSI 294912-edge      ahci[0000:00:12.0]
 128:          0          1          0          0   PCI-MSI 1048576-edge      ens4
 129:          0          0      38006          0   PCI-MSI 1048577-edge      ens4-TxRx-0
 130:          0          0          0      17499   PCI-MSI 1048578-edge      ens4-TxRx-1
 131:      15357          0          0          0   PCI-MSI 1048579-edge      ens4-TxRx-2
 132:          0      32620          0          0   PCI-MSI 1048580-edge      ens4-TxRx-3
 133:          0          0          0          0   PCI-MSI 344064-edge      xhci_hcd
 135:          0          0          0          4   PCI-MSI 49152-edge      intel-ipu4
And following info in kernel log:
[    0.789202] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
[    0.789203] e100: Copyright(c) 1999-2006 Intel Corporation
[    0.789237] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    0.789238] igb: Copyright (c) 2007-2014 Intel Corporation.
[    0.818600] pps pps0: new PPS source ptp0
[    0.818607] igb 0000:02:00.0: added PHC on eth0
[    0.818609] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
[    0.818612] igb 0000:02:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 98:4f:ee:13:ce:43
[    0.818653] igb 0000:02:00.0: eth0: PBA No: 000300-000
[    0.818655] igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[    0.818711] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[    0.818713] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
Can you also share the kernel/interrupt dump when there is no HV?
Thanks.
Regards
Yin, Fengwei


I have uploaded new dmesg and ifconfig logs to https://www.amazon.com/clouddrive/share/3wZ8oRCroPr9wpfcl60XAJov69Do8Hg5RiX1GRRpYRI, 76 kB RAR archive:

  * log_hyper.log files are with hypervisor running (resulting in all
    the problems, including no network)
  * 'hyper kernel no hyper' files are with regular non-hypervisor boot,
    plus the same kernel that hypervisor is using
    (lts2018-sos-4.19.64-76) - less problems e.g. terminal works, but
    still no network
  * 'native kernel' files are with regular non-hypervisor boot, and
    native-something kernel - no problems and network works; this is the
    combination I'm using when SSH works

The date is always off by three days, it doesn't seem to affect anything: with hypervisor there are problems, without hypervisor there aren't. I'll fix it, just in case.

It looks like maybe the hypervisor is actually working fine all the time, but the kernel it is loading is somehow misconfigured so it doesn't initialize network interface(s) and user account(s)?

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Geoffroy Van Cutsem via Lists.Projectacrn.Org <geoffroy.vancutsem=intel.com@...>
*Sent:* Thursday, September 12, 2019 5:46 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko,

*From:*acrn-users@... <acrn-users@...> *On Behalf Of *Dubravko Moravski | Exor Embedded S.r.l.
*Sent:* Thursday, September 12, 2019 5:15 PM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi everyone,

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

[Geoffroy] A few questions and one suggestion:

* Have you tried to start the ACRN hypervisor with “uart=disabled”? (see https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition). A quick test if you have an EFI shell on your platform is to run “acrn.efi uart=disabled” from it.

* Does SSH work when you are not running in the hypervisor?

* Which user are you using when trying to connect over SSH? (note that root is blocked by default)

* When you say “networking is not working in general”, does that mean you are not even getting an IP address?

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).

[Geoffroy] Is the date always off by three days (regardless of whether you’re running in the hypervisor or not)? I don’t know if that is the cause but dates in the past can cause problems with certificates (and hence something to look at as a potential issue related to SSH).

I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

These all work when not running with the hypervisor.

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@... <mailto:fengwei.yin=intel.com@...>>
*Sent:* Thursday, September 12, 2019 3:21 PM
*To:* acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko,


*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:



+38 512455659 <tel:+38%20512455659>  m: +38 5915402413 <tel:+38%205915402413>

a:



Slavonska avenija, 50, Zagreb, Croatia, 10000

w:



exorint.com <https://exorint.com/>



  Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>




  Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,

We haven't observed any issues on the UP2 either, except one, screen updates been erratic and mouse pointer jumping around.

We've just followed instructions from https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
automated setup with some manual fine tuning.
I guess since we are using the same software and UP2 works (mostly) fine, the causes of our problems must be either different hardware or BIOS. But the trouble is figuring out what exactly.

All mentioned issues are for the Service OS. We can't even install/launch Guest OS because the Terminal no longer works after we restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei


Now that the memory log issue is fixed, could we use that to get additional debug information? If yes, how can I save that information for later? (We managed to create some scripts so we can save data like dmesg even when terminal can't be invoked.)

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...>> on behalf of Liu, Fuzhong via
Lists.Projectacrn.Org <fuzhong.liu=intel.com@...
<mailto:fuzhong.liu=intel.com@...>>
*Sent:* Thursday, September 12, 2019 11:47 AM
*To:* acrn-users@...
<mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

No observed the issue on the UP2 board in our lab.

Need detail steps to reproduce.

BR.

Fuzhong

*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
*Sent:* Thursday, September 12, 2019 4:42 PM
*To:* acrn-users@... <mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Have you observed the same problems on the UP2 board?

Thanks,

Geoffroy

*From:*acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
*Sent:* Thursday, September 12, 2019 7:15 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue

BR.

Fuzhong

*From:*acrn-users@... <mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Dubravko
Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 8:01 PM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Yin Fenghwei,

Thank you for replying.

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17

kB zip file, containing 4 text files). The zip archive contains:

   * dmesg output from SOS
   * /proc/interrupts from SOS
   * loader.conf from EFI partition
   * acrn.conf (here I've verified that root UUID is correct, and I've
     had to delete settings related to hypervisor memory log because
     otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@... <mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
Lists.Projectacrn.Org <fengwei.yin=intel.com@...
<mailto:fengwei.yin=intel.com@...%20%0b>> <mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,

I work for Exor Embedded and we are developing a new board with an Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed, following the instructions on https://projectacrn.github.io/latest/index.htmlI've managed to get it
running, dmesg among other things says:

Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.

However, it's not really usable. We have following issues:

   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual

When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same kernel with and without hypervisor, but so far we were unable to figure out if any of these differences is significant.

We would appreciate if you have any hints for us, either what is exactly our problem, or at least how we should continue to debug the system.
Could you share:
     1. ACRN last git commit hash
     2. dmesg output in your SOS
     3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei


Best regards,
Dubravko Moravski



*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659
<tel:+38%20512455659%20%3ctel:+38%20512455659>>>  m:+38 5915402413 <tel:+38
<tel:+38%20%0b>> 5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>



   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:


+38 512455659 <tel:+38%20512455659>  m: +38 5915402413
<tel:+38%205915402413>

a:


Slavonska avenija, 50, Zagreb, Croatia, 10000

w:


exorint.com <https://exorint.com/>


imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg

imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>



   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>



ACRN Project Technical Community Meeting (2019/7~2019/12): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

Notice: Join ACRN’s TCM meeting with WebEx till Nov. 2019
Due to Zoom’s connection issue in PRC mainland recently, we have to use WebEx to replace Zoom for ACRN TCM meeting temporarily. We’ll monitor Zoom’s availability closely and switch back to Zoom as it comes back to normal. Sorry for the inconvenience!
Cisco WebEx: https://intel.webex.com/intel/j.php?MTID=m682b81bf58c7ec875c32c469ff22116f  (More details is at the bottom of the meeting request)
 
 
Date: 9/18
Topic:  ACRN Fuzz Testing
Description: This talk will introduce Fuzzing tool and its using in ACRN hypervisor.
 
WW Topic Presenter Status
WW02 TPM2.0 virtualization in ACRN DENG, Wei 1/9
WW03 Polling mode Virtio and its advantage for RT VM DENG, Jie 1/16
WW04 Buffer sharing from UOS to SOS, HyperDMA usage LIU, Xinyun 1/23
WW05 USB HUB Virtualization WU, Xiaoguang 1/30
WW07 ACRN Device Model QoS Design LIU, Long 2/13
WW08 ACRN Debug Tips CHEN, Jason 2/20
WW09 GVT-g debug trace tool GONG, Zhipeng 2/27
WW10 Kata Container Architecture: First Steps with ACRN Dhanraj, Vijay  3/6
WW11 One ACRN hypervisor to support multi-platform WU, Xiangyang 3/13
WW12 Power button key mediator design in ACRN LIU, Yuan 3/20
WW13 Local APIC Virtualization Enhancement for Intel KBL platform LI, Fei 3/27
WW14 Safety VM Support YIN, FengWei 4/3
WW15 How to customize GPIO in ACRN LIU, Yuan 4/10
WW16 ACRN Cache QoS support based on CAT TAO, Yuhong 4/17
WW17 ACRN Real-Time measurement Methodology LI, Wilson 4/24
WW19 I2C Virtualization CHEN, Conghui 5/8
WW20 SGX Virtualization in ACRN WU, Binbin 5/15
WW21 Logger Improvement on acrn-dm CAO, Minggui 5/22
WW22 AcrnGT Virtual Display Deep Dive HE, Min 5/29
WW23 Local APIC Emulation and Pass-through Grandhi, Sainath 6/5
WW25
 
ACRN Functional Safety: Understanding and Mitigating Inter-VM Interference MAO, Junjie 6/19
WW26 Enable VwWorks as RTVM on ACRN FU, Kaige 6/26
WW27 ACRN 2.0 New Architecture Sharing - 1/2 REN, Jack 7/3
WW28 ACRN 2.0 New Architecture Sharing- 2/2 REN, Jack 7/10
WW29 How to enable open vSwitch on ACRN LIU, Yuan 7/17
WW30 ACRN Configuration Design Tool SUN, Victor 7/24
WW31 Design of GOP Driver for GVT-g HE, Min 7/31
WW32 On Dynamic Resource Allocation Mao, Junjie 8/7
WW33 ACRN virtual secure boot and key enrollment Wang, Kai 8/14
WW34 Split ACRN’s Device Model CHEN, Jason CJ 8/21
WW35 ACRN Continuous Integration System Introduction ZHANG, Wenling 8/28
WW36 BKC for ACRN RT and vmexit analysis YAN, Like 9/4
WW37 ACRN Windows As A Guest (WaaG) Overview CHEN, Jianjun 9/11
WW38 ACRN Fuzzing Test HUANG, Yonghua 9/18
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
-- Do not delete or change any of the following text. --  
When it's time, join your Webex meeting here.
 
 
Meeting number (access code): 590 576 334
 
 
 
Join
 
 
 
 
Join by phone 
Tap to call in from a mobile device (attendees only) 
+1-210-795-1110 US Toll 
+1-866-662-9987 US Toll Free 
Global call-in numbers  |  Toll-free calling restrictions  
 
Join from a video system or application
Dial 590576334@... 
You can also dial 173.243.2.68 and enter your meeting number.  
 
 
Need help? Go to http://help.webex.com
 
 
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 


Re: Setting up ACRN on a new board with Apollo Lake

Yin, Fengwei <fengwei.yin@...>
 

Hi Dubravko,

On 2019/9/13 上午1:01, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Geoffroy,
I have already tried with uart=disabled, and repeated the test now; everything remains the same, all the issues are present.
SSH works when not running in the hypervisor.
I'm using the default "clear" user from projectacrn.github.io tutorial.
When network doesn't work, I'm not even getting the IP address (we have a DHCP server in our LAN, it's otherwise working fine).
I only see following message from your sos kernel log:
[ 1.200862] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[ 1.201695] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.

And there is no interrupt for network chip in interrupt dump.

On my MRB (I don't have UP2 env setup. Will check UP2 info later):
I could see following interrupt info:
126: 40 0 0 0 PCI-MSI 245760-edge mei_me

127: 0 0 0 0 PCI-MSI 294912-edge ahci[0000:00:12.0]

128: 0 1 0 0 PCI-MSI 1048576-edge ens4

129: 0 0 38006 0 PCI-MSI 1048577-edge ens4-TxRx-0

130: 0 0 0 17499 PCI-MSI 1048578-edge ens4-TxRx-1

131: 15357 0 0 0 PCI-MSI 1048579-edge ens4-TxRx-2

132: 0 32620 0 0 PCI-MSI 1048580-edge ens4-TxRx-3

133: 0 0 0 0 PCI-MSI 344064-edge xhci_hcd
135: 0 0 0 4 PCI-MSI 49152-edge intel-ipu4

And following info in kernel log:
[ 0.789202] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
[ 0.789203] e100: Copyright(c) 1999-2006 Intel Corporation
[ 0.789237] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[ 0.789238] igb: Copyright (c) 2007-2014 Intel Corporation.
[ 0.818600] pps pps0: new PPS source ptp0
[ 0.818607] igb 0000:02:00.0: added PHC on eth0
[ 0.818609] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
[ 0.818612] igb 0000:02:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 98:4f:ee:13:ce:43
[ 0.818653] igb 0000:02:00.0: eth0: PBA No: 000300-000
[ 0.818655] igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[ 0.818711] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 4.1.0-k
[ 0.818713] ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.

Can you also share the kernel/interrupt dump when there is no HV?
Thanks.

Regards
Yin, Fengwei

I have uploaded new dmesg and ifconfig logs to https://www.amazon.com/clouddrive/share/3wZ8oRCroPr9wpfcl60XAJov69Do8Hg5RiX1GRRpYRI, 76 kB RAR archive:
* log_hyper.log files are with hypervisor running (resulting in all
the problems, including no network)
* 'hyper kernel no hyper' files are with regular non-hypervisor boot,
plus the same kernel that hypervisor is using
(lts2018-sos-4.19.64-76) - less problems e.g. terminal works, but
still no network
* 'native kernel' files are with regular non-hypervisor boot, and
native-something kernel - no problems and network works; this is the
combination I'm using when SSH works
The date is always off by three days, it doesn't seem to affect anything: with hypervisor there are problems, without hypervisor there aren't. I'll fix it, just in case.
It looks like maybe the hypervisor is actually working fine all the time, but the kernel it is loading is somehow misconfigured so it doesn't initialize network interface(s) and user account(s)?
Best regards,
Dubravko
------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Geoffroy Van Cutsem via Lists.Projectacrn.Org <geoffroy.vancutsem=intel.com@...>
*Sent:* Thursday, September 12, 2019 5:46 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Dubravko,
*From:*acrn-users@... <acrn-users@...> *On Behalf Of *Dubravko Moravski | Exor Embedded S.r.l.
*Sent:* Thursday, September 12, 2019 5:15 PM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi everyone,
SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.
[Geoffroy] A few questions and one suggestion:
* Have you tried to start the ACRN hypervisor with “uart=disabled”? (see https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition). A quick test if you have an EFI shell on your platform is to run “acrn.efi uart=disabled” from it.
* Does SSH work when you are not running in the hypervisor?
* Which user are you using when trying to connect over SSH? (note that root is blocked by default)
* When you say “networking is not working in general”, does that mean you are not even getting an IP address?
200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).
[Geoffroy] Is the date always off by three days (regardless of whether you’re running in the hypervisor or not)? I don’t know if that is the cause but dates in the past can cause problems with certificates (and hence something to look at as a potential issue related to SSH).
I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.
These all work when not running with the hypervisor.
Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.
Best regards,
Dubravko
------------------------------------------------------------------------
*From:*acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@... <mailto:fengwei.yin=intel.com@...>>
*Sent:* Thursday, September 12, 2019 3:21 PM
*To:* acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Dubravko,
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:

+38 512455659 <tel:+38%20512455659>  m: +38 5915402413 <tel:+38%205915402413>
a:

Slavonska avenija, 50, Zagreb, Croatia, 10000
w:

exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,
We haven't observed any issues on the UP2 either, except one, screen updates been erratic and mouse pointer jumping around.
We've just followed instructions from https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
automated setup with some manual fine tuning.
I guess since we are using the same software and UP2 works (mostly) fine, the causes of our problems must be either different hardware or BIOS. But the trouble is figuring out what exactly.
All mentioned issues are for the Service OS. We can't even install/launch Guest OS because the Terminal no longer works after we restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.
When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?
Regards
Yin, Fengwei

Now that the memory log issue is fixed, could we use that to get additional debug information? If yes, how can I save that information for later? (We managed to create some scripts so we can save data like dmesg even when terminal can't be invoked.)
Best regards,
Dubravko
------------------------------------------------------------------------
*From:* acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...>> on behalf of Liu, Fuzhong via
Lists.Projectacrn.Org <fuzhong.liu=intel.com@...
<mailto:fuzhong.liu=intel.com@...>>
*Sent:* Thursday, September 12, 2019 11:47 AM
*To:* acrn-users@...
<mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
No observed the issue on the UP2 board in our lab.
Need detail steps to reproduce.
BR.
Fuzhong
*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
*Sent:* Thursday, September 12, 2019 4:42 PM
*To:* acrn-users@... <mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Have you observed the same problems on the UP2 board?
Thanks,
Geoffroy
*From:*acrn-users@... <mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
*Sent:* Thursday, September 12, 2019 7:15 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Dubravko
All issues you mentioned  are for Service OS, right?
Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
BR.
Fuzhong
*From:*acrn-users@... <mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Dubravko
Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 8:01 PM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Yin Fenghwei,
Thank you for replying.
ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).
I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
kB zip file, containing 4 text files). The zip archive contains:
   * dmesg output from SOS
   * /proc/interrupts from SOS
   * loader.conf from EFI partition
   * acrn.conf (here I've verified that root UUID is correct, and I've
     had to delete settings related to hypervisor memory log because
     otherwise it crashed)
Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.
Best regards,
Dubravko
------------------------------------------------------------------------
*From:*acrn-users@... <mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
Lists.Projectacrn.Org <fengwei.yin=intel.com@...
<mailto:fengwei.yin=intel.com@...%20%0b>> <mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@... <mailto:acrn-users@...>
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...%20%0b>> <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi,
On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,
I work for Exor Embedded and we are developing a new board with an Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed, following the instructions on https://projectacrn.github.io/latest/index.htmlI've managed to get it
running, dmesg among other things says:
Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.
However, it's not really usable. We have following issues:
   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual
When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same kernel with and without hypervisor, but so far we were unable to figure out if any of these differences is significant.
We would appreciate if you have any hints for us, either what is exactly our problem, or at least how we should continue to debug the system.
Could you share:
     1. ACRN last git commit hash
     2. dmesg output in your SOS
     3. the output of cat /proc/interrupt in your SOS
Regards
Yin, Fengwei

Best regards,
Dubravko Moravski
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659
<tel:+38%20512455659%20%3ctel:+38%20512455659>>>  m:+38 5915402413 <tel:+38
<tel:+38%20%0b>> 5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>
   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:
+38 512455659 <tel:+38%20512455659>  m: +38 5915402413
<tel:+38%205915402413>
a:
Slavonska avenija, 50, Zagreb, Croatia, 10000
w:
exorint.com <https://exorint.com/>
imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>
   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>


Re: Setting up ACRN on a new board with Apollo Lake

Yin, Fengwei <fengwei.yin@...>
 

Hi Dubravko,

On 9/14/2019 12:49 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,
I have tried:
* compiling ACRN release version and setting console=ttyS2 (an earlier
suggestion) = no improvement
Which ttyS is used for console depends on your hardware design. You need
to check with your hardware engineer to figure out which ttyS is used
for console.

If you are using the UP2 configuration, you console should connect to
ttyS0 (which is defined by default up2 config file:
CONFIG_SERIAL_PCI_BDF="0:18.0")

* using ACRN release version and removing console=ttySx setting = the
system is noticeably faster overall, unfortunately all other
I believe using release version WITHOUT removing console=ttyS0 in SOS
kernel cmdline could give us terminal on serial port back.

symptoms are still present (no terminal, no network, bad screen refresh)
So this is certainly progress, but we would like to solve remaining issues. I have uploaded newest logs to https://www.amazon.com/clouddrive/share/PRkJdMWE8Yu6O7uK0aN0cNUtVWDazQvHeUdHbvFT1av
This is the log with ACRN release version. Right?

We've noticed some differences in configuration files for various boot options. Our acrn.conf was (mostly) generated by the script used for automated setup, but it seems quite different from other configuration files: other conf files have initrd entries, init file for kernel to launch, noreplace-smp and other additional settings... Could any of these be unintentionally missing from our acrn.conf and causing our problems? I've tried several combinations of blindly adding some of these settings into our acrn.conf, but nothing helped.
which "other conf file" are you referring here?

Regards
Yin, Fengwei

Another thing we've tried was setting up our Up2 device with identical kernel and acrn hypervisor as on our test board. Hypervisor on Up2 seems to always run fine, whatever particular versions of software we use; and it never runs fine on our board... since the hardware is reasonably similar, and otherwise working well, could there be some issue with our BIOS? We've got it from a specialized vendor and they are willing to fix it, if we could tell them what exactly they need to change. >
Best regards,
Dubravko
------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Xu, Anthony via Lists.Projectacrn.Org <anthony.xu=intel.com@...>
*Sent:* Friday, September 13, 2019 1:53 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Dubravko,
In SOS, vuart is sharing irq 4 with idma, that may cause issue. I think you are using debug version, can you try release version? There is no ttyS0 in release version.
And please remove “console=ttyS0” from SOS kernel parameter.
   4:         33   IO-APIC    4-fasteoi   idma64.5, ttyS0
**
**
*Anthony*
**
*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Dubravko Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 5:01 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi Yin Fenghwei,
Thank you for replying.
ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).
I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:
* dmesg output from SOS
* /proc/interrupts from SOS
* loader.conf from EFI partition
* acrn.conf (here I've verified that root UUID is correct, and I've
had to delete settings related to hypervisor memory log because
otherwise it crashed)
Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.
Best regards,
Dubravko
------------------------------------------------------------------------
*From:*acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@... <mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@... <mailto:acrn-users@...> <acrn-users@... <mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
Hi,
On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,
I work for Exor Embedded and we are developing a new board with an Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed, following the instructions on https://projectacrn.github.io/latest/index.html I've managed to get it
running, dmesg among other things says:
Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.
However, it's not really usable. We have following issues:
   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual
When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same kernel with and without hypervisor, but so far we were unable to figure out if any of these differences is significant.
We would appreciate if you have any hints for us, either what is exactly our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS
Regards
Yin, Fengwei

Best regards,
Dubravko Moravski
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>
   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:

+38 512455659 <tel:+38%20512455659>  m: +38 5915402413 <tel:+38%205915402413>
a:

Slavonska avenija, 50, Zagreb, Croatia, 10000
w:

exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>


Re: Setting up ACRN on a new board with Apollo Lake

Yin, Fengwei <fengwei.yin@...>
 

Hi Conghui,

On 9/12/2019 10:41 PM, Chen, Conghui wrote:
Hi Fengwei,

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,

We haven't observed any issues on the UP2 either, except one, screen
updates been erratic and mouse pointer jumping around.

We've just followed instructions from
https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
automated setup with some manual fine tuning.
I guess since we are using the same software and UP2 works (mostly)
fine, the causes of our problems must be either different hardware or
BIOS. But the trouble is figuring out what exactly.

All mentioned issues are for the Service OS. We can't even
install/launch Guest OS because the Terminal no longer works after we
restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.
It seems irq 4 is shared between two uarts.
For ttyS0, it's interrupt is default as 4, and it is correct in the log.
[ 1.084989] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
But the other pci uart ttyS1 also use irq 4.
[ 1.173842] dw-apb-uart.5: ttyS1 at MMIO 0x91228000 (irq = 4, base_baud = 115200) is a 16550A
ttyS0 is a vuart simulated in hypervisor, and ttyS1 is a physical uart, should not share the same irq.
Currently, I'm not sure, why the lpc uart will choose to use irq 4. Is it possible that there is some lpc definition
in acpi table?
Yes. Emulated ttyS0 and physical ttyS1 share same irq. But there is no
user of ttyS1. we can't see ttyS0/ttyS1 share irq4 in interrupt dump.

But as Anthony said, the emulated ttyS0 share irq4 with uart dma. Which
is risky. The irq of emulated SOS serial port is hardcoded in
configuration. For UP2, it's:
define SOS_COM1_IRQ 4U

We should change it to 6U here. But there is no irq flooding from
interrupt dump.... But let's try it.

Hi Dubravko,
I suppose you are define your board based on UP2 configuration. Could
you please change the line:
#define SOS_COM1_IRQ 4U
in hypervisor/arch/x86/configs/apl-up2/misc_cfg.h to
#define SOS_COM1_IRQ 6U

To see whether there are still slow issue?

Regards
Yin, Fengwei

Regards,
Conghui.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei


Now that the memory log issue is fixed, could we use that to get
additional debug information? If yes, how can I save that information
for later? (We managed to create some scripts so we can save data like
dmesg even when terminal can't be invoked.)

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@...
<acrn-users@...> on behalf of Liu, Fuzhong via
Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
*Sent:* Thursday, September 12, 2019 11:47 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

No observed the issue on the UP2 board in our lab.

Need detail steps to reproduce.

BR.

Fuzhong

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
*Sent:* Thursday, September 12, 2019 4:42 PM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Have you observed the same problems on the UP2 board?

Thanks,

Geoffroy

*From:*acrn-users@...
<mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
*Sent:* Thursday, September 12, 2019 7:15 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000”
issue

BR.

Fuzhong

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Dubravko
Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 8:01 PM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Yin Fenghwei,

Thank you for replying.

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I
am
getting identical behavior with ACRN that I've compiled, as well as with
ACRN installed by Clear Linux (version id 30770).

I'm not sure if this mailing lists allows attachments, so I've uploaded
the requested files to
https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKg
NisSUkQdR03mr4sY(17
kB zip file, containing 4 text files). The zip archive contains:

* dmesg output from SOS
* /proc/interrupts from SOS
* loader.conf from EFI partition
* acrn.conf (here I've verified that root UUID is correct, and I've
had to delete settings related to hypervisor memory log because
otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
and Leaf Hill CRB.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@...
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...>> on behalf of Yin, Fengwei via
Lists.Projectacrn.Org <fengwei.yin=intel.com@...
<mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@...
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,

I work for Exor Embedded and we are developing a new board with an
Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
following the instructions on
https://projectacrn.github.io/latest/index.htmlI've managed to get it
running, dmesg among other things says:

Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.

However, it's not really usable. We have following issues:

   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual

When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same
kernel with and without hypervisor, but so far we were unable to figure
out if any of these differences is significant.

We would appreciate if you have any hints for us, either what is exactly
our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei


Best regards,
Dubravko Moravski



*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659
<tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>




   Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:



+38 512455659 <tel:+38%20512455659>  m: +38 5915402413
<tel:+38%205915402413>

a:



Slavonska avenija, 50, Zagreb, Croatia, 10000

w:



exorint.com <https://exorint.com/>



imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?he
ader=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8
e9a8c2b3e4c.jpg

imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?he
ader=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-
8a58-167a2248b283.jpg Prima
di stampare pensa ai costi ambientali. Please consider the environment
before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38
5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>




 Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>


Re: Setting up ACRN on a new board with Apollo Lake

Liu, Fuzhong
 

Hi Dubravko

Checked ifconfig_log_20190913.log you uploaded:

 

acrn-br0  Link encap:Ethernet  HWaddr F2:14:63:23:6B:FD 

          inet6 addr: fe80::f014:63ff:fe23:6bfd/64 Scope:Link  fe80::f014:63ff:fe23:6bfd

          UP BROADCAST MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:0 (0.0 b)  TX bytes:1524 (1.4 Kb)

 

It is link-local address for your IPv6; Could you please share ifconfig info on UP2? Is it the same link-local address for IPv6?

Following is the procedures from MAC to IPv6:

1.     1) take the mac address: f2:14:63:23:6b:fd 

2.     2) throw ff:fe in the middle: f2:14:63:ff:fe:23:6bfd 

3.     3) reformat to IPv6 notation f214:63ff:fe23:6bfd

4.     4) convert the first octet from hexadecimal to binary: f2 -> 11110010

5.     5) invert the bit at index 6 (counting from 0): 11110010 -> 11110000

6.     6) convert octet back to hexadecimal: 01010000 -> f0

7.     7) replace first octet with newly calculated one: f014:63ff:fe23:6bfd

8.     8) prepend the link-local prefix: fe80:: f014:63ff:fe23:6bfd

You can double check it with online tool:

https://ben.akrin.com/?p=1347

9.     BR.

10. Fuzhong

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Saturday, September 14, 2019 12:50 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

 

I have tried:

  • compiling ACRN release version and setting console=ttyS2 (an earlier suggestion) = no improvement
  • using ACRN release version and removing console=ttySx setting = the system is noticeably faster overall, unfortunately all other symptoms are still present (no terminal, no network, bad screen refresh)

So this is certainly progress, but we would like to solve remaining issues. I have uploaded newest logs to https://www.amazon.com/clouddrive/share/PRkJdMWE8Yu6O7uK0aN0cNUtVWDazQvHeUdHbvFT1av

 

We've noticed some differences in configuration files for various boot options. Our acrn.conf was (mostly) generated by the script used for automated setup, but it seems quite different from other configuration files: other conf files have initrd entries, init file for kernel to launch, noreplace-smp and other additional settings... Could any of these be unintentionally missing from our acrn.conf and causing our problems? I've tried several combinations of blindly adding some of these settings into our acrn.conf, but nothing helped.

 

Another thing we've tried was setting up our Up2 device with identical kernel and acrn hypervisor as on our test board. Hypervisor on Up2 seems to always run fine, whatever particular versions of software we use; and it never runs fine on our board... since the hardware is reasonably similar, and otherwise working well, could there be some issue with our BIOS? We've got it from a specialized vendor and they are willing to fix it, if we could tell them what exactly they need to change.

 

Best regards,

Dubravko

 


From: acrn-users@... <acrn-users@...> on behalf of Xu, Anthony via Lists.Projectacrn.Org <anthony.xu=intel.com@...>
Sent: Friday, September 13, 2019 1:53 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko,

 

In SOS, vuart is sharing irq 4 with idma, that may cause issue. I think you are using debug version, can you try release version? There is no ttyS0 in release version.

And please remove “console=ttyS0” from SOS kernel parameter.

 

   4:         33   IO-APIC    4-fasteoi   idma64.5, ttyS0

 

 

Anthony

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 5:01 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
> https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>

 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi,

I have tried:
  • compiling ACRN release version and setting console=ttyS2 (an earlier suggestion) = no improvement
  • using ACRN release version and removing console=ttySx setting = the system is noticeably faster overall, unfortunately all other symptoms are still present (no terminal, no network, bad screen refresh)
So this is certainly progress, but we would like to solve remaining issues. I have uploaded newest logs to https://www.amazon.com/clouddrive/share/PRkJdMWE8Yu6O7uK0aN0cNUtVWDazQvHeUdHbvFT1av

We've noticed some differences in configuration files for various boot options. Our acrn.conf was (mostly) generated by the script used for automated setup, but it seems quite different from other configuration files: other conf files have initrd entries, init file for kernel to launch, noreplace-smp and other additional settings... Could any of these be unintentionally missing from our acrn.conf and causing our problems? I've tried several combinations of blindly adding some of these settings into our acrn.conf, but nothing helped.

Another thing we've tried was setting up our Up2 device with identical kernel and acrn hypervisor as on our test board. Hypervisor on Up2 seems to always run fine, whatever particular versions of software we use; and it never runs fine on our board... since the hardware is reasonably similar, and otherwise working well, could there be some issue with our BIOS? We've got it from a specialized vendor and they are willing to fix it, if we could tell them what exactly they need to change.

Best regards,
Dubravko


From: acrn-users@... <acrn-users@...> on behalf of Xu, Anthony via Lists.Projectacrn.Org <anthony.xu=intel.com@...>
Sent: Friday, September 13, 2019 1:53 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
 

Hi Dubravko,

 

In SOS, vuart is sharing irq 4 with idma, that may cause issue. I think you are using debug version, can you try release version? There is no ttyS0 in release version.

And please remove “console=ttyS0” from SOS kernel parameter.

 

   4:         33   IO-APIC    4-fasteoi   idma64.5, ttyS0

 

 

Anthony

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 5:01 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
> https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>


 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Xu, Anthony
 

Hi Dubravko,

 

In SOS, vuart is sharing irq 4 with idma, that may cause issue. I think you are using debug version, can you try release version? There is no ttyS0 in release version.

And please remove “console=ttyS0” from SOS kernel parameter.

 

   4:         33   IO-APIC    4-fasteoi   idma64.5, ttyS0

 

 

Anthony

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 5:01 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
> https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>


 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Xu, Anthony
 

I mean removing both hvlog=2M@0x1FE00000 and memmap=2M$0x1FE00000, that disables HV log.

We can make hvlog optional, if hvlog is not specified, acrn log driver should allocate sbuf just like acrn trace does.

 

 

 

Anthony

 

 

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Liu, Fuzhong
Sent: Thursday, September 12, 2019 3:32 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

“memmap=2M$0x1FE00000” will  reserve the memory for HV log;

There is stability issue if no this parameter in cmdline(memory used by kernel was overwrite by HV log); Fengwei can give more detail info.

 

BR.

Fuzhong

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Friday, September 13, 2019 2:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

“memmap=2M$0x1FE00000” SOS kernel parameter is platform specific, can we remove “memmap=2M$0x1FE00000” by default?

 

The only usage of “memmap=2M$0x1FE00000” is to check crack log, right? That’s not the major case. People can add the parameter back if needed.

 

 

Anthony

 

 

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Liu, Fuzhong
Sent: Wednesday, September 11, 2019 10:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue 

 

BR.

Fuzhong

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 8:01 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To:
acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
>
https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <
tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <
https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <
https://www.exorint.com/it/privacy>
>

 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Liu, Fuzhong
 

“memmap=2M$0x1FE00000” will  reserve the memory for HV log;

There is stability issue if no this parameter in cmdline(memory used by kernel was overwrite by HV log); Fengwei can give more detail info.

 

BR.

Fuzhong

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Friday, September 13, 2019 2:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

“memmap=2M$0x1FE00000” SOS kernel parameter is platform specific, can we remove “memmap=2M$0x1FE00000” by default?

 

The only usage of “memmap=2M$0x1FE00000” is to check crack log, right? That’s not the major case. People can add the parameter back if needed.

 

 

Anthony

 

 

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Liu, Fuzhong
Sent: Wednesday, September 11, 2019 10:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue 

 

BR.

Fuzhong

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 8:01 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To:
acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
>
https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <
tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <
https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <
https://www.exorint.com/it/privacy>
>

 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Xu, Anthony
 

“memmap=2M$0x1FE00000” SOS kernel parameter is platform specific, can we remove “memmap=2M$0x1FE00000” by default?

 

The only usage of “memmap=2M$0x1FE00000” is to check crack log, right? That’s not the major case. People can add the parameter back if needed.

 

 

Anthony

 

 

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Liu, Fuzhong
Sent: Wednesday, September 11, 2019 10:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue 

 

BR.

Fuzhong

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Wednesday, September 11, 2019 8:01 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Yin Fenghwei,

 

Thank you for replying.

 

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am getting identical behavior with ACRN that I've compiled, as well as with ACRN installed by Clear Linux (version id 30770).

 

I'm not sure if this mailing lists allows attachments, so I've uploaded the requested files to https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY (17 kB zip file, containing 4 text files). The zip archive contains:

  • dmesg output from SOS
  • /proc/interrupts from SOS
  • loader.conf from EFI partition
  • acrn.conf (here I've verified that root UUID is correct, and I've had to delete settings related to hypervisor memory log because otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared and Leaf Hill CRB.

 

Best regards,

Dubravko

 

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Wednesday, September 11, 2019 4:41 AM
To:
acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello everyone,
>
> I work for Exor Embedded and we are developing a new board with an
> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
> following the instructions on
>
https://projectacrn.github.io/latest/index.html I've managed to get it
> running, dmesg among other things says:
>
> Hypervisor detected: ACRN
> ...
> ACRNTrace: Initialized acrn trace module with 4 cpu
> ...
> systemd[1]: Detected virtualization acrn.
>
> However, it's not really usable. We have following issues:
>
>   * mouse cursor jumps around in a weird way, like there is some problem
>     with switching video buffers
>   * we can't open the Terminal in any way, there's always a message:
>     "There was an error creating a child process for this terminal.
>     Failed to open PTY: Permission denied"
>   * network doesn't work
>   * everything works much slower than usual
>
> When not running with hypervisor, everything works.
> We've noticed some differences between dmesgs when running the same
> kernel with and without hypervisor, but so far we were unable to figure
> out if any of these differences is significant.
>
> We would appreciate if you have any hints for us, either what is exactly
> our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei

>
> Best regards,
> Dubravko Moravski
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <
tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <
https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <
https://www.exorint.com/it/privacy>
>

 

Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Re: Setting up ACRN on a new board with Apollo Lake

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi Geoffroy,

I have already tried with uart=disabled, and repeated the test now; everything remains the same, all the issues are present.

SSH works when not running in the hypervisor.

I'm using the default "clear" user from projectacrn.github.io tutorial.

When network doesn't work, I'm not even getting the IP address (we have a DHCP server in our LAN, it's otherwise working fine).

I have uploaded new dmesg and ifconfig logs to https://www.amazon.com/clouddrive/share/3wZ8oRCroPr9wpfcl60XAJov69Do8Hg5RiX1GRRpYRI, 76 kB RAR archive:
  • log_hyper.log files are with hypervisor running (resulting in all the problems, including no network)
  • 'hyper kernel no hyper' files are with regular non-hypervisor boot, plus the same kernel that hypervisor is using (lts2018-sos-4.19.64-76) - less problems e.g. terminal works, but still no network
  • 'native kernel' files are with regular non-hypervisor boot, and native-something kernel - no problems and network works; this is the combination I'm using when SSH works
The date is always off by three days, it doesn't seem to affect anything: with hypervisor there are problems, without hypervisor there aren't. I'll fix it, just in case.

It looks like maybe the hypervisor is actually working fine all the time, but the kernel it is loading is somehow misconfigured so it doesn't initialize network interface(s) and user account(s)?

Best regards,
Dubravko



From: acrn-users@... <acrn-users@...> on behalf of Geoffroy Van Cutsem via Lists.Projectacrn.Org <geoffroy.vancutsem=intel.com@...>
Sent: Thursday, September 12, 2019 5:46 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
 

Hi Dubravko,

 

From: acrn-users@... <acrn-users@...> On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Thursday, September 12, 2019 5:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi everyone,

 

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

[Geoffroy] A few questions and one suggestion:

* Have you tried to start the ACRN hypervisor with “uart=disabled”? (see https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition). A quick test if you have an EFI shell on your platform is to run “acrn.efi uart=disabled” from it.

* Does SSH work when you are not running in the hypervisor?

* Which user are you using when trying to connect over SSH? (note that root is blocked by default)

* When you say “networking is not working in general”, does that mean you are not even getting an IP address?

 

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).

[Geoffroy] Is the date always off by three days (regardless of whether you’re running in the hypervisor or not)? I don’t know if that is the cause but dates in the past can cause problems with certificates (and hence something to look at as a potential issue related to SSH).

 

I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

 

These all work when not running with the hypervisor.

 

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

 

Best regards,

Dubravko

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Thursday, September 12, 2019 3:21 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko,


Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi,
>
> We haven't observed any issues on the UP2 either, except one, screen
> updates been erratic and mouse pointer jumping around.
>
> We've just followed instructions from
> https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
> automated setup with some manual fine tuning.
> I guess since we are using the same software and UP2 works (mostly)
> fine, the causes of our problems must be either different hardware or
> BIOS. But the trouble is figuring out what exactly.
>
> All mentioned issues are for the Service OS. We can't even
> install/launch Guest OS because the Terminal no longer works after we
> restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei

>
> Now that the memory log issue is fixed, could we use that to get
> additional debug information? If yes, how can I save that information
> for later? (We managed to create some scripts so we can save data like
> dmesg even when terminal can't be invoked.)
>
> Best regards,
> Dubravko
>
>
> ------------------------------------------------------------------------
> *From:* acrn-users@...
> <acrn-users@...> on behalf of Liu, Fuzhong via
> Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
> *Sent:* Thursday, September 12, 2019 11:47 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> No observed the issue on the UP2 board in our lab.
>
> Need detail steps to reproduce.
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
> *Sent:* Thursday, September 12, 2019 4:42 PM
> *To:* acrn-users@...
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Have you observed the same problems on the UP2 board?
>
> Thanks,
>
> Geoffroy
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> <acrn-users@...
> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
> *Sent:* Thursday, September 12, 2019 7:15 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Dubravko
>
> All issues you mentioned  are for Service OS, right?
>
> Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> [mailto:acrn-users@...] *On Behalf Of *Dubravko
> Moravski | Exor Embedded S.r.l.
> *Sent:* Wednesday, September 11, 2019 8:01 PM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Yin Fenghwei,
>
> Thank you for replying.
>
> ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am
> getting identical behavior with ACRN that I've compiled, as well as with
> ACRN installed by Clear Linux (version id 30770).
>
> I'm not sure if this mailing lists allows attachments, so I've uploaded
> the requested files to
> https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
> kB zip file, containing 4 text files). The zip archive contains:
>
>   * dmesg output from SOS
>   * /proc/interrupts from SOS
>   * loader.conf from EFI partition
>   * acrn.conf (here I've verified that root UUID is correct, and I've
>     had to delete settings related to hypervisor memory log because
>     otherwise it crashed)
>
> Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
> and Leaf Hill CRB.
>
> Best regards,
>
> Dubravko
>
> ------------------------------------------------------------------------
>
> *From:*acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
> Lists.Projectacrn.Org <fengwei.yin=intel.com@...
> <mailto:fengwei.yin=intel.com@...>>
> *Sent:* Wednesday, September 11, 2019 4:41 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi,
>
> On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hello everyone,
>>
>> I work for Exor Embedded and we are developing a new board with an
>> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
>> following the instructions on
>> https://projectacrn.github.io/latest/index.htmlI've managed to get it
>> running, dmesg among other things says:
>>
>> Hypervisor detected: ACRN
>> ...
>> ACRNTrace: Initialized acrn trace module with 4 cpu
>> ...
>> systemd[1]: Detected virtualization acrn.
>>
>> However, it's not really usable. We have following issues:
>>
>>   * mouse cursor jumps around in a weird way, like there is some problem
>>     with switching video buffers
>>   * we can't open the Terminal in any way, there's always a message:
>>     "There was an error creating a child process for this terminal.
>>     Failed to open PTY: Permission denied"
>>   * network doesn't work
>>   * everything works much slower than usual
>>
>> When not running with hypervisor, everything works.
>> We've noticed some differences between dmesgs when running the same
>> kernel with and without hypervisor, but so far we were unable to figure
>> out if any of these differences is significant.
>>
>> We would appreciate if you have any hints for us, either what is exactly
>> our problem, or at least how we should continue to debug the system.
> Could you share:
>     1. ACRN last git commit hash
>     2. dmesg output in your SOS
>     3. the output of cat /proc/interrupt in your SOS
>
> Regards
> Yin, Fengwei
>
>>
>> Best regards,
>> Dubravko Moravski
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
> 5915402413 <tel:+38%205915402413>>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>       
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>
>
> *Dubravko Moravski*
>
> /SW engineering/
>
> *Exor Embedded S.r.l.*
>
> p:
>
>       
>
> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
> <tel:+38%205915402413>
>
> a:
>
>       
>
> Slavonska avenija, 50, Zagreb, Croatia, 10000
>
> w:
>
>       
>
> exorint.com <https://exorint.com/>
>
>       
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
> di stampare pensa ai costi ambientali. Please consider the environment
> before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>




Re: Setting up ACRN on a new board with Apollo Lake

Geoffroy Van Cutsem
 

Hi Dubravko,

 

From: acrn-users@... <acrn-users@...> On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Thursday, September 12, 2019 5:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi everyone,

 

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

[Geoffroy] A few questions and one suggestion:

* Have you tried to start the ACRN hypervisor with “uart=disabled”? (see https://projectacrn.github.io/latest/getting-started/apl-nuc.html#add-the-acrn-hypervisor-to-the-efi-partition). A quick test if you have an EFI shell on your platform is to run “acrn.efi uart=disabled” from it.

* Does SSH work when you are not running in the hypervisor?

* Which user are you using when trying to connect over SSH? (note that root is blocked by default)

* When you say “networking is not working in general”, does that mean you are not even getting an IP address?

 

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).

[Geoffroy] Is the date always off by three days (regardless of whether you’re running in the hypervisor or not)? I don’t know if that is the cause but dates in the past can cause problems with certificates (and hence something to look at as a potential issue related to SSH).

 

I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

 

These all work when not running with the hypervisor.

 

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

 

Best regards,

Dubravko

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Thursday, September 12, 2019 3:21 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko,


Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi,
>
> We haven't observed any issues on the UP2 either, except one, screen
> updates been erratic and mouse pointer jumping around.
>
> We've just followed instructions from
> https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
> automated setup with some manual fine tuning.
> I guess since we are using the same software and UP2 works (mostly)
> fine, the causes of our problems must be either different hardware or
> BIOS. But the trouble is figuring out what exactly.
>
> All mentioned issues are for the Service OS. We can't even
> install/launch Guest OS because the Terminal no longer works after we
> restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei

>
> Now that the memory log issue is fixed, could we use that to get
> additional debug information? If yes, how can I save that information
> for later? (We managed to create some scripts so we can save data like
> dmesg even when terminal can't be invoked.)
>
> Best regards,
> Dubravko
>
>
> ------------------------------------------------------------------------
> *From:* acrn-users@...
> <acrn-users@...> on behalf of Liu, Fuzhong via
> Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
> *Sent:* Thursday, September 12, 2019 11:47 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> No observed the issue on the UP2 board in our lab.
>
> Need detail steps to reproduce.
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
> *Sent:* Thursday, September 12, 2019 4:42 PM
> *To:* acrn-users@...
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Have you observed the same problems on the UP2 board?
>
> Thanks,
>
> Geoffroy
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> <acrn-users@...
> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
> *Sent:* Thursday, September 12, 2019 7:15 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Dubravko
>
> All issues you mentioned  are for Service OS, right?
>
> Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> [mailto:acrn-users@...] *On Behalf Of *Dubravko
> Moravski | Exor Embedded S.r.l.
> *Sent:* Wednesday, September 11, 2019 8:01 PM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Yin Fenghwei,
>
> Thank you for replying.
>
> ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am
> getting identical behavior with ACRN that I've compiled, as well as with
> ACRN installed by Clear Linux (version id 30770).
>
> I'm not sure if this mailing lists allows attachments, so I've uploaded
> the requested files to
> https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
> kB zip file, containing 4 text files). The zip archive contains:
>
>   * dmesg output from SOS
>   * /proc/interrupts from SOS
>   * loader.conf from EFI partition
>   * acrn.conf (here I've verified that root UUID is correct, and I've
>     had to delete settings related to hypervisor memory log because
>     otherwise it crashed)
>
> Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
> and Leaf Hill CRB.
>
> Best regards,
>
> Dubravko
>
> ------------------------------------------------------------------------
>
> *From:*acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
> Lists.Projectacrn.Org <fengwei.yin=intel.com@...
> <mailto:fengwei.yin=intel.com@...>>
> *Sent:* Wednesday, September 11, 2019 4:41 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi,
>
> On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hello everyone,
>>
>> I work for Exor Embedded and we are developing a new board with an
>> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
>> following the instructions on
>> https://projectacrn.github.io/latest/index.htmlI've managed to get it
>> running, dmesg among other things says:
>>
>> Hypervisor detected: ACRN
>> ...
>> ACRNTrace: Initialized acrn trace module with 4 cpu
>> ...
>> systemd[1]: Detected virtualization acrn.
>>
>> However, it's not really usable. We have following issues:
>>
>>   * mouse cursor jumps around in a weird way, like there is some problem
>>     with switching video buffers
>>   * we can't open the Terminal in any way, there's always a message:
>>     "There was an error creating a child process for this terminal.
>>     Failed to open PTY: Permission denied"
>>   * network doesn't work
>>   * everything works much slower than usual
>>
>> When not running with hypervisor, everything works.
>> We've noticed some differences between dmesgs when running the same
>> kernel with and without hypervisor, but so far we were unable to figure
>> out if any of these differences is significant.
>>
>> We would appreciate if you have any hints for us, either what is exactly
>> our problem, or at least how we should continue to debug the system.
> Could you share:
>     1. ACRN last git commit hash
>     2. dmesg output in your SOS
>     3. the output of cat /proc/interrupt in your SOS
>
> Regards
> Yin, Fengwei
>
>>
>> Best regards,
>> Dubravko Moravski
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
> 5915402413 <tel:+38%205915402413>>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>       
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>
>
> *Dubravko Moravski*
>
> /SW engineering/
>
> *Exor Embedded S.r.l.*
>
> p:
>
>       
>
> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
> <tel:+38%205915402413>
>
> a:
>
>       
>
> Slavonska avenija, 50, Zagreb, Croatia, 10000
>
> w:
>
>       
>
> exorint.com <https://exorint.com/>
>
>       
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
> di stampare pensa ai costi ambientali. Please consider the environment
> before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>




Re: Setting up ACRN on a new board with Apollo Lake

Conghui Chen
 

Hi Dubravko,

 

Can you try to use /dev/ttyS2 as SOS console?

Replace the “console=ttyS0” in acrn.conf with “console=ttyS2”.

 

Regards,

Conghui.

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Thursday, September 12, 2019 11:15 PM
To: acrn-users@...
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi everyone,

 

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

 

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).

I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

 

These all work when not running with the hypervisor.

 

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

 

Best regards,

Dubravko

 


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Thursday, September 12, 2019 3:21 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

 

Hi Dubravko,


Dubravko Moravski

SW engineering

Exor Embedded S.r.l.

p:

+38 512455659  m: +38 5915402413

a:

Slavonska avenija, 50, Zagreb, Croatia, 10000

w:

exorint.com 

 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi,
>
> We haven't observed any issues on the UP2 either, except one, screen
> updates been erratic and mouse pointer jumping around.
>
> We've just followed instructions from
> https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
> automated setup with some manual fine tuning.
> I guess since we are using the same software and UP2 works (mostly)
> fine, the causes of our problems must be either different hardware or
> BIOS. But the trouble is figuring out what exactly.
>
> All mentioned issues are for the Service OS. We can't even
> install/launch Guest OS because the Terminal no longer works after we
> restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei

>
> Now that the memory log issue is fixed, could we use that to get
> additional debug information? If yes, how can I save that information
> for later? (We managed to create some scripts so we can save data like
> dmesg even when terminal can't be invoked.)
>
> Best regards,
> Dubravko
>
>
> ------------------------------------------------------------------------
> *From:* acrn-users@...
> <acrn-users@...> on behalf of Liu, Fuzhong via
> Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
> *Sent:* Thursday, September 12, 2019 11:47 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> No observed the issue on the UP2 board in our lab.
>
> Need detail steps to reproduce.
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
> *Sent:* Thursday, September 12, 2019 4:42 PM
> *To:* acrn-users@...
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Have you observed the same problems on the UP2 board?
>
> Thanks,
>
> Geoffroy
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> <acrn-users@...
> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
> *Sent:* Thursday, September 12, 2019 7:15 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Dubravko
>
> All issues you mentioned  are for Service OS, right?
>
> Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> [mailto:acrn-users@...] *On Behalf Of *Dubravko
> Moravski | Exor Embedded S.r.l.
> *Sent:* Wednesday, September 11, 2019 8:01 PM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Yin Fenghwei,
>
> Thank you for replying.
>
> ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am
> getting identical behavior with ACRN that I've compiled, as well as with
> ACRN installed by Clear Linux (version id 30770).
>
> I'm not sure if this mailing lists allows attachments, so I've uploaded
> the requested files to
> https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
> kB zip file, containing 4 text files). The zip archive contains:
>
>   * dmesg output from SOS
>   * /proc/interrupts from SOS
>   * loader.conf from EFI partition
>   * acrn.conf (here I've verified that root UUID is correct, and I've
>     had to delete settings related to hypervisor memory log because
>     otherwise it crashed)
>
> Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
> and Leaf Hill CRB.
>
> Best regards,
>
> Dubravko
>
> ------------------------------------------------------------------------
>
> *From:*acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
> Lists.Projectacrn.Org <fengwei.yin=intel.com@...
> <mailto:fengwei.yin=intel.com@...>>
> *Sent:* Wednesday, September 11, 2019 4:41 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi,
>
> On 2019/9/10
下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hello everyone,
>>
>> I work for Exor Embedded and we are developing a new board with an
>> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
>> following the instructions on
>> https://projectacrn.github.io/latest/index.htmlI've managed to get it
>> running, dmesg among other things says:
>>
>> Hypervisor detected: ACRN
>> ...
>> ACRNTrace: Initialized acrn trace module with 4 cpu
>> ...
>> systemd[1]: Detected virtualization acrn.
>>
>> However, it's not really usable. We have following issues:
>>
>>   * mouse cursor jumps around in a weird way, like there is some problem
>>     with switching video buffers
>>   * we can't open the Terminal in any way, there's always a message:
>>     "There was an error creating a child process for this terminal.
>>     Failed to open PTY: Permission denied"
>>   * network doesn't work
>>   * everything works much slower than usual
>>
>> When not running with hypervisor, everything works.
>> We've noticed some differences between dmesgs when running the same
>> kernel with and without hypervisor, but so far we were unable to figure
>> out if any of these differences is significant.
>>
>> We would appreciate if you have any hints for us, either what is exactly
>> our problem, or at least how we should continue to debug the system.
> Could you share:
>     1. ACRN last git commit hash
>     2. dmesg output in your SOS
>     3. the output of cat /proc/interrupt in your SOS
>
> Regards
> Yin, Fengwei
>
>>
>> Best regards,
>> Dubravko Moravski
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
> 5915402413 <tel:+38%205915402413>>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>       
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>
>
> *Dubravko Moravski*
>
> /SW engineering/
>
> *Exor Embedded S.r.l.*
>
> p:
>
>       
>
> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
> <tel:+38%205915402413>
>
> a:
>
>       
>
> Slavonska avenija, 50, Zagreb, Croatia, 10000
>
> w:
>
>       
>
> exorint.com <https://exorint.com/>
>
>       
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
> di stampare pensa ai costi ambientali. Please consider the environment
> before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>




Re: Setting up ACRN on a new board with Apollo Lake

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi everyone,

SOS console/terminal can't be opened at all; the window actually opens but it says "There was an error creating a child process for this terminal. Failed to open PTY: Permission denied" and we can't do anything. It is the same if we use launcher on the left side on the screen, or ctrl+alt+t. Also ssh doesn't work (as well as networking in general) so we can't connect remotely either.

200K JPG image uploaded to https://www.amazon.com/clouddrive/share/lWcIh67jPu7uhWrOzTBlOfcVwkfZ4ofYyEYTenbJrjo (ignore the date, I don't know why it's three days in the past).
I don't see why the console would have anything to do with the hypervisor, but simply it doesn't work with it, and works without it, so it's a symptom of some problem.

These all work when not running with the hypervisor.

Regarding ACPI and LPC, unfortunately we are not that familiar with ACPI, I have uploaded our disassembled ACPI dump to https://www.amazon.com/clouddrive/share/cQik3C5S0ey40L9L2EtMumII322kV0rjmJ34dBpHjD3, 35 kB RAR file. String "LPC" occurs in dsdt, ssdt1, 3 and 4.

Best regards,
Dubravko


From: acrn-users@... <acrn-users@...> on behalf of Yin, Fengwei via Lists.Projectacrn.Org <fengwei.yin=intel.com@...>
Sent: Thursday, September 12, 2019 3:21 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
 
Hi Dubravko,



Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi,
>
> We haven't observed any issues on the UP2 either, except one, screen
> updates been erratic and mouse pointer jumping around.
>
> We've just followed instructions from
> https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
> automated setup with some manual fine tuning.
> I guess since we are using the same software and UP2 works (mostly)
> fine, the causes of our problems must be either different hardware or
> BIOS. But the trouble is figuring out what exactly.
>
> All mentioned issues are for the Service OS. We can't even
> install/launch Guest OS because the Terminal no longer works after we
> restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei

>
> Now that the memory log issue is fixed, could we use that to get
> additional debug information? If yes, how can I save that information
> for later? (We managed to create some scripts so we can save data like
> dmesg even when terminal can't be invoked.)
>
> Best regards,
> Dubravko
>
>
> ------------------------------------------------------------------------
> *From:* acrn-users@...
> <acrn-users@...> on behalf of Liu, Fuzhong via
> Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
> *Sent:* Thursday, September 12, 2019 11:47 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> No observed the issue on the UP2 board in our lab.
>
> Need detail steps to reproduce.
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> [mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
> *Sent:* Thursday, September 12, 2019 4:42 PM
> *To:* acrn-users@...
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Have you observed the same problems on the UP2 board?
>
> Thanks,
>
> Geoffroy
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> <acrn-users@...
> <mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
> *Sent:* Thursday, September 12, 2019 7:15 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Dubravko
>
> All issues you mentioned  are for Service OS, right?
>
> Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000” issue
>
> BR.
>
> Fuzhong
>
> *From:*acrn-users@...
> <mailto:acrn-users@...>
> [mailto:acrn-users@...] *On Behalf Of *Dubravko
> Moravski | Exor Embedded S.r.l.
> *Sent:* Wednesday, September 11, 2019 8:01 PM
> *To:* acrn-users@...
> <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi Yin Fenghwei,
>
> Thank you for replying.
>
> ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I am
> getting identical behavior with ACRN that I've compiled, as well as with
> ACRN installed by Clear Linux (version id 30770).
>
> I'm not sure if this mailing lists allows attachments, so I've uploaded
> the requested files to
> https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKgNisSUkQdR03mr4sY(17
> kB zip file, containing 4 text files). The zip archive contains:
>
>   * dmesg output from SOS
>   * /proc/interrupts from SOS
>   * loader.conf from EFI partition
>   * acrn.conf (here I've verified that root UUID is correct, and I've
>     had to delete settings related to hypervisor memory log because
>     otherwise it crashed)
>
> Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
> and Leaf Hill CRB.
>
> Best regards,
>
> Dubravko
>
> ------------------------------------------------------------------------
>
> *From:*acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>> on behalf of Yin, Fengwei via
> Lists.Projectacrn.Org <fengwei.yin=intel.com@...
> <mailto:fengwei.yin=intel.com@...>>
> *Sent:* Wednesday, September 11, 2019 4:41 AM
> *To:* acrn-users@...
> <mailto:acrn-users@...><acrn-users@...
> <mailto:acrn-users@...>>
> *Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake
>
> Hi,
>
> On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hello everyone,
>>
>> I work for Exor Embedded and we are developing a new board with an
>> Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
>> following the instructions on
>> https://projectacrn.github.io/latest/index.htmlI've managed to get it
>> running, dmesg among other things says:
>>
>> Hypervisor detected: ACRN
>> ...
>> ACRNTrace: Initialized acrn trace module with 4 cpu
>> ...
>> systemd[1]: Detected virtualization acrn.
>>
>> However, it's not really usable. We have following issues:
>>
>>   * mouse cursor jumps around in a weird way, like there is some problem
>>     with switching video buffers
>>   * we can't open the Terminal in any way, there's always a message:
>>     "There was an error creating a child process for this terminal.
>>     Failed to open PTY: Permission denied"
>>   * network doesn't work
>>   * everything works much slower than usual
>>
>> When not running with hypervisor, everything works.
>> We've noticed some differences between dmesgs when running the same
>> kernel with and without hypervisor, but so far we were unable to figure
>> out if any of these differences is significant.
>>
>> We would appreciate if you have any hints for us, either what is exactly
>> our problem, or at least how we should continue to debug the system.
> Could you share:
>     1. ACRN last git commit hash
>     2. dmesg output in your SOS
>     3. the output of cat /proc/interrupt in your SOS
>
> Regards
> Yin, Fengwei
>
>>
>> Best regards,
>> Dubravko Moravski
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659 <tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
> 5915402413 <tel:+38%205915402413>>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>       
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the
>> environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>>
>
> *Dubravko Moravski*
>
> /SW engineering/
>
> *Exor Embedded S.r.l.*
>
> p:
>
>       
>
> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413
> <tel:+38%205915402413>
>
> a:
>
>       
>
> Slavonska avenija, 50, Zagreb, Croatia, 10000
>
> w:
>
>       
>
> exorint.com <https://exorint.com/>
>
>       
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8e9a8c2b3e4c.jpg
>
> imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?header=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-8a58-167a2248b283.jpg Prima
> di stampare pensa ai costi ambientali. Please consider the environment
> before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the
> environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>





Re: Setting up ACRN on a new board with Apollo Lake

Conghui Chen
 

Hi Fengwei,

On 9/12/2019 7:32 PM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi,

We haven't observed any issues on the UP2 either, except one, screen
updates been erratic and mouse pointer jumping around.

We've just followed instructions from
https://projectacrn.github.io/latest/getting-started/apl-nuc.html,
automated setup with some manual fine tuning.
I guess since we are using the same software and UP2 works (mostly)
fine, the causes of our problems must be either different hardware or
BIOS. But the trouble is figuring out what exactly.

All mentioned issues are for the Service OS. We can't even
install/launch Guest OS because the Terminal no longer works after we
restart the board with ACRN.
We checked the log you shared and didn't see anything unusual. We had
one issue which could make serial console use polling mode. I suspected
you hit that issue. But from the interrupt info, everything is fine (
there will be irq flooding if we hit that issue).We need more info.
It seems irq 4 is shared between two uarts.
For ttyS0, it's interrupt is default as 4, and it is correct in the log.
[ 1.084989] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
But the other pci uart ttyS1 also use irq 4.
[ 1.173842] dw-apb-uart.5: ttyS1 at MMIO 0x91228000 (irq = 4, base_baud = 115200) is a 16550A

ttyS0 is a vuart simulated in hypervisor, and ttyS1 is a physical uart, should not share the same irq.
Currently, I'm not sure, why the lpc uart will choose to use irq 4. Is it possible that there is some lpc definition
in acpi table?

Regards,
Conghui.

When you say "Terminal no longer work", do you mean the SOS console
can't work any more? Or it could work for a while and then can't work?

Regards
Yin, Fengwei


Now that the memory log issue is fixed, could we use that to get
additional debug information? If yes, how can I save that information
for later? (We managed to create some scripts so we can save data like
dmesg even when terminal can't be invoked.)

Best regards,
Dubravko


------------------------------------------------------------------------
*From:* acrn-users@...
<acrn-users@...> on behalf of Liu, Fuzhong via
Lists.Projectacrn.Org <fuzhong.liu=intel.com@...>
*Sent:* Thursday, September 12, 2019 11:47 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

No observed the issue on the UP2 board in our lab.

Need detail steps to reproduce.

BR.

Fuzhong

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Geoffroy Van Cutsem
*Sent:* Thursday, September 12, 2019 4:42 PM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Have you observed the same problems on the UP2 board?

Thanks,

Geoffroy

*From:*acrn-users@...
<mailto:acrn-users@...>
<acrn-users@...
<mailto:acrn-users@...>> *On Behalf Of *Liu, Fuzhong
*Sent:* Thursday, September 12, 2019 7:15 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Dubravko

All issues you mentioned  are for Service OS, right?

Clear Linux image newer than 30900 fixed “memmap=2M$0x1FE00000”
issue

BR.

Fuzhong

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Dubravko
Moravski | Exor Embedded S.r.l.
*Sent:* Wednesday, September 11, 2019 8:01 PM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi Yin Fenghwei,

Thank you for replying.

ACRN git commit hash is 555a03db99621c6a9d3bea0933c65db9dee44c46. I
am
getting identical behavior with ACRN that I've compiled, as well as with
ACRN installed by Clear Linux (version id 30770).

I'm not sure if this mailing lists allows attachments, so I've uploaded
the requested files to
https://www.amazon.com/clouddrive/share/V7pJPEUKnBoDd3vkpM0ca4YLEKg
NisSUkQdR03mr4sY(17
kB zip file, containing 4 text files). The zip archive contains:

* dmesg output from SOS
* /proc/interrupts from SOS
* loader.conf from EFI partition
* acrn.conf (here I've verified that root UUID is correct, and I've
had to delete settings related to hypervisor memory log because
otherwise it crashed)

Our board has 8 GB of memory and 64 GB MMC, and is similar to Up Squared
and Leaf Hill CRB.

Best regards,

Dubravko

------------------------------------------------------------------------

*From:*acrn-users@...
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...>> on behalf of Yin, Fengwei via
Lists.Projectacrn.Org <fengwei.yin=intel.com@...
<mailto:fengwei.yin=intel.com@...>>
*Sent:* Wednesday, September 11, 2019 4:41 AM
*To:* acrn-users@...
<mailto:acrn-users@...><acrn-users@...
<mailto:acrn-users@...>>
*Subject:* Re: [acrn-users] Setting up ACRN on a new board with Apollo Lake

Hi,

On 2019/9/10 下午9:00, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello everyone,

I work for Exor Embedded and we are developing a new board with an
Apollo Lake E3940 CPU. We would like to run ACRN on it. Indeed,
following the instructions on
https://projectacrn.github.io/latest/index.htmlI've managed to get it
running, dmesg among other things says:

Hypervisor detected: ACRN
...
ACRNTrace: Initialized acrn trace module with 4 cpu
...
systemd[1]: Detected virtualization acrn.

However, it's not really usable. We have following issues:

   * mouse cursor jumps around in a weird way, like there is some problem
     with switching video buffers
   * we can't open the Terminal in any way, there's always a message:
     "There was an error creating a child process for this terminal.
     Failed to open PTY: Permission denied"
   * network doesn't work
   * everything works much slower than usual

When not running with hypervisor, everything works.
We've noticed some differences between dmesgs when running the same
kernel with and without hypervisor, but so far we were unable to figure
out if any of these differences is significant.

We would appreciate if you have any hints for us, either what is exactly
our problem, or at least how we should continue to debug the system.
Could you share:
   1. ACRN last git commit hash
   2. dmesg output in your SOS
   3. the output of cat /proc/interrupt in your SOS

Regards
Yin, Fengwei


Best regards,
Dubravko Moravski



*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659
<tel:+38%20512455659>>  m:+38 5915402413 <tel:+38
5915402413 <tel:+38%205915402413>>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>




   Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
*Dubravko Moravski*

/SW engineering/

*Exor Embedded S.r.l.*

p:



+38 512455659 <tel:+38%20512455659>  m: +38 5915402413
<tel:+38%205915402413>

a:



Slavonska avenija, 50, Zagreb, Croatia, 10000

w:



exorint.com <https://exorint.com/>



imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?he
ader=quotebody&part=1.4&filename=x-sign_alone_89a12ff2-fa07-4acd-b9a0-8
e9a8c2b3e4c.jpg

imap://fyin@...:993/fetch%3EUID%3E/INBOX%3E265992?he
ader=quotebody&part=1.5&filename=environment-verde_4010c4a3-6fcf-4e6d-
8a58-167a2248b283.jpg Prima
di stampare pensa ai costi ambientali. Please consider the environment
before printing this email.

Privacy <https://www.exorint.com/it/privacy>


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38
5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>




 Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>

861 - 880 of 1252