Hi Dubravko,

I'm not familiar with GVT, and let other guys to comment on this topic.
You meant lgb i211 driver is working on SOS now? I'm sure it has nothing to do with acrn-dm which has impacts on UOS only.

The invisible mouse cursor issue, you meant it works on regular (native) Linux with same SOS kernel? Did you use the same
rootfs with SOS? I feel it more likely a rootfs issue and may not related Hypervisor. We need to look at Hypervisor only
when SOS Kernel + rootfs work on native Linux but not on top of ACRN.

Best Regards,
Zide


On 3/3/20 5:41 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi Zide,
>
> Your advice was helpful again. I have recompiled acrn-dm, edited the launch script, and now UOS works. I'm not exactly
> sure why, but Ethernet works too! I'm using the same board and the same kernel as few days ago. Does acrn-dm somehow
> affect it?
>
> I have changed the launch script like this:
>
>     acrn-dm -A -m $mem_size -s 0:0,hostbridge \
>        -s 5,virtio-console,@stdio:stdio_port \
>        -s 6,virtio-hyper_dmabuf \
>        -s 3,virtio-blk,/home/clear/work/uos/uos.img \
>        -s 4,virtio-net,tap0 \
>        -s 7,virtio-rnd \
>        --ovmf /usr/share/acrn/bios/OVMF.fd \
>        $pm_channel $pm_by_vuart $pm_vuart_node \
>        $logger_setting \
>        --mac_seed $mac_seed \
>        $vm_name
>     }
>
> Apparently it doesn't like this line, now commented out:
>
>     #  -s 2,pci-gvt -G "$2" \
>
> This happens if it's enabled:
>
>     vm_init_vdevs
>     polling 38...
>     Listening 38...
>     pci init hostbridge
>     pci init lpc
>     pci init pci-gvt
>     GVT: open /sys/kernel/gvt/control/create_gvt_instance failed
>     GVT: init failed
>     pci pci-gvt init failed
>     mngr_client_new: Failed to accept from fd 38, err: Invalid argument
>     Stop listening 38...
>     Stop polling 38...
>     Unable to init vdev (2)
>
> Reading the documentation, I'm not precisely sure what GVT does. Would it be beneficial to have it for using more display
> outputs simultaneously and drawing/compositing 2D graphics?
>
> About using the same kernel as acrn in regular Linux and testing Ethernet and the mouse, both work in regular Linux. In
> acrn, the mouse is still functional, but invisible (it's a plain Logitech USB mouse, nothing special about it).
>
> Best regards,
> Dubravko
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Saturday, February 29, 2020 9:04 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
>
> Hi Dubravko,
>
> Since you are using the latest hypervisor code, I’d suggest build your own acrn-dm and use the launch script from the same
> acrn tree to avoid any version compatibility issue.
>
> $ cd acrn-hypervisor
>
> $ make devicemodel
>
> $ scp build/devicemodel/acrn-dm target:/usr/bin
>
> For the script, you may start from acrn-hypervisor/devicemodel/samples/nuc /launch_uos.sh. If it runs into problem, you
> may try customize the script, for example, remove some passthru device from acrn-dm command line if it complains issue of
> initializing that device.
>
> Regarding the Ethernet/mouse driver, did you boot the native Linux with the SOS kernel and the same SOS rootfs? It seems
> not very likely a kernel config issue, but probably it’s still worthy to rule out kernel issue.
>
> Best Regards,
>
> Zide
>
> *From:* acrn-users@... <acrn-users@...> *On Behalf Of *Dubravko Moravski | Exor
> Embedded S.r.l.
> *Sent:* Friday, February 28, 2020 9:57 AM
> *To:* acrn-users@...
> *Cc:* Wang, Hongbo <hongbo.wang@...>; Wang, Yu1 <yu1.wang@...>; Li, Fei1 <fei1.li@...>; Chen, Conghui
> <conghui.chen@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
>
> Hi everyone,
>
> I confirm my ACRN sources did include 65ed6c3529de8b3f3d890e95a7d816afba7bf379 commit. I've tried reverting it - due to
> the subsequent modifications it wasn't trivial but in the end I've managed - the behavior however remained identical,
> "PCIe link lost". If you have any other ideas for me to try, please let me know.
>
> Regarding the invisible mouse pointer, I've found mouse acceleration settings in gnome-tweaks, but they relate only to the
> acceleration of the movement of the mouse pointer. I've checked all the other tweak pages, still I couldn't find anything
> related to hardware acceleration. If you have any other ideas...
>
> Unfortunately I'm also having issues launching User OS. I've followed instructions from here:
> https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html#kbl-nuc-sdc, Set up Reference User VM chapter. Setting it
> up goes well, but in the end it fails to launch:
>
>     clear@clr-sos-guest~/work/uos $ sudo ./launch_uos.sh
>
>     cpu1 online=1
>
>     cpu2 online=1
>
>     cpu3 online=1
>
>     cat: '/sys/class/net/e*/address': No such file or directory
>
>     passed gvt-g optargs low_gm 64, high_gm 448, fence 8
>
>     SW_LOAD: get ovmf path /usr/share/acrn/bios/OVMF.fd, size 0x200000
>
>     pm by vuart node-index = 0
>
>     logger: name=console, level=4
>
>     logger: name=kmsg, level=3
>
>     logger: name=disk, level=5
>
>     vm_create: vm1
>
>     VHM api version 1.0
>
>     vm_setup_memory: size=0x80000000
>
>     open hugetlbfs file /run/hugepage/acrn/huge_lv1/vm1/D279543825D611E8864ECB7A18B34643
>
>     open hugetlbfs file /run/hugepage/acrn/huge_lv2/vm1/D279543825D611E8864ECB7A18B34643
>
>     level 0 free/need pages:0/1 page size:0x200000
>
>     level 1 free/need pages:0/2 page size:0x40000000
>
>     to reserve more free pages:
>
>     to reserve pages (+orig 0): echo 2 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
>
>     to reserve pages (+orig 0): echo 1 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
>
>     now enough free pages are reserved!
>
>     try to setup hugepage with:
>
>              level 0 - lowmem 0x0, biosmem 0x200000, highmem 0x0
>
>              level 1 - lowmem 0x80000000, biosmem 0x0, highmem 0x0
>
>     total_size 0x180000000
>
>     mmap ptr 0x0x7fdbbc4e2000 -> baseaddr 0x0x7fdbc0000000
>
>     mmap 0x80000000@0x7fdbc0000000
>
>     touch 2 pages with pagesz 0x40000000
>
>     mmap 0x200000@0x7fdcbfe00000
>
>     touch 1 pages with pagesz 0x200000
>
>     really setup hugepage with:
>
>              level 0 - lowmem 0x0, biosmem 0x200000, highmem 0x0
>
>              level 1 - lowmem 0x80000000, biosmem 0x0, highmem 0x0
>
>     vm_init_vdevs
>
>     polling 38...
>
>     Listening 38...
>
>     pci init hostbridge
>
>     pci init lpc
>
>     pci init pci-gvt
>
>     GVT: open /sys/kernel/gvt/control/create_gvt_instance failed
>
>     GVT: init failed
>
>     pci pci-gvt init failed
>
>     mngr_client_new: Failed to accept from fd 38, err: Invalid argument
>
>     Stop listening 38...
>
>     Stop polling 38...
>
>     Unable to init vdev (2)
>
> I've used the regular ACRN to test it (without reverted PCIe-related commits). I might have missed some required step in
> configuring ACRN or UOS, I'll re-check what I have done.
>
> Best regards,
>
> Dubravko
>
> *Dubravko Moravski*
>
> /SW engineering/
>
> *Exor Embedded S.r.l.*
>
> p:
>
>       
>
> +38 512455659 <tel:+38%20512455659>  m: +38 5915402413 <tel:+38%205915402413>
>
> a:
>
>       
>
> Slavonska avenija, 50, Zagreb, Croatia, 10000
>
> w:
>
>       
>
> exorint.com <https://exorint.com/>
>
>       
>
>   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
>
> --------------------------------------------------------------------------------------------------------------------------
>
> *From:*acrn-users@... <mailto:acrn-users@...> <acrn-users@...
> <mailto:acrn-users@...>> on behalf of Shuo A Liu via Lists.Projectacrn.Org
> <shuo.a.liu=intel.com@... <mailto:shuo.a.liu=intel.com@...>>
> *Sent:* Friday, February 28, 2020 9:41 AM
> *To:* acrn-users@... <mailto:acrn-users@...> <acrn-users@...
> <mailto:acrn-users@...>>
> *Cc:* Wang, Hongbo <hongbo.wang@... <mailto:hongbo.wang@...>>; Wang, Yu1 <yu1.wang@...
> <mailto:yu1.wang@...>>; Li, Fei1 <fei1.li@... <mailto:fei1.li@...>>; Chen, Conghui
> <conghui.chen@... <mailto:conghui.chen@...>>
> *Subject:* Re: [acrn-users] Getting ACRN to work
>
> Hi Dubravko,
>
> I learned from Fei that one of his patch(merged, for security) might impact PCIe devices in SOS, could you please help
> check if your HV include it?
>
> commit 65ed6c3529de8b3f3d890e95a7d816afba7bf379
>
> Author: Li Fei1 <fei1.li@... <mailto:fei1.li@...>>
>
> Date:   Thu Dec 5 22:51:06 2019 +0800
>
>      hv: vpci: trap PCIe ECAM access for SOS
>
>      SOS will use PCIe ECAM access PCIe external configuration space. HV should trap this
>
>      access for security(Now pre-launched VM doesn't want to support PCI ECAM; post-launched
>
>      VM trap PCIe ECAM access in DM).
>
>      Besides, update PCIe MMCONFIG region to be owned by hypervisor and expose and pass through
>
>      platform hide PCI devices by BIOS to SOS.
>
>      Tracked-On: #3475
>
>      Signed-off-by: Li Fei1 <fei1.li@... <mailto:fei1.li@...>>
>
> You can have a quick try to revert it firstly if it is included.
>
> Thanks
>
> shuo
>
> *From:* acrn-users@... <mailto:acrn-users@...> <acrn-users@...
> <mailto:acrn-users@...>> *On Behalf Of *Dubravko Moravski | Exor Embedded S.r.l.
> *Sent:* Thursday, February 27, 2020 02:19
> *To:* acrn-users@... <mailto:acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
>
> Hi Zide,
>
> Thank you for explaining all the kernel and Ubuntu options. It looks like all the use cases we are interested in are covered.
>
> I've got a replacement board and I'm continuing with ACRN. I've followed your instructions regarding rebuilding the
> kernel, in which I've enabled the Intel igb network driver we need for the I211 chip.
> The new kernel mostly works, however igb driver doesn't (and we really need the network):
>
>     clear@clr-sos-guest~/work/acrn-kernel $ cat ../../dmesg_bad.txt | grep igb
>
>     [    3.708554] calling  igb_init_module+0x0/0x4d @ 1
>
>     [    3.709199] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
>
>     [    3.710159] igb: Copyright (c) 2007-2014 Intel Corporation.
>
>     [    3.802746] igb 0000:04:00.0 0000:04:00.0 (uninitialized): *PCIe link lost*
>
>     [    4.120516] igb 0000:04:00.0: PHY reset is blocked due to SOL/IDER session.
>
>     [    7.504456] igb 0000:04:00.0: Invalid MAC Address
>
>     [    7.505779] igb: probe of 0000:04:00.0 failed with error -5
>
>     [    7.507585] initcall igb_init_module+0x0/0x4d returned 0 after 3709354 usecs
>
> The "PCIe link lost" message originates in igb_main.c, approx line 740:
>
>     u32 igb_rd32(struct e1000_hw *hw, u32 reg)
>
>     {
>
>              struct igb_adapter *igb = container_of(hw, struct igb_adapter, hw);
>
>              u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr);
>
>              u32 value = 0;
>
>              if (E1000_REMOVED(hw_addr))
>
>                      return ~value;
>
>              value = readl(&hw_addr[reg]);
>
>              /* reads should not return all F's */
>
>              if (!(~value) && (!reg || !(~readl(hw_addr)))) {
>
>                      struct net_device *netdev = igb->netdev;
>
>                      hw->hw_addr = NULL;
>
>                      netdev_err(netdev, "*PCIe link lost*\n");
>
>              }
>
>              return value;
>
>     }
>
> I think readl() for PCIe devices is basically a single PCIe transfer; in other words it's already the lowest possible
> level, looking from the software side of things, so it doesn't look like I could do much debugging here.
>
> In regular Clear Linux, the driver consistently works and there are never any link issues:
>
>     clear@clr-sos-guest~ $ sudo dmesg | grep -i igb
>
>     Password:
>
>     [    4.473282] calling  igb_init_module+0x0/0x1000 [igb] @ 317
>
>     [    4.473306] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k
>
>     [    4.473307] igb: Copyright (c) 2007-2014 Intel Corporation.
>
>     [    4.519304] igb 0000:04:00.0: added PHC on eth0
>
>     [    4.519307] igb 0000:04:00.0: Intel(R) Gigabit Ethernet Network Connection
>
>     [    4.519310] igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 00:30:d8:05:55:84
>
>     [    4.519314] igb 0000:04:00.0: eth0: PBA No: FFFFFF-0FF
>
>     [    4.519316] igb 0000:04:00.0: Using MSI-X interrupts. 2 rx queue(s), 2 tx queue(s)
>
>     [    4.519418] initcall igb_init_module+0x0/0x1000 [igb] returned 0 after 33714 usecs
>
>     [    5.390826] igb 0000:04:00.0 enp4s0: renamed from eth0
>
>     [    8.766862] igb 0000:04:00.0 enp4s0: igb: enp4s0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
>
> So is there some setting in ACRN that can affect PCIe communication, that I need to adjust for (external) PCIe devices?
>
> Also with my ACRN-kernel, the mouse works but the cursor is invisible. In regular Clear Linux with 'native' kernel, with
> the same file system and GUI and system settings, the mouse works and the cursor is visible.
>
> Best regards,
>
> Dubravko
>
>