How to pass through off-chip USB controller


Tzeng, Tonny <tonny.tzeng@...>
 

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny


Zhai, Edwin
 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...



On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny


-- 
Best Rgds,
Edwin


Tzeng, Tonny <tonny.tzeng@...>
 

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny



-- 
Best Rgds,
Edwin


Zhai, Edwin
 

Tonny,

The config is ok.

Could you pls. try gdb to catch the seg fault? Just replace the 'acrn-dm' with 'gdb --args acrn-dm...' in your launch script.

Did you paste all your console output when launch UOS?  And 'lspci -s 1:0.0 -vvv' in SOS should give some helpful info.

thanks,

Edwin


On 2018/7/30 11:21, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny



-- 
Best Rgds,
Edwin

-- 
Best Rgds,
Edwin


Tzeng, Tonny <tonny.tzeng@...>
 

Hi Edwin,

 

Thanks for helping out. The `lspci –s 1:0.0 –vvv’ generates below output:

 

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20) (prog-if 30 [XHCI])

       Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller

       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

       Latency: 0, Cache Line Size: 64 bytes

       Interrupt: pin A routed to IRQ 22

       Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]

       Region 2: Memory at 91511000 (64-bit, non-prefetchable) [size=4K]

       Region 4: Memory at 91510000 (64-bit, non-prefetchable) [size=4K]

       Capabilities: <access denied>

       Kernel driver in use: xhci_hcd

       Kernel modules: xhci_pci

 

Debugging DM using gdb indicates the program stops at hw/pci/passthrough.c:560. Any idea what could be the cause? I don’t know how to capture the whole screen output from the UOS kernel, the detailed gdb log is attached for your inspection. Thanks.

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 1:46 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

The config is ok.

Could you pls. try gdb to catch the seg fault? Just replace the 'acrn-dm' with 'gdb --args acrn-dm...' in your launch script.

Did you paste all your console output when launch UOS?  And 'lspci -s 1:0.0 -vvv' in SOS should give some helpful info.

thanks,

Edwin

 

On 2018/7/30 11:21, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny




-- 
Best Rgds,
Edwin



-- 
Best Rgds,
Edwin


Zhai, Edwin
 

You didn't dump the pci capabilities, so pls. use sudo for lspci.

The error comes from MSIx accessing, and need debug...


On 2018/7/30 15:39, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for helping out. The `lspci –s 1:0.0 –vvv’ generates below output:

 

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20) (prog-if 30 [XHCI])

       Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller

       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

       Latency: 0, Cache Line Size: 64 bytes

       Interrupt: pin A routed to IRQ 22

       Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]

       Region 2: Memory at 91511000 (64-bit, non-prefetchable) [size=4K]

       Region 4: Memory at 91510000 (64-bit, non-prefetchable) [size=4K]

       Capabilities: <access denied>

       Kernel driver in use: xhci_hcd

       Kernel modules: xhci_pci

 

Debugging DM using gdb indicates the program stops at hw/pci/passthrough.c:560. Any idea what could be the cause? I don’t know how to capture the whole screen output from the UOS kernel, the detailed gdb log is attached for your inspection. Thanks.

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 1:46 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

The config is ok.

Could you pls. try gdb to catch the seg fault? Just replace the 'acrn-dm' with 'gdb --args acrn-dm...' in your launch script.

Did you paste all your console output when launch UOS?  And 'lspci -s 1:0.0 -vvv' in SOS should give some helpful info.

thanks,

Edwin

 

On 2018/7/30 11:21, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny




-- 
Best Rgds,
Edwin



-- 
Best Rgds,
Edwin

-- 
Best Rgds,
Edwin


Tzeng, Tonny <tonny.tzeng@...>
 

Hi Edwin,

 

Oops, the lspci output generated by root this time. Thanks.

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20) (prog-if 30 [XHCI])

       Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller

       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

       Latency: 0, Cache Line Size: 64 bytes

       Interrupt: pin A routed to IRQ 22

       Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]

       Region 2: Memory at 91511000 (64-bit, non-prefetchable) [size=4K]

       Region 4: Memory at 91510000 (64-bit, non-prefetchable) [size=4K]

       Capabilities: [40] Power Management version 3

             Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold+)

             Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

       Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+

             Address: 0000000000000000  Data: 0000

       Capabilities: [70] Express (v2) Endpoint, MSI 00

             DevCap:      MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us

                    ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W

             DevCtl:      CorrErr- NonFatalErr- FatalErr- UnsupReq-

                    RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+

                    MaxPayload 256 bytes, MaxReadReq 512 bytes

             DevSta:      CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-

             LnkCap:      Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 unlimited

                    ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-

             LnkCtl:      ASPM Disabled; RCB 64 bytes Disabled- CommClk+

                    ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

             LnkSta:      Speed 5GT/s (ok), Width x1 (ok)

                    TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

             DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported

                    AtomicOpsCap: 32bit- 64bit- 128bitCAS-

             DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled

                    AtomicOpsCtl: ReqEn-

             LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-

                    Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

                    Compliance De-emphasis: -6dB

             LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-

                    EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-

       Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-

             Vector table: BAR=2 offset=00000000

             PBA: BAR=4 offset=00000000

       Capabilities: [100 v1] Advanced Error Reporting

             UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

             UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

             UESvrt:      DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-

             CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-

             CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+

             AERCap:      First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-

                    MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-

             HeaderLog: 00000000 00000000 00000000 00000000

       Kernel driver in use: xhci_hcd

       Kernel modules: xhci_pci

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 4:14 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

You didn't dump the pci capabilities, so pls. use sudo for lspci.

The error comes from MSIx accessing, and need debug...

 

On 2018/7/30 15:39, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for helping out. The `lspci –s 1:0.0 –vvv’ generates below output:

 

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20) (prog-if 30 [XHCI])

       Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller

       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

       Latency: 0, Cache Line Size: 64 bytes

       Interrupt: pin A routed to IRQ 22

       Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]

       Region 2: Memory at 91511000 (64-bit, non-prefetchable) [size=4K]

       Region 4: Memory at 91510000 (64-bit, non-prefetchable) [size=4K]

       Capabilities: <access denied>

       Kernel driver in use: xhci_hcd

       Kernel modules: xhci_pci

 

Debugging DM using gdb indicates the program stops at hw/pci/passthrough.c:560. Any idea what could be the cause? I don’t know how to capture the whole screen output from the UOS kernel, the detailed gdb log is attached for your inspection. Thanks.

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 1:46 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

The config is ok.

Could you pls. try gdb to catch the seg fault? Just replace the 'acrn-dm' with 'gdb --args acrn-dm...' in your launch script.

Did you paste all your console output when launch UOS?  And 'lspci -s 1:0.0 -vvv' in SOS should give some helpful info.

thanks,

Edwin

 

On 2018/7/30 11:21, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny





-- 
Best Rgds,
Edwin




-- 
Best Rgds,
Edwin



-- 
Best Rgds,
Edwin


Geoffroy Van Cutsem
 

Any update on this?

Thanks!

Geoffroy


Wu, Binbin
 

Hi,

/The following instruction will take PCI device 'USB Host 00:15.0' as an example to show the set-up steps./

/
/

/If lspci is not found in your clearlinux sos, you can install the tool by/

/swupd bundle-add sysadmin-basic
/

/
/


[edit
<http://los-vmm.sc.intel.com/mediawiki//index.php?title=VTd_Enabling_Guide&action=edit§ion=3>]Hide
PCI device in SOS

1) Find out the physical BDF (bus:device.function) for USB Host

$ lspci
<<<<<<<output>>>>>>>
...
*00:15.0* USB controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series USB xHCI (rev 0b)
...

2) Find out the numeric PCI ID for USB Host

$ lspci -n
<<<<<<<output>>>>>>>
...
*00:15.0* 0c03:*8086:5aa8* (rev 0b)
...

3) Hide the PCI device in SOS via stub driver

Execute the following commands to hide the PCI device "00:15.0" in SOS

echo "8086 5aa8" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:00:15.0" > /sys/bus/pci/devices/0000:00:15.0/driver/unbind
echo "0000:00:15.0" > /sys/bus/pci/drivers/pci-stub/bind

You could double-check whether the PCI device is hidden successfully via the following command.

$ cat /sys/bus/pci/devices/0000:00:15.0/uevent

If it is done successfully, the driver will be shown as 'pci-stub'.

Example:

root@jiechen-xubuntu:~# cat /sys/bus/pci/devices/0000:00:15.0/uevent
*DRIVER=pci-stub*
PCI_CLASS=C0330
PCI_ID=8086:5AA8
PCI_SUBSYS_ID=0000:0000
PCI_SLOT_NAME=0000:00:15.0
MODALIAS=pci:v00008086d00005AA8sv00000000sd00000000bc0Csc03i30


[edit
<http://los-vmm.sc.intel.com/mediawiki//index.php?title=VTd_Enabling_Guide&action=edit§ion=4>]Add
parameters in the boot-up command for UOS

Add the following parameters in the boot-up command to pass PCI device 0:15.0 to guest @ virtual PCI 0:7.0

-s 7,passthru,0/15/0

You could change '7' to any other number as long as it's not used by other devices yet.

On 9/6/2018 8:47 PM, Geoffroy Van Cutsem wrote:

Any update on this?

Thanks!

Geoffroy


Wu, Binbin
 

Resend due to format issue.

If lspci is not found in your clearlinux sos, you can install the tool by:
$ swupd bundle-add sysadmin-basic

The following instruction will take PCI device 'USB Host 00:15.0' as an example to show the set-up steps.

Hide PCI device in SOS
1) Find out the physical BDF (bus:device.function) for USB Host

$ lspci
<<<<<<<output>>>>>>>
...
00:15.0 USB controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series USB xHCI (rev 0b)
...
2) Find out the numeric PCI ID for USB Host

$ lspci -n
<<<<<<<output>>>>>>>
...
00:15.0 0c03: 8086:5aa8 (rev 0b)
...
3) Hide the PCI device in SOS via stub driver

Execute the following commands to hide the PCI device "00:15.0" in SOS

echo "8086 5aa8" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:00:15.0" > /sys/bus/pci/devices/0000:00:15.0/driver/unbind
echo "0000:00:15.0" > /sys/bus/pci/drivers/pci-stub/bind
You could double-check whether the PCI device is hidden successfully via the following command.

$ cat /sys/bus/pci/devices/0000:00:15.0/uevent
If it is done successfully, the driver will be shown as 'pci-stub'.

Example:

root@jiechen-xubuntu:~# cat /sys/bus/pci/devices/0000:00:15.0/uevent
DRIVER=pci-stub
PCI_CLASS=C0330
PCI_ID=8086:5AA8
PCI_SUBSYS_ID=0000:0000
PCI_SLOT_NAME=0000:00:15.0
MODALIAS=pci:v00008086d00005AA8sv00000000sd00000000bc0Csc03i30

4) Add parameters in the boot-up command for UOS
Add the following parameters in the boot-up command to pass PCI device 0:15.0 to guest @ virtual PCI 0:7.0

-s 7,passthru,0/15/0
You could change '7' to any other number as long as it's not used by other devices yet.

On 9/6/2018 8:47 PM, Geoffroy Van Cutsem wrote:

Any update on this?

Thanks!

Geoffroy


Geoffroy Van Cutsem
 

Hi Binbin,

Thanks for jumping in! I think Tonny (who initiated this thread) was further than that, you suspected

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Thursday, September 6, 2018 3:49 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Resend due to format issue.

If lspci is not found in your clearlinux sos, you can install the tool by:
$ swupd bundle-add sysadmin-basic

The following instruction will take PCI device 'USB Host 00:15.0' as an example
to show the set-up steps.

Hide PCI device in SOS
1) Find out the physical BDF (bus:device.function) for USB Host

$ lspci
<<<<<<<output>>>>>>>
...
00:15.0 USB controller: Intel Corporation Celeron N3350/Pentium
N4200/Atom E3900 Series USB xHCI (rev 0b) ...
2) Find out the numeric PCI ID for USB Host

$ lspci -n
<<<<<<<output>>>>>>>
...
00:15.0 0c03: 8086:5aa8 (rev 0b)
...
3) Hide the PCI device in SOS via stub driver

Execute the following commands to hide the PCI device "00:15.0" in SOS

echo "8086 5aa8" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:00:15.0" > /sys/bus/pci/devices/0000:00:15.0/driver/unbind
echo "0000:00:15.0" > /sys/bus/pci/drivers/pci-stub/bind
You could double-check whether the PCI device is hidden successfully via the
following command.

$ cat /sys/bus/pci/devices/0000:00:15.0/uevent
If it is done successfully, the driver will be shown as 'pci-stub'.

Example:

root@jiechen-xubuntu:~# cat /sys/bus/pci/devices/0000:00:15.0/uevent
DRIVER=pci-stub
PCI_CLASS=C0330
PCI_ID=8086:5AA8
PCI_SUBSYS_ID=0000:0000
PCI_SLOT_NAME=0000:00:15.0
MODALIAS=pci:v00008086d00005AA8sv00000000sd00000000bc0Csc03i30

4) Add parameters in the boot-up command for UOS Add the following
parameters in the boot-up command to pass PCI device
0:15.0 to guest @ virtual PCI 0:7.0

-s 7,passthru,0/15/0
You could change '7' to any other number as long as it's not used by other
devices yet.


On 9/6/2018 8:47 PM, Geoffroy Van Cutsem wrote:

Any update on this?

Thanks!

Geoffroy



Geoffroy Van Cutsem
 

Ops, hit "send" by mistake earlier

Thanks for jumping in! I think Tonny (who initiated this thread) was further than that. Edwin had indicated the error came from an MSIx access error and had asked for the 'lspci' output to debug this. Tonny provided it here: https://lists.projectacrn.org/g/acrn-users/message/23. I haven't seen any follow-up after that and that was what I was asking in my mail below (unfortunately, I responded from the web interface and it did not include any of the thread history :()

Thanks,
Geoffroy

-----Original Message-----
From: VanCutsem, Geoffroy
Sent: Thursday, September 6, 2018 10:46 PM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: RE: [acrn-users] How to pass through off-chip USB controller

Hi Binbin,

Thanks for jumping in! I think Tonny (who initiated this thread) was further
than that, you suspected


-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Thursday, September 6, 2018 3:49 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Resend due to format issue.

If lspci is not found in your clearlinux sos, you can install the tool by:
$ swupd bundle-add sysadmin-basic

The following instruction will take PCI device 'USB Host 00:15.0' as
an example to show the set-up steps.

Hide PCI device in SOS
1) Find out the physical BDF (bus:device.function) for USB Host

$ lspci
<<<<<<<output>>>>>>>
...
00:15.0 USB controller: Intel Corporation Celeron N3350/Pentium
N4200/Atom E3900 Series USB xHCI (rev 0b) ...
2) Find out the numeric PCI ID for USB Host

$ lspci -n
<<<<<<<output>>>>>>>
...
00:15.0 0c03: 8086:5aa8 (rev 0b)
...
3) Hide the PCI device in SOS via stub driver

Execute the following commands to hide the PCI device "00:15.0" in SOS

echo "8086 5aa8" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:00:15.0" > /sys/bus/pci/devices/0000:00:15.0/driver/unbind
echo "0000:00:15.0" > /sys/bus/pci/drivers/pci-stub/bind
You could double-check whether the PCI device is hidden successfully
via the following command.

$ cat /sys/bus/pci/devices/0000:00:15.0/uevent
If it is done successfully, the driver will be shown as 'pci-stub'.

Example:

root@jiechen-xubuntu:~# cat /sys/bus/pci/devices/0000:00:15.0/uevent
DRIVER=pci-stub
PCI_CLASS=C0330
PCI_ID=8086:5AA8
PCI_SUBSYS_ID=0000:0000
PCI_SLOT_NAME=0000:00:15.0
MODALIAS=pci:v00008086d00005AA8sv00000000sd00000000bc0Csc03i30

4) Add parameters in the boot-up command for UOS Add the following
parameters in the boot-up command to pass PCI device
0:15.0 to guest @ virtual PCI 0:7.0

-s 7,passthru,0/15/0
You could change '7' to any other number as long as it's not used by
other devices yet.


On 9/6/2018 8:47 PM, Geoffroy Van Cutsem wrote:

Any update on this?

Thanks!

Geoffroy



Zhai, Edwin
 


Geoffroy,

Sorry, I seemed miss Tonny's reply.

From the lspci output from him, this controller support MSI-x, which probably trigger potential issue in ACRN side.


Geoffroy/Tonny,

Are you using the latest ACRN release? Do you have the source code of ACRN? If so, could you pls.  paste your code of ' hw/pci/passthrough.c:560' ?

Binbin,

Any potential issue for this MSI-x? I suspect this issue comes from invalid pointer access in MSI-x emulation code.

       Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-

             Vector table: BAR=2 offset=00000000

             PBA: BAR=4 offset=00000000


The pci config of this dev from Tonny is:

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20) (prog-if 30 [XHCI])

       Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller

       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

       Latency: 0, Cache Line Size: 64 bytes

       Interrupt: pin A routed to IRQ 22

       Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]

       Region 2: Memory at 91511000 (64-bit, non-prefetchable) [size=4K]

       Region 4: Memory at 91510000 (64-bit, non-prefetchable) [size=4K]

       Capabilities: [40] Power Management version 3

             Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold+)

             Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

       Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+

             Address: 0000000000000000  Data: 0000

       Capabilities: [70] Express (v2) Endpoint, MSI 00

             DevCap:      MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us

                    ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W

             DevCtl:      CorrErr- NonFatalErr- FatalErr- UnsupReq-

                    RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+

                    MaxPayload 256 bytes, MaxReadReq 512 bytes

             DevSta:      CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-

             LnkCap:      Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 unlimited

                    ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-

             LnkCtl:      ASPM Disabled; RCB 64 bytes Disabled- CommClk+

                    ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

             LnkSta:      Speed 5GT/s (ok), Width x1 (ok)

                    TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

             DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported

                    AtomicOpsCap: 32bit- 64bit- 128bitCAS-

             DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled

                    AtomicOpsCtl: ReqEn-

             LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-

                    Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

                    Compliance De-emphasis: -6dB

             LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-

                    EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-

       Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-

             Vector table: BAR=2 offset=00000000

             PBA: BAR=4 offset=00000000

       Capabilities: [100 v1] Advanced Error Reporting

             UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

             UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

             UESvrt:      DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-

             CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-

             CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+

             AERCap:      First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-

                    MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-

             HeaderLog: 00000000 00000000 00000000 00000000

       Kernel driver in use: xhci_hcd

       Kernel modules: xhci_pci
On 2018/9/6 20:47, Geoffroy Van Cutsem wrote:

Any update on this?

Thanks!

Geoffroy


-- 
Best Rgds,
Edwin


Wu, Binbin
 

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether hw/pci/passthrough.c:560 is the line of code in bold?

static void
msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev *ptdev,
uint64_t offset, int size, uint64_t data)
{
struct pci_vdev *dev;
struct msix_table_entry *entry;
uint8_t *dest8;
uint16_t *dest16;
uint32_t *dest32;
uint64_t *dest64;
size_t entry_offset;
uint32_t vector_control;
int index;
 
dev = ptdev->dev;
if (offset >= dev->msix.pba_offset &&
    offset < dev->msix.pba_offset + dev->msix.pba_size) {
switch (size) {
case 1:
dest8 = (uint8_t *)(dev->msix.pba_page + offset -
    dev->msix.pba_page_offset);
*dest8 = data;
break;
case 2:
dest16 = (uint16_t *)(dev->msix.pba_page + offset -
    dev->msix.pba_page_offset);
*dest16 = data;
break;
case 4:
dest32 = (uint32_t *)(dev->msix.pba_page + offset -
    dev->msix.pba_page_offset);
*dest32 = data;                //is hw/pci/passthrough.c:560 this line of code?
break;
case 8:
dest64 = (uint64_t *)(dev->msix.pba_page + offset -
    dev->msix.pba_page_offset);
*dest64 = data;
break;
default:
break;
}
return;
}
 


Geoffroy Van Cutsem
 

Hi Binbin, Edwin,

 

The error Tonny got at the time was with this combination (so not the latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

 

Looking it up on Github, this is the line you’re asking for: https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4-140000p/devicemodel/hw/pci/passthrough.c#L560

 

And yes, that’s the one you have below J

Thanks,

Geoffroy

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Wu, Binbin
Sent: Friday, September 7, 2018 9:16 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev *ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

 

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest32 = data;                //is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}

 


Wu, Binbin
 

Hi Geoffroy,

Thanks for providing the info.
After checking the code, I did found a logic error when handling msix table read/write in passthrough.
I have sent out a patch to fix the issue to acrn-dev@... for review.

On 9/7/2018 9:50 PM, Geoffroy Van Cutsem wrote:

Hi Binbin, Edwin,

The error Tonny got at the time was with this combination (so not the latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

Looking it up on Github, this is the line you’re asking for: https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4-140000p/devicemodel/hw/pci/passthrough.c#L560

And yes, that’s the one you have below J

Thanks,

Geoffroy

*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Wu, Binbin
*Sent:* Friday, September 7, 2018 9:16 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] How to pass through off-chip USB controller

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev *ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

**dest32 = data; *//is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}


Geoffroy Van Cutsem
 

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Sunday, September 9, 2018 11:57 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Hi Geoffroy,

Thanks for providing the info.
After checking the code, I did found a logic error when handling msix table
read/write in passthrough.
I have sent out a patch to fix the issue to acrn-dev@... for
review.
Thanks Binbin! I guess that's the one: https://lists.projectacrn.org/g/acrn-dev/message/11368?

We will keep an eye on it (to see when it gets merged) and perhaps even test it beforehand on our side if we get a chance!

Thanks,
Geoffroy


On 9/7/2018 9:50 PM, Geoffroy Van Cutsem wrote:

Hi Binbin, Edwin,

The error Tonny got at the time was with this combination (so not the
latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

Looking it up on Github, this is the line you’re asking for:
https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4-140
000p/devicemodel/hw/pci/passthrough.c#L560

And yes, that’s the one you have below J

Thanks,

Geoffroy

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Wu, Binbin
*Sent:* Friday, September 7, 2018 9:16 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] How to pass through off-chip USB
controller

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether
hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev
*ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

**dest32 = data; *//is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}



Wu, Binbin
 

Hi Geoffroy,

Yes, and the PR has been sent out.
https://github.com/projectacrn/acrn-hypervisor/pull/1210

On 9/10/2018 6:36 PM, Geoffroy Van Cutsem wrote:

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Sunday, September 9, 2018 11:57 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Hi Geoffroy,

Thanks for providing the info.
After checking the code, I did found a logic error when handling msix table
read/write in passthrough.
I have sent out a patch to fix the issue to acrn-dev@... for
review.
Thanks Binbin! I guess that's the one: https://lists.projectacrn.org/g/acrn-dev/message/11368?

We will keep an eye on it (to see when it gets merged) and perhaps even test it beforehand on our side if we get a chance!

Thanks,
Geoffroy

On 9/7/2018 9:50 PM, Geoffroy Van Cutsem wrote:
Hi Binbin, Edwin,

The error Tonny got at the time was with this combination (so not the
latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

Looking it up on Github, this is the line you’re asking for:
https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4-140
000p/devicemodel/hw/pci/passthrough.c#L560

And yes, that’s the one you have below J

Thanks,

Geoffroy

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Wu, Binbin
*Sent:* Friday, September 7, 2018 9:16 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] How to pass through off-chip USB
controller

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether
hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev
*ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

**dest32 = data; *//is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}


Geoffroy Van Cutsem
 

Thanks Binbin!

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Tuesday, September 11, 2018 7:47 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Hi Geoffroy,

Yes, and the PR has been sent out.
https://github.com/projectacrn/acrn-hypervisor/pull/1210


On 9/10/2018 6:36 PM, Geoffroy Van Cutsem wrote:

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Sunday, September 9, 2018 11:57 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Hi Geoffroy,

Thanks for providing the info.
After checking the code, I did found a logic error when handling msix
table read/write in passthrough.
I have sent out a patch to fix the issue to
acrn-dev@... for review.
Thanks Binbin! I guess that's the one: https://lists.projectacrn.org/g/acrn-
dev/message/11368?

We will keep an eye on it (to see when it gets merged) and perhaps even
test it beforehand on our side if we get a chance!

Thanks,
Geoffroy

On 9/7/2018 9:50 PM, Geoffroy Van Cutsem wrote:
Hi Binbin, Edwin,

The error Tonny got at the time was with this combination (so not
the latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

Looking it up on Github, this is the line you’re asking for:
https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4-1
40
000p/devicemodel/hw/pci/passthrough.c#L560

And yes, that’s the one you have below J

Thanks,

Geoffroy

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Wu, Binbin
*Sent:* Friday, September 7, 2018 9:16 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] How to pass through off-chip USB
controller

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether
hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev
*ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

**dest32 = data; *//is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}




Tzeng, Tonny <tonny.tzeng@...>
 

Thanks Binbin, the Movidius UP AI Core [1] now can be passed through to the User OS. Thanks for the great help!!

[1] https://up-shop.org/featured/204-up-ai-core.html

Regards,
Tonny

-----Original Message-----
From: VanCutsem, Geoffroy
Sent: Tuesday, September 11, 2018 2:19 PM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: RE: [acrn-users] How to pass through off-chip USB controller

Thanks Binbin!

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Tuesday, September 11, 2018 7:47 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB controller

Hi Geoffroy,

Yes, and the PR has been sent out.
https://github.com/projectacrn/acrn-hypervisor/pull/1210


On 9/10/2018 6:36 PM, Geoffroy Van Cutsem wrote:

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Wu, Binbin
Sent: Sunday, September 9, 2018 11:57 AM
To: acrn-users@...
Cc: Tzeng, Tonny <tonny.tzeng@...>
Subject: Re: [acrn-users] How to pass through off-chip USB
controller

Hi Geoffroy,

Thanks for providing the info.
After checking the code, I did found a logic error when handling
msix table read/write in passthrough.
I have sent out a patch to fix the issue to
acrn-dev@... for review.
Thanks Binbin! I guess that's the one:
https://lists.projectacrn.org/g/acrn-
dev/message/11368?

We will keep an eye on it (to see when it gets merged) and perhaps
even
test it beforehand on our side if we get a chance!

Thanks,
Geoffroy

On 9/7/2018 9:50 PM, Geoffroy Van Cutsem wrote:
Hi Binbin, Edwin,

The error Tonny got at the time was with this combination (so not
the latest as of today):

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

Looking it up on Github, this is the line you’re asking for:
https://github.com/projectacrn/acrn-hypervisor/blob/acrn-2018w30.4
-1
40
000p/devicemodel/hw/pci/passthrough.c#L560

And yes, that’s the one you have below J

Thanks,

Geoffroy

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Wu,
Binbin
*Sent:* Friday, September 7, 2018 9:16 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] How to pass through off-chip USB
controller

Hi Tonny,
Which version of your clearlinux SOS?
Is there any chance that can you confirm that whether
hw/pci/passthrough.c:560 is the line of code in bold?

static void

msix_table_write(struct vmctx *ctx, int vcpu, struct passthru_dev
*ptdev,

uint64_t offset, int size, uint64_t data)

{

struct pci_vdev *dev;

struct msix_table_entry *entry;

uint8_t *dest8;

uint16_t *dest16;

uint32_t *dest32;

uint64_t *dest64;

size_t entry_offset;

uint32_t vector_control;

int index;

dev = ptdev->dev;

if (offset >= dev->msix.pba_offset &&

    offset < dev->msix.pba_offset + dev->msix.pba_size) {

switch (size) {

case 1:

dest8 = (uint8_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest8 = data;

break;

case 2:

dest16 = (uint16_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest16 = data;

break;

case 4:

dest32 = (uint32_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

**dest32 = data; *//is hw/pci/passthrough.c:560 this line of code?

break;

case 8:

dest64 = (uint64_t *)(dev->msix.pba_page + offset -

    dev->msix.pba_page_offset);

*dest64 = data;

break;

default:

break;

}

return;

}