Date   

Couple of questions regarding 'acrn-kernel'

Geoffroy Van Cutsem
 

Hi folks,

A couple of questions regarding the https://github.com/projectacrn/acrn-kernel repository:
* Where can I find instructions on how to use it?
* Should we switch the Clear Linux reference package to use it?
(I vote 'yes' to this latter question :-))

Geoffroy


Re: Couple of questions regarding 'acrn-kernel'

Tzeng, Tonny <tonny.tzeng@...>
 

Hi Geoffroy,

To build SOS using the acrn-kernel repo, you could follow the instructions on this wiki page [1]. Though it provides flexibility to customize your own kernel, I am concerned to replace the PKT kernels with it, as it's going to fragment "Intel kernel". IMO

[1] https://wiki.ith.intel.com/display/OTCCWPQA/%5BIntegration%5D+BKM+for+engineer+build+from+Clearlinux+MRB+build

Regards,
Tonny

-----Original Message-----
From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Geoffroy Van Cutsem
Sent: Monday, June 18, 2018 4:39 PM
To: acrn-users@...
Subject: [acrn-users] Couple of questions regarding 'acrn-kernel'

Hi folks,

A couple of questions regarding the https://github.com/projectacrn/acrn-kernel repository:
* Where can I find instructions on how to use it?
* Should we switch the Clear Linux reference package to use it?
(I vote 'yes' to this latter question :-))

Geoffroy


Re: Couple of questions regarding 'acrn-kernel'

Geoffroy Van Cutsem
 

-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Tzeng, Tonny
Sent: Monday, June 18, 2018 11:05 AM
To: acrn-users@...
Subject: Re: [acrn-users] Couple of questions regarding 'acrn-kernel'

Hi Geoffroy,

To build SOS using the acrn-kernel repo, you could follow the instructions on
this wiki page [1]. Though it provides flexibility to customize your own kernel,
I am concerned to replace the PKT kernels with it, as it's going to fragment
"Intel kernel". IMO
I think we need to look at this from the perspective of what gets tested. It's a great idea to have this kernel easily available here so people can recompile and modify it. My concern is how do we ensure that it is the same as what is being tested by the validation teams.


[1]
https://wiki.ith.intel.com/display/OTCCWPQA/%5BIntegration%5D+BKM+for
+engineer+build+from+Clearlinux+MRB+build
We need similar instructions posted on the public website (this link is Intel-only).


Regards,
Tonny
-----Original Message-----
From: acrn-users@... [mailto:acrn-
users@...] On Behalf Of Geoffroy Van Cutsem
Sent: Monday, June 18, 2018 4:39 PM
To: acrn-users@...
Subject: [acrn-users] Couple of questions regarding 'acrn-kernel'

Hi folks,

A couple of questions regarding the https://github.com/projectacrn/acrn-
kernel repository:
* Where can I find instructions on how to use it?
* Should we switch the Clear Linux reference package to use it?
(I vote 'yes' to this latter question :-))

Geoffroy






question regarding acrn-hypervisor vioapic implementation

Abdul
 

Hi folks, 
 
I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked. 
 
A condition in hypervisor at the following location:
https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396
is preventing the physical ioapic tables from being updated. 
 
The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?
 
Thanks, 
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Xu, Anthony
 

Hi Abdul,

 

May I ask in which OS you saw the behavior ( update IOAPIC entry without mask it)?

 

Thanks,

Anthony

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Thursday, June 28, 2018 6:34 AM
To: acrn-users@...
Subject: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Hi folks, 

 

I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked. 

 

A condition in hypervisor at the following location:

https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396

is preventing the physical ioapic tables from being updated. 

 

The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?

 

Thanks, 
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Chen, Jason CJ
 

Hi, Abdul,

 

It’s designed for supporting pass-thru devices, when vioapic RTE got unmask, it will go to check if a pass-thru entry valid for this vioapic pin, if yes, then native IOAPIC will got updated and the mapping between native IOAPIC pin & VIOAPIC pin is created.

Similar behavior is done for RTE mask & update.

 

Thanks & Best Regards,

 

Jason Chen

 

SSG -> OTC -> Intel Auto Hypervisor Team

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Friday, June 29, 2018 7:24 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Hi Abdul,

 

May I ask in which OS you saw the behavior ( update IOAPIC entry without mask it)?

 

Thanks,

Anthony

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Thursday, June 28, 2018 6:34 AM
To: acrn-users@...
Subject: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Hi folks, 

 

I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked. 

 

A condition in hypervisor at the following location:

https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396

is preventing the physical ioapic tables from being updated. 

 

The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?

 

Thanks, 
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Abdul
 

Hi Anthony, 

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository. 

Platform: Apollo Lake 

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:
ACRN:\>vioapic 0
 
PIN     VEC     DM      DEST    TM      DELM    IRR     MASK
14      0x3E    logic   0x1     level   1       0       0
ACRN:\>dump_ioapic
 
IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM      TM      DELM    IRR     MASK
014     014     0x01000000      0x0000098E      0x8E    0x01    logic   edge    1       0       0
This IRQ corresponds to device INT3452
Logs:
root@:Dom0 ~ $ cat /proc/interrupts 
            CPU0       
  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01, INT3452:02, INT3452:03
As a result sos is not receiving interrupts from the device. 

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000
[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000
[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e                                 
[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e                               ----->   Update is made while the interrupt is unmasked
[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Xu, Anthony
 

Jason/Fengwei,

 

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

 

Anthony

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Friday, June 29, 2018 5:42 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Hi Anthony, 

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository. 

Platform: Apollo Lake 

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:

ACRN:\>vioapic 0

 

PIN     VEC     DM      DEST    TM      DELM    IRR     MASK

14      0x3E    logic   0x1     level   1       0       0

ACRN:\>dump_ioapic

 

IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM      TM      DELM    IRR     MASK

014     014     0x01000000      0x0000098E      0x8E    0x01    logic   edge    1       0       0

This IRQ corresponds to device INT3452

Logs:

root@:Dom0 ~ $ cat /proc/interrupts 

            CPU0       

  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01, INT3452:02, INT3452:03

As a result sos is not receiving interrupts from the device. 

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e                                 

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e                               ----->   Update is made while the interrupt is unmasked

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Chen, Jason CJ
 

Ok, it does one case that we don’t take care – the update is done during the interrupt is unmasked. Let’s figure out one solution for this.

 

Thanks & Best Regards,

 

Jason Chen

 

SSG -> OTC -> Intel Auto Hypervisor Team

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Saturday, June 30, 2018 6:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Jason/Fengwei,

 

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

 

Anthony

 

 

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Friday, June 29, 2018 5:42 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

 

Hi Anthony, 

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository. 

Platform: Apollo Lake 

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:

ACRN:\>vioapic 0

 

PIN     VEC     DM      DEST    TM      DELM    IRR     MASK

14      0x3E    logic   0x1     level   1       0       0

ACRN:\>dump_ioapic

 

IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM      TM      DELM    IRR     MASK

014     014     0x01000000      0x0000098E      0x8E    0x01    logic   edge    1       0       0

This IRQ corresponds to device INT3452

Logs:

root@:Dom0 ~ $ cat /proc/interrupts 

            CPU0       

  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01, INT3452:02, INT3452:03

As a result sos is not receiving interrupts from the device. 

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e                                 

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e                               ----->   Update is made while the interrupt is unmasked

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Yin, Fengwei <fengwei.yin@...>
 

This change update the interrupt from edge triggered to low level
triggered. Which is unusual case.

Is it possible that this is related with workaround in linux kernel
ioapic for version 0x11?

Regards
Yin, Fengwei

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done during the interrupt is unmasked. Let’s figure out one solution for this.
Thanks & Best Regards,
Jason Chen
SSG -> OTC -> Intel Auto Hypervisor Team
*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation
Jason/Fengwei,
Our assumption is not correct.
HV needs to handle RTL change even when it is unmasked.
Anthony
*From:*acrn-users@... <mailto:acrn-users@...> [mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@... <mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation
Hi Anthony,
I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository.
Platform: Apollo Lake
I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:
/ACRN:\>vioapic 0/
/PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/
/14      0x3E    logic   0x1     level   1       0       0/
/ACRN:\>dump_ioapic/
/IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM     TM      DELM    IRR     MASK/
/014     014     0x01000000      0x0000098E      0x8E    0x01 logic   edge    1       0       0/
This IRQ corresponds to device INT3452
Logs:
/root@:Dom0 ~ $ cat /proc/interrupts /
/            CPU0 /
/  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
INT3452:02, INT3452:03/
As a result sos is not receiving interrupts from the device.
I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e                        ----->   Update is made while the interrupt
is unmasked/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Abdul
 
Edited

It happens because in the acpi table - DSDT - the interrupt is defined as level triggered active low for this device.
Snippet from DSDT:

Name (_ADR, Zero) // _ADR: Address
Name (_HID, "INT3452") // _HID: Hardware ID
Name (_CID, "INT3452") // _CID: Compatible ID
Name (_DDN, "General Purpose Input/Output (GPIO) Controller - North") // _DDN: DOS Device Name
Name (_UID, One) // _UID: Unique ID
Name (LINK, "\_SB.GPO0")
Name (RBUF, ResourceTemplate ()
{
Memory32Fixed (ReadWrite,
    0x00000000, // Address Base
    0x00004000, // Address Length
    _Y07)
Interrupt (ResourceConsumer, Level, ActiveLow, Shared, ,, )
{
   0x0000000E,
}
})

Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Chen, Jason CJ
 

Hi, Fengwei,

Just checked the ioapic workaround code, it will first do MASK before update. So I think it's not related with this IOAPIC workaround.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

-----Original Message-----
From: Yin, Fengwei
Sent: Sunday, July 1, 2018 9:28 PM
To: acrn-users@...
Cc: Chen, Jason CJ <jason.cj.chen@...>
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

This change update the interrupt from edge triggered to low level triggered.
Which is unusual case.

Is it possible that this is related with workaround in linux kernel ioapic for
version 0x11?

Regards
Yin, Fengwei

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done
during the interrupt is unmasked. Let’s figure out one solution for this.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Hi Anthony,

I am using sos kernel from
https://github.com/projectacrn/acrn-kernel/commits/master commit-id:
9bba4539d654af06d4642f9773e5444da7ee055d
and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the
physical ioapic RTL is not updated
Logs:

/ACRN:\>vioapic 0/

/PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/

/14      0x3E    logic   0x1     level   1       0       0/

/ACRN:\>dump_ioapic/

/IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM
    TM      DELM    IRR     MASK/

/014     014     0x01000000      0x0000098E      0x8E    0x01
logic   edge    1       0       0/

This IRQ corresponds to device INT3452

Logs:

/root@:Dom0 ~ $ cat /proc/interrupts /

/            CPU0 /

/  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
INT3452:02, INT3452:03/

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while
the interrupt was unmasked
Logs:

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
                       ----->   Update is made while the interrupt
is unmasked/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Yin, Fengwei <fengwei.yin@...>
 

Yes. It's not related with IOAPIC workaround. Here is what I got from SOS:

^M[ 0.041021] ioapic_set_affinity: yfw: eu.w1: 0x931, eu.w2: 0x1000000^M
^M[ 0.041085] ioapic_set_affinity: yfw: eu.w1: 0x930, eu.w2: 0x1000000^M
^M[ 0.041097] ioapic_set_affinity: yfw: eu.w1: 0x933, eu.w2: 0x1000000^M
^M[ 0.041136] ioapic_set_affinity: yfw: eu.w1: 0x934, eu.w2: 0x1000000^M
^M[ 0.041155] ioapic_set_affinity: yfw: eu.w1: 0x935, eu.w2: 0x1000000^M
^M[ 0.041195] ioapic_set_affinity: yfw: eu.w1: 0x936, eu.w2: 0x1000000^M
^M[ 0.041235] ioapic_set_affinity: yfw: eu.w1: 0x937, eu.w2: 0x1000000^M
^M[ 0.041275] ioapic_set_affinity: yfw: eu.w1: 0x938, eu.w2: 0x1000000^M
^M[ 0.041316] ioapic_set_affinity: yfw: eu.w1: 0x1a939, eu.w2: 0x1000000^M
^M[ 0.041335] ioapic_set_affinity: yfw: eu.w1: 0x93a, eu.w2: 0x1000000^M
^M[ 0.041377] ioapic_set_affinity: yfw: eu.w1: 0x93b, eu.w2: 0x1000000^M
^M[ 0.041417] ioapic_set_affinity: yfw: eu.w1: 0x93c, eu.w2: 0x1000000^M
^M[ 0.041457] ioapic_set_affinity: yfw: eu.w1: 0x93d, eu.w2: 0x1000000^M
^M[ 0.041497] ioapic_set_affinity: yfw: eu.w1: 0x93e, eu.w2: 0x1000000^M

^M[ 0.041567] yfw: io apic_write: 0x93e
^M[ 0.041570] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G U 4.14.47-sos+ #5^M
^M[ 0.041571] Call Trace:^M
^M[ 0.041574] dump_stack+0x85/0xc4^M
^M[ 0.041578] io_apic_write.cold.21+0x17/0x20^M
^M[ 0.041581] ? ioapic_set_affinity.cold.22+0x30/0x3f^M
^M[ 0.041584] ? setup_ioapic_dest+0xf5/0x109^M
^M[ 0.041586] ? do_early_param+0x8e/0x8e^M
^M[ 0.041588] ? native_smp_cpus_done+0xe4/0xeb^M
^M[ 0.041590] ? kernel_init_freeable+0xfe/0x20e^M
^M[ 0.041592] ? rest_init+0xb5/0xb5^M
^M[ 0.041595] ? kernel_init+0xa/0xf3^M
^M[ 0.041597] ? ret_from_fork+0x3a/0x50^M
^M[ 0.041632] ioapic_set_affinity: yfw: eu.w1: 0x93f, eu.w2: 0x1000000^M


We could see the first 0x93e is written in very early stage. It's
updated to 0xa93e when related devices is probed:
^M[ 0.117972] yfw: io apic_write: 0xa93e
^M[ 0.117975] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G U 4.14.47-sos+ #5^M
^M[ 0.117975] Call Trace:^M
^M[ 0.117979] dump_stack+0x85/0xc4^M
^M[ 0.117982] io_apic_write.cold.21+0x17/0x20^M
^M[ 0.117985] ? mp_irqdomain_activate+0x34/0x50^M
^M[ 0.117988] ? irq_domain_activate_irq+0x1a/0x30^M
^M[ 0.117990] ? __irq_startup+0x16/0x70^M
^M[ 0.117993] ? irq_startup+0x50/0xe0^M
^M[ 0.117996] ? __setup_irq+0x5e9/0x650^M
^M[ 0.117999] ? request_threaded_irq+0xf5/0x160^M
^M[ 0.118000] ? devm_request_threaded_irq+0x6d/0xc0^M
^M[ 0.118000] ? intel_gpio_irq_wake+0x80/0x80^M
^M[ 0.118000] ? intel_pinctrl_probe+0x586/0x680^M
^M[ 0.118000] ? platform_drv_probe+0x38/0x90^M
^M[ 0.118000] ? driver_probe_device+0x2a3/0x400^M
^M[ 0.118000] ? __driver_attach+0xe0/0xe0^M
^M[ 0.118000] ? bus_for_each_drv+0x64/0x90^M
^M[ 0.118000] ? __device_attach+0xb5/0x130^M
^M[ 0.118000] ? bus_probe_device+0x94/0xb0^M
^M[ 0.118000] ? device_add+0x3d2/0x5f0^M
^M[ 0.118000] ? preempt_count_add+0x68/0x90^M
^M[ 0.118000] ? __insert_resource+0x3f/0xd0^M
^M[ 0.118000] ? platform_device_add+0x111/0x260^M
^M[ 0.118000] ? platform_device_register_full+0xc3/0x100^M
^M[ 0.118000] ? acpi_create_platform_device+0x1d8/0x270^M

And the second update was not programmed to native ioapic.

It looks like our current implementation can't handle this case.

Regards
Yin, Fengwei

On 07/03/2018 02:18 PM, Chen, Jason CJ wrote:
Hi, Fengwei,
Just checked the ioapic workaround code, it will first do MASK before update. So I think it's not related with this IOAPIC workaround.
Thanks & Best Regards,
Jason Chen
SSG -> OTC -> Intel Auto Hypervisor Team

-----Original Message-----
From: Yin, Fengwei
Sent: Sunday, July 1, 2018 9:28 PM
To: acrn-users@...
Cc: Chen, Jason CJ <jason.cj.chen@...>
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

This change update the interrupt from edge triggered to low level triggered.
Which is unusual case.

Is it possible that this is related with workaround in linux kernel ioapic for
version 0x11?

Regards
Yin, Fengwei

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done
during the interrupt is unmasked. Let’s figure out one solution for this.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Hi Anthony,

I am using sos kernel from
https://github.com/projectacrn/acrn-kernel/commits/master commit-id:
9bba4539d654af06d4642f9773e5444da7ee055d
and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the
physical ioapic RTL is not updated
Logs:

/ACRN:\>vioapic 0/

/PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/

/14      0x3E    logic   0x1     level   1       0       0/

/ACRN:\>dump_ioapic/

/IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM
    TM      DELM    IRR     MASK/

/014     014     0x01000000      0x0000098E      0x8E    0x01
logic   edge    1       0       0/

This IRQ corresponds to device INT3452

Logs:

/root@:Dom0 ~ $ cat /proc/interrupts /

/            CPU0 /

/  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
INT3452:02, INT3452:03/

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while
the interrupt was unmasked
Logs:

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
                       ----->   Update is made while the interrupt
is unmasked/

/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
trigger-mode reg/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

Thanks!
Best Regards,
Abdul


Re: question regarding acrn-hypervisor vioapic implementation

Abdul
 
Edited

During the second update function "acpi_create_platform_device" calls "acpi_dev_get_resources" which reads the acpi resources and updates the trigger mode and polarity of the interrupt.

Best Regards,
Abdul


Running ACRN on Intel Atom® Processor E3845

Joel <joeldhopkins@...>
 

Hello All,

I would like to run ACRN on the MinnowBoard Turbot QUAD Core Board which has the Intel Atom® Processor E3845.

I understand that this target is not tested, but I am up for the challenge.

However, if anyone has done this, or can tell me about what roadblocks are ahead, I'd appreciate it any advice.

I see that  ACRN checks CPU capabilities at runtime so perhaps obstacles will become apparent first time I try to boot it.

Best Regards,
Joel




Re: Running ACRN on Intel Atom® Processor E3845

Jack Ren
 

Hi Joel,

                Welcome to try it on MinnowBoard and any patches are welcome!

                If you need any help, please send it to the mail list, we will answer your questions.

 

--

Best regards,

Jack Ren

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Joel
Sent: Monday, July 30, 2018 05:30
To: acrn-users@...
Subject: [acrn-users] Running ACRN on Intel Atom® Processor E3845

 

Hello All,

I would like to run ACRN on the MinnowBoard Turbot QUAD Core Board which has the Intel Atom® Processor E3845.

I understand that this target is not tested, but I am up for the challenge.

However, if anyone has done this, or can tell me about what roadblocks are ahead, I'd appreciate it any advice.

I see that  ACRN checks CPU capabilities at runtime so perhaps obstacles will become apparent first time I try to boot it.

Best Regards,
Joel



How to pass through off-chip USB controller

Tzeng, Tonny <tonny.tzeng@...>
 

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny


Re: How to pass through off-chip USB controller

Zhai, Edwin <edwin.zhai@...>
 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...



On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny


-- 
Best Rgds,
Edwin


Re: How to pass through off-chip USB controller

Tzeng, Tonny <tonny.tzeng@...>
 

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny



-- 
Best Rgds,
Edwin


Re: How to pass through off-chip USB controller

Zhai, Edwin <edwin.zhai@...>
 

Tonny,

The config is ok.

Could you pls. try gdb to catch the seg fault? Just replace the 'acrn-dm' with 'gdb --args acrn-dm...' in your launch script.

Did you paste all your console output when launch UOS?  And 'lspci -s 1:0.0 -vvv' in SOS should give some helpful info.

thanks,

Edwin


On 2018/7/30 11:21, Tzeng, Tonny wrote:

Hi Edwin,

 

Thanks for asking, here are my setup:

- HV/DM: f815415

- SOS: Clear 24030 w/ 4.14.57-69 kernel

- List of PCI devices

...

00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)

00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)

01:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 20)

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

- List of USB devices

...

Bus 003 Device 002: ID 03e7:2150 Intel Myriad VPU [Movidius Neural Compute Stick]

...

- I am using my own customized launch_uos.sh as attached. It’s basically logically the same as the reference script, I just added “-s 26,passthru,01/0/0” to pass through the USB controller.

- I got Segmentation fault during the DM boots the UOS kernel. The same Segfault happened both in Ubuntu UOS and ClearLinux UOS.

cpu1 online=1

cpu2 online=1

cpu3 online=1

creating tap device acrn_tap-vm3

adding new tap device to acrn-br0...

...

[    6.965056]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    6.980481]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.010356]  [01] BAD  20 20 01 aa 20 20 01 aa 20 20 01 aa 20 20 01 aa

[    7.043065] xhci_hcd 0000:00:1a.0: hcc params 0x200071e9 hci version 0x100 qu

./launch_uos.sh: line 4:   449 Segmentation fault      (core dumped) acrn-dm -A

s 1:0,lpc -l com1,stdio -s 2,pci-gvt -G "$4" -s 3,virtio-blk,$6.img -s 4,virtio-

ty:pty_port -s 6,virtio-hyper_dmabuf -s 11,wdt-i6300esb -k "$8" -B "maxcpus=$2 $

" $extra_arguments $vm_name

 

Regards,

Tonny

 

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Zhai, Edwin
Sent: Monday, July 30, 2018 10:18 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to pass through off-chip USB controller

 

Tonny,

Could you pls. paste the detailed error mesg.

We do have some passthrough rules user need follow, e.g. have reset capability for PCIe dev...

 

 

On 2018/7/30 9:07, Tzeng, Tonny wrote:

Hi there,

 

I got an Aaeon’s AI Core, which integrates the Movidius Myriad2 chip and a USB3 controller on a PCIe card, and I’d like to drive it from the UOS on the UP2 board. I follow the Getting Started Guide and I am able to pass through the built-in USB controller (PCI 00:15.0) to the UOS. Since my Movidius PCIe card is presented to the system as a PCI 01:00.0 device, I tried to use the similar way to pass through it to the UOS, but the DM aborts and can’t launch the UOS.

 

Is there any restrictions to pass through PCI devices to UOS? I’ve tried to unbind the device thru the pci-stub, and  add “-s 21,passthru,1/0/0” option to the DM, but the DM can’t launch the UOS with these modifications. What’s the proper procedures to pass through an *external* USB controller? Any pointers would be appreciated.

 

Regards,

Tonny



-- 
Best Rgds,
Edwin

-- 
Best Rgds,
Edwin