acrn-users@lists.projectacrn.org | question regarding acrn-hypervisor vioapic implementation

Abdul

#4

Hi folks,

I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked.

A condition in hypervisor at the following location:

https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396

is preventing the physical ioapic tables from being updated.

The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?

Thanks,
Abdul

Xu, Anthony

#5

Hi Abdul,

May I ask in which OS you saw the behavior ( update IOAPIC entry without mask it)?

Thanks,

Anthony

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Thursday, June 28, 2018 6:34 AM
To: acrn-users@...
Subject: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Hi folks,

I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked.

A condition in hypervisor at the following location:

https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396

is preventing the physical ioapic tables from being updated.

The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?

Thanks,
Abdul

Chen, Jason CJ

#6

Hi, Abdul,

It’s designed for supporting pass-thru devices, when vioapic RTE got unmask, it will go to check if a pass-thru entry valid for this vioapic pin, if yes, then native IOAPIC will got updated and the mapping between native IOAPIC pin & VIOAPIC pin is created.

Similar behavior is done for RTE mask & update.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Friday, June 29, 2018 7:24 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Hi Abdul,

May I ask in which OS you saw the behavior ( update IOAPIC entry without mask it)?

Thanks,

Anthony

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Thursday, June 28, 2018 6:34 AM
To: acrn-users@...
Subject: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Hi folks,

I have come across a guest behavior wherein guest updates the ioapic tables while the interrupt is unmasked.

A condition in hypervisor at the following location:

https://github.com/projectacrn/acrn-hypervisor/blob/4ec690fde3c336dfad5977ff9c13fe786013ade0/hypervisor/arch/x86/guest/vioapic.c#L396

is preventing the physical ioapic tables from being updated.

The hypervisor is assuming at the above mentioned location that physical ioapic table entries would only be updated if interrupt masking state is changed. Can someone point me to from where does this assumption originates?

Thanks,
Abdul

Abdul

#7

Hi Anthony,

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:

ACRN:\>vioapic 0

PIN VEC DM DEST TM DELM IRR MASK

14 0x3E logic 0x1 level 1 0 0

ACRN:\>dump_ioapic

IRQ PIN RTE.HI32 RTE.LO32 VEC DST DM TM DELM IRR MASK

014 014 0x01000000 0x0000098E 0x8E 0x01 logic edge 1 0 0

This IRQ corresponds to device INT3452

Logs:

root@:Dom0 ~ $ cat /proc/interrupts

CPU0

14: 0 IO-APIC 14-fasteoi INT3452:00, INT3452:01, INT3452:02, INT3452:03

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e -----> Update is made while the interrupt is unmasked

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

Thanks!
Best Regards,
Abdul

Xu, Anthony

#8

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Friday, June 29, 2018 5:42 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Hi Anthony,

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:

ACRN:\>vioapic 0

PIN VEC DM DEST TM DELM IRR MASK

14 0x3E logic 0x1 level 1 0 0

ACRN:\>dump_ioapic

IRQ PIN RTE.HI32 RTE.LO32 VEC DST DM TM DELM IRR MASK

014 014 0x01000000 0x0000098E 0x8E 0x01 logic edge 1 0 0

This IRQ corresponds to device INT3452

Logs:

root@:Dom0 ~ $ cat /proc/interrupts

CPU0

14: 0 IO-APIC 14-fasteoi INT3452:00, INT3452:01, INT3452:02, INT3452:03

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e -----> Update is made while the interrupt is unmasked

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

Thanks!
Best Regards,
Abdul

Chen, Jason CJ

#9

Ok, it does one case that we don’t take care – the update is done during the interrupt is unmasked. Let’s figure out one solution for this.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Xu, Anthony
Sent: Saturday, June 30, 2018 6:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

From: acrn-users@... [mailto:acrn-users@...] On Behalf Of Abdul
Sent: Friday, June 29, 2018 5:42 AM
To: acrn-users@...
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation

Hi Anthony,

I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:

ACRN:\>vioapic 0

PIN VEC DM DEST TM DELM IRR MASK

14 0x3E logic 0x1 level 1 0 0

ACRN:\>dump_ioapic

IRQ PIN RTE.HI32 RTE.LO32 VEC DST DM TM DELM IRR MASK

014 014 0x01000000 0x0000098E 0x8E 0x01 logic edge 1 0 0

This IRQ corresponds to device INT3452

Logs:

root@:Dom0 ~ $ cat /proc/interrupts

CPU0

14: 0 IO-APIC 14-fasteoi INT3452:00, INT3452:01, INT3452:02, INT3452:03

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e -----> Update is made while the interrupt is unmasked

[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic trigger-mode reg

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e

Thanks!
Best Regards,
Abdul

Yin, Fengwei <fengwei.yin@...>

#10

This change update the interrupt from edge triggered to low level
triggered.  Which is unusual case.

Is it possible that this is related with workaround in linux kernel
ioapic for version 0x11?

Regards
Yin, Fengwei

Show quoted text

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done during the interrupt is unmasked. Let’s figure out one solution for this.
Thanks & Best Regards,
Jason Chen
SSG -> OTC -> Intel Auto Hypervisor Team
*From:*acrn-users@... [mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation
Jason/Fengwei,
Our assumption is not correct.
HV needs to handle RTL change even when it is unmasked.
Anthony
*From:*acrn-users@... <mailto:acrn-users@...> [mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@... <mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic implementation
Hi Anthony,
I am using sos kernel from https://github.com/projectacrn/acrn-kernel/commits/master commit-id: 9bba4539d654af06d4642f9773e5444da7ee055d and the kernel_config_sos present in the same repository.
Platform: Apollo Lake
I can observe that vioapic RTL is updated for IRQ - 14 but the physical ioapic RTL is not updated
Logs:
/ACRN:\>vioapic 0/
/PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/
/14      0x3E    logic   0x1     level   1       0       0/
/ACRN:\>dump_ioapic/
/IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM     TM      DELM    IRR     MASK/
/014     014     0x01000000      0x0000098E      0x8E    0x01   logic   edge    1       0       0/
This IRQ corresponds to device INT3452
Logs:
/root@:Dom0 ~ $ cat /proc/interrupts /
/            CPU0 /
/  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
    INT3452:02, INT3452:03/
As a result sos is not receiving interrupts from the device.
I dumped the ioapic writes and saw that RTL was being updated while the interrupt was unmasked
Logs:
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e                              ----->   Update is made while the interrupt
    is unmasked/
/[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
/[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/
Thanks!
Best Regards,
Abdul

Abdul

#11
Edited

It happens because in the acpi table - DSDT - the interrupt is defined as level triggered active low for this device.
Snippet from DSDT:

Name (_ADR, Zero) // _ADR: Address
Name (_HID, "INT3452") // _HID: Hardware ID
Name (_CID, "INT3452") // _CID: Compatible ID
Name (_DDN, "General Purpose Input/Output (GPIO) Controller - North") // _DDN: DOS Device Name
Name (_UID, One) // _UID: Unique ID
Name (LINK, "\_SB.GPO0")
Name (RBUF, ResourceTemplate ()
{
Memory32Fixed (ReadWrite,
0x00000000, // Address Base
0x00004000, // Address Length
_Y07)
Interrupt (ResourceConsumer, Level, ActiveLow, Shared, ,, )
{
0x0000000E,
}
})

Best Regards,
Abdul

Chen, Jason CJ

#12

Hi, Fengwei,

Just checked the ioapic workaround code,  it will first do MASK before update. So I think it's not related with this IOAPIC workaround.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

Show quoted text

-----Original Message-----
From: Yin, Fengwei
Sent: Sunday, July 1, 2018 9:28 PM
To: acrn-users@...
Cc: Chen, Jason CJ <jason.cj.chen@...>
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

This change update the interrupt from edge triggered to low level triggered.
Which is unusual case.

Is it possible that this is related with workaround in linux kernel ioapic for
version 0x11?

Regards
Yin, Fengwei

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done
during the interrupt is unmasked. Let’s figure out one solution for this.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Hi Anthony,

I am using sos kernel from
https://github.com/projectacrn/acrn-kernel/commits/master commit-id:
9bba4539d654af06d4642f9773e5444da7ee055d
and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the
physical ioapic RTL is not updated
Logs:

    /ACRN:\>vioapic 0/

    /PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/

    /14      0x3E    logic   0x1     level   1       0       0/

    /ACRN:\>dump_ioapic/

    /IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM
         TM      DELM    IRR     MASK/

    /014     014     0x01000000      0x0000098E      0x8E    0x01
    logic   edge    1       0       0/

This IRQ corresponds to device INT3452

Logs:

    /root@:Dom0 ~ $ cat /proc/interrupts /

    /            CPU0 /

    /  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
    INT3452:02, INT3452:03/

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while
the interrupt was unmasked
Logs:

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /

    /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
                            ----->   Update is made while the interrupt
    is unmasked/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
    trigger-mode reg/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

    /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

Thanks!
Best Regards,
Abdul

Yin, Fengwei <fengwei.yin@...>

#13

Yes. It's not related with IOAPIC workaround. Here is what I got from SOS:

^M[    0.041021] ioapic_set_affinity: yfw: eu.w1: 0x931, eu.w2: 0x1000000^M
^M[    0.041085] ioapic_set_affinity: yfw: eu.w1: 0x930, eu.w2: 0x1000000^M
^M[    0.041097] ioapic_set_affinity: yfw: eu.w1: 0x933, eu.w2: 0x1000000^M
^M[    0.041136] ioapic_set_affinity: yfw: eu.w1: 0x934, eu.w2: 0x1000000^M
^M[    0.041155] ioapic_set_affinity: yfw: eu.w1: 0x935, eu.w2: 0x1000000^M
^M[    0.041195] ioapic_set_affinity: yfw: eu.w1: 0x936, eu.w2: 0x1000000^M
^M[    0.041235] ioapic_set_affinity: yfw: eu.w1: 0x937, eu.w2: 0x1000000^M
^M[    0.041275] ioapic_set_affinity: yfw: eu.w1: 0x938, eu.w2: 0x1000000^M
^M[    0.041316] ioapic_set_affinity: yfw: eu.w1: 0x1a939, eu.w2: 0x1000000^M
^M[    0.041335] ioapic_set_affinity: yfw: eu.w1: 0x93a, eu.w2: 0x1000000^M
^M[    0.041377] ioapic_set_affinity: yfw: eu.w1: 0x93b, eu.w2: 0x1000000^M
^M[    0.041417] ioapic_set_affinity: yfw: eu.w1: 0x93c, eu.w2: 0x1000000^M
^M[    0.041457] ioapic_set_affinity: yfw: eu.w1: 0x93d, eu.w2: 0x1000000^M
^M[    0.041497] ioapic_set_affinity: yfw: eu.w1: 0x93e, eu.w2: 0x1000000^M

^M[    0.041567] yfw: io apic_write: 0x93e
^M[    0.041570] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G     U 4.14.47-sos+ #5^M
^M[    0.041571] Call Trace:^M
^M[    0.041574]  dump_stack+0x85/0xc4^M
^M[    0.041578]  io_apic_write.cold.21+0x17/0x20^M
^M[    0.041581]  ? ioapic_set_affinity.cold.22+0x30/0x3f^M
^M[    0.041584]  ? setup_ioapic_dest+0xf5/0x109^M
^M[    0.041586]  ? do_early_param+0x8e/0x8e^M
^M[    0.041588]  ? native_smp_cpus_done+0xe4/0xeb^M
^M[    0.041590]  ? kernel_init_freeable+0xfe/0x20e^M
^M[    0.041592]  ? rest_init+0xb5/0xb5^M
^M[    0.041595]  ? kernel_init+0xa/0xf3^M
^M[    0.041597]  ? ret_from_fork+0x3a/0x50^M
^M[    0.041632] ioapic_set_affinity: yfw: eu.w1: 0x93f, eu.w2: 0x1000000^M


We could see the first 0x93e is written in very early stage. It's
updated to 0xa93e when related devices is probed:
^M[    0.117972] yfw: io apic_write: 0xa93e
^M[    0.117975] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G     U 4.14.47-sos+ #5^M
^M[    0.117975] Call Trace:^M
^M[    0.117979]  dump_stack+0x85/0xc4^M
^M[    0.117982]  io_apic_write.cold.21+0x17/0x20^M
^M[    0.117985]  ? mp_irqdomain_activate+0x34/0x50^M
^M[    0.117988]  ? irq_domain_activate_irq+0x1a/0x30^M
^M[    0.117990]  ? __irq_startup+0x16/0x70^M
^M[    0.117993]  ? irq_startup+0x50/0xe0^M
^M[    0.117996]  ? __setup_irq+0x5e9/0x650^M
^M[    0.117999]  ? request_threaded_irq+0xf5/0x160^M
^M[    0.118000]  ? devm_request_threaded_irq+0x6d/0xc0^M
^M[    0.118000]  ? intel_gpio_irq_wake+0x80/0x80^M
^M[    0.118000]  ? intel_pinctrl_probe+0x586/0x680^M
^M[    0.118000]  ? platform_drv_probe+0x38/0x90^M
^M[    0.118000]  ? driver_probe_device+0x2a3/0x400^M
^M[    0.118000]  ? __driver_attach+0xe0/0xe0^M
^M[    0.118000]  ? bus_for_each_drv+0x64/0x90^M
^M[    0.118000]  ? __device_attach+0xb5/0x130^M
^M[    0.118000]  ? bus_probe_device+0x94/0xb0^M
^M[    0.118000]  ? device_add+0x3d2/0x5f0^M
^M[    0.118000]  ? preempt_count_add+0x68/0x90^M
^M[    0.118000]  ? __insert_resource+0x3f/0xd0^M
^M[    0.118000]  ? platform_device_add+0x111/0x260^M
^M[    0.118000]  ? platform_device_register_full+0xc3/0x100^M
^M[    0.118000]  ? acpi_create_platform_device+0x1d8/0x270^M

And the second update was not programmed to native ioapic.

It looks like our current implementation can't handle this case.

Regards
Yin, Fengwei

Show quoted text

On 07/03/2018 02:18 PM, Chen, Jason CJ wrote:
Hi, Fengwei,
Just checked the ioapic workaround code,  it will first do MASK before update. So I think it's not related with this IOAPIC workaround.
Thanks & Best Regards,
Jason Chen
SSG -> OTC -> Intel Auto Hypervisor Team

-----Original Message-----
From: Yin, Fengwei
Sent: Sunday, July 1, 2018 9:28 PM
To: acrn-users@...
Cc: Chen, Jason CJ <jason.cj.chen@...>
Subject: Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

This change update the interrupt from edge triggered to low level triggered.
Which is unusual case.

Is it possible that this is related with workaround in linux kernel ioapic for
version 0x11?

Regards
Yin, Fengwei

On 7/1/2018 8:27 PM, Chen, Jason CJ wrote:
Ok, it does one case that we don’t take care – the update is done
during the interrupt is unmasked. Let’s figure out one solution for this.

Thanks & Best Regards,

Jason Chen

SSG -> OTC -> Intel Auto Hypervisor Team

*From:*acrn-users@...
[mailto:acrn-users@...] *On Behalf Of *Xu, Anthony
*Sent:* Saturday, June 30, 2018 6:12 AM
*To:* acrn-users@...
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Jason/Fengwei,

Our assumption is not correct.

HV needs to handle RTL change even when it is unmasked.

Anthony

*From:*acrn-users@...
<mailto:acrn-users@...>
[mailto:acrn-users@...] *On Behalf Of *Abdul
*Sent:* Friday, June 29, 2018 5:42 AM
*To:* acrn-users@...
<mailto:acrn-users@...>
*Subject:* Re: [acrn-users] question regarding acrn-hypervisor vioapic
implementation

Hi Anthony,

I am using sos kernel from
https://github.com/projectacrn/acrn-kernel/commits/master commit-id:
9bba4539d654af06d4642f9773e5444da7ee055d
and the kernel_config_sos present in the same repository.

Platform: Apollo Lake

I can observe that vioapic RTL is updated for IRQ - 14 but the
physical ioapic RTL is not updated
Logs:

     /ACRN:\>vioapic 0/

     /PIN     VEC     DM      DEST    TM      DELM    IRR     MASK/

     /14      0x3E    logic   0x1     level   1       0       0/

     /ACRN:\>dump_ioapic/

     /IRQ     PIN     RTE.HI32        RTE.LO32        VEC     DST     DM
          TM      DELM    IRR     MASK/

     /014     014     0x01000000      0x0000098E      0x8E    0x01
     logic   edge    1       0       0/

This IRQ corresponds to device INT3452

Logs:

     /root@:Dom0 ~ $ cat /proc/interrupts /

     /            CPU0 /

     /  14:          0   IO-APIC   14-fasteoi   INT3452:00, INT3452:01,
     INT3452:02, INT3452:03/

As a result sos is not receiving interrupts from the device.

I dumped the ioapic writes and saw that RTL was being updated while
the interrupt was unmasked
Logs:

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
     trigger-mode reg/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 10000/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
     trigger-mode reg/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry93e /

     /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
     trigger-mode reg/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry 93e/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e
                             ----->   Update is made while the interrupt
     is unmasked/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: recalculate vlapic
     trigger-mode reg/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

     /[cpu=0][sev=3]:vioapic: ioapic pin14: redir table entry a93e/

Thanks!
Best Regards,
Abdul

Abdul

#14
Edited

During the second update function "acpi_create_platform_device" calls "acpi_dev_get_resources" which reads the acpi resources and updates the trigger mode and polarity of the interrupt.

Best Regards,
Abdul