Hi Dubravko,
1) Enable X2APIC from guest Linux
Actually ACRN checks X2APIC capability and it goes panic w/o it. So the host CPUID must have X2APIC support.
We met one case where the BIOS DMAR setting has some thing like "do not use x2apic", which prevents Linux from enabling X2APIC. Or you may check why the guest doesn't switch to X2APIC from native Linux dmesg?
2) Need Linux logs to help debugging.
Attached is the ACRN built with the following hack, which directs guest serial prints directly to HV console.
Seems Linux goes to the path of numachip_wakeup_secondary() to send SIPIs to secondary CPUs, which we didn't see before.
It's good to see Linux early pintk t help address the issue.
diff --git a/hypervisor/dm/vuart.c b/hypervisor/dm/vuart.c
index 8771a7d8a15f..be5f0da943b8 100644
--- a/hypervisor/dm/vuart.c
+++ b/hypervisor/dm/vuart.c
@@ -295,6 +295,7 @@ static void write_reg(struct acrn_vuart *vu, uint16_t reg, uint8_t value_u8)
                                vu->lsr |= LSR_OE;
                        } else {
                                fifo_putchar(&vu->txfifo, (char)value_u8);
+                               printf("%c", value_u8);
                        }
                        vu->thre_int_pending = true;
                        break;
3) "naked" HV is working?
In the meantime, if you want, you can hack ACRN with the following and it won't launch any VMs and the HV console is supposed to be working. It's just a way to check that ACRN is "generally" working on the HW and you may play with some HV console commands.
diff --git a/hypervisor/arch/x86/guest/vm.c b/hypervisor/arch/x86/guest/vm.c
index 0471324be065..53ccb6b9acae 100644
--- a/hypervisor/arch/x86/guest/vm.c
+++ b/hypervisor/arch/x86/guest/vm.c
@@ -873,6 +873,8 @@ void launch_vms(uint16_t pcpu_id)
        uint16_t vm_id, bsp_id;
        struct acrn_vm_config *vm_config;
+       return;
+
        for (vm_id = 0U; vm_id < CONFIG_MAX_VM_NUM; vm_id++) {
                vm_config = get_vm_config(vm_id);
                if ((vm_config->load_order == SOS_VM) || (vm_config->load_order == PRE_LAUNCHED_VM)) {
Best Regards,
Zide
On 2/12/20 11:05 AM, Zide Chen wrote:

Hi Dubravko,

With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"?

The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS?

Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now.

BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any Linux logs.

 From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage?

Best Regards,
Zide

On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:

Hi Zide,

I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the mentioned setting or similar or related to it.

We are using coreboot, built according to instructions from https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added support for our board (the link apparently no longer works). In general it works very well.

Best regards,
Dubravko


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>




  Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Wednesday, February 12, 2020 6:47 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,

In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference?

Best Regards,
Zide

On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:

Hello,

Please find attached the log file and my acrn.conf.

My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and
I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall
behavior is the same.

New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to
reference Leaf Hill CRB.

After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and
the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).

Best regards,
Dubravko


*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>



   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------

*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Wednesday, February 12, 2020 1:04 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,

"Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be
ignored for now.

For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and
need to debug it.

I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached
bzImage?

Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in
acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console
is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you
may need to apply the attached patch to ACRN hypervisor.

Hopefully the full logs can give us some clues.

Best Regards,
Zide

zide.chen@...