Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hello,
My schedule finally allows me to continue working on getting ACRN to work on our board. I have downloaded the newest version few days ago, added a configuration for our board, used the python/web tool to generate code patches, menuconfig, and built acrn.efi.
Everything went well so far. Unfortunately the hypervisor crashes quite early, but at least I've got debug prints to work.
Here is a part from mmu.c from hypervisor/arch/x86/, debug printouts added by me:
/*
* remove 'NX' bit for pages that contain hv code section, as by default XD bit is set for
* all pages, including pages for guests.
*/
printf("paging 6\n");
mmu_modify_or_del((uint64_t *)ppt_mmu_pml4_addr, round_pde_down(hv_hpa),
round_pde_up(text_end) - round_pde_down(hv_hpa), 0UL,
PAGE_NX, &ppt_mem_ops, MR_MODIFY);
mmu_modify_or_del((uint64_t *)ppt_mmu_pml4_addr, (uint64_t)get_reserve_sworld_memory_base(),
TRUSTY_RAM_SIZE * (CONFIG_MAX_VM_NUM - 1U), PAGE_USER, 0UL, &ppt_mem_ops, MR_MODIFY);
printf("paging 7\n");
/* Enable paging */
enable_paging();
"paging 6" gets printed out, "7" doesn't. That's the last sign of life from ACRN, nothing happens afterwards. What should I do to make it work?
My board has 8 GB of RAM, verified with multiple passes of MemTest to work correctly. CPU is Apollo Lake E3940. Some configuration parameters from menuconfig that might be of relevance:
- LOW_RAM_SIZE = 0x10000 (default value)
- HV_RAM_START = 0x13151000 (default value)
- HV_RAM_SIZE = 0x7800000 (default value)
- PLATFORM_RAM_SIZE = 0x200000000 (8 GB)
- SOS_RAM_SIZE = 0x200000000 (8 GB)
- UOS_RAM_SIZE = 0x100000000 (4 GB)
I suspect, since it seems to crash while adjusting memory parameters, that some of the parameter values above are bad. I think all the initial values were set for some 16 or 32 GB board, so perhaps I didn't adjust everything that I should have? Unfortunately
I'm not familiar with inner workings of x86 CPUs at this low level.
Best regards,
Dubravko Moravski
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
|
|

Chen, Zide
Hi Dubravko,
Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned.
Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue.
In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820() in misc/efi-stub/boot.c, before emalloc_reserved_aligned().
Best Regards,
Zide
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
Thank you very much, you were right. I've had RELOC already enabled, but apparently one also need to have CONFIG_HV_RAM_START 2MB aligned. I've set it to 0x1400 0000 and now my hypervisor manages to execute entire init_paging() function.
... but now it crashes in cpu.c, init_percpu_lapic_id(). A block of code from init_pcpu_pre():
printf("aaaaaaa7\n");
if (!init_percpu_lapic_id()) {
panic("failed to init_percpu_lapic_id!");
}
printf("aaaaaaa8\n");
"7" gets printed out, neither "8" nor panic does.
I guess something is wrong with the interrupts? If it means anything, I've made no particular modifications related to interrupts, and our board generally works well with Linux.
Best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Saturday, February 8, 2020 6:33 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned.
Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue.
In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820()
in misc/efi-stub/boot.c, before emalloc_reserved_aligned().
Best Regards,
Zide
|
|

Chen, Zide
Hi Dubravko, Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others. For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table. Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus. BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages. https://github.com/projectacrn/acrn-hypervisor/pull/4381Best Regards, Zide
toggle quoted message
Show quoted text
On 2/10/20 5:13 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hi Zide, Thank you very much, you were right. I've had RELOC already enabled, but apparently one also need to have CONFIG_HV_RAM_START 2MB aligned. I've set it to 0x1400 0000 and now my hypervisor manages to execute entire init_paging() function. ... but now it crashes in cpu.c, init_percpu_lapic_id(). A block of code from init_pcpu_pre(): printf("aaaaaaa7\n"); if (!init_percpu_lapic_id()) { panic("failed to init_percpu_lapic_id!"); } printf("aaaaaaa8\n"); "7" gets printed out, neither "8" nor panic does. I guess something is wrong with the interrupts? If it means anything, I've made no particular modifications related to interrupts, and our board generally works well with Linux. Best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Saturday, February 8, 2020 6:33 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned. Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue. In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820() in misc/efi-stub/boot.c, before emalloc_reserved_aligned(). Best Regards, Zide
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
MADT and DMAR tables seem to be correct.
I'm using acrn.efi, RELOC is enabled.
Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these:
[836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
[836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
[836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
[836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts.
RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem,
I would expect at least some warning message to be displayed. I'll check that first anyway.
Thank you and best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Monday, February 10, 2020 5:50 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
others.
For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
CPU number from ACPI table.
Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
it finds 0 pcpus.
BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
messages.
https://github.com/projectacrn/acrn-hypervisor/pull/4381
Best Regards,
Zide
|
|

Chen, Zide
Hi Dubravko, If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel. You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6. Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged to do that during bring-up. I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list", "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by "vm_console 0", and switch back to HV console by pressing "Ctrl + Space". https://projectacrn.github.io/latest/tutorials/debug.html?highlight=consoleIf you don't see any VM running, you may following this tutorial (of cause you can try other scenario): https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.htmlThe key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample: misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you installed. Best Regards, Zide
toggle quoted message
Show quoted text
On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hi Zide, You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs. MADT and DMAR tables seem to be correct. I'm using acrn.efi, RELOC is enabled. Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these: [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001 [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001 [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001 [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001 They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts. RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen. I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway. Thank you and best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 5:50 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others. For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table. Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus. BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages. https://github.com/projectacrn/acrn-hypervisor/pull/4381 Best Regards, Zide
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
Your instructions were clear, unfortunately I'm still having issues.
I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
Then I've set clr-boot-manager options according to instructions. This all went smoothly.
After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
[21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50.
[21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50.
[21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50.
[21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50.
[21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.
When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
[34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting VCPU 3
[34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]...
[34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3
[34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]...
[34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3
[34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
... and it never gets to SOS or ACRN console.
As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something.
Best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Monday, February 10, 2020 8:00 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.
You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.
Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged
to do that during bring-up.
I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list",
"vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by
"vm_console 0", and switch back to HV console by pressing "Ctrl + Space".
https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console
If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html
The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample:
misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you
installed.
Best Regards,
Zide
On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi Zide,
>
> You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
>
> MADT and DMAR tables seem to be correct.
> I'm using acrn.efi, RELOC is enabled.
>
> Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number
> of printouts like these:
>
> [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
> [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
> [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
> [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
>
> They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also
> nothing else happens, just these printouts.
>
> RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
>
> I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just
> doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if
> that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
>
> Thank you and best regards,
> Dubravko
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
> w: exorint.com < https://exorint.com/>
>
>
>
>
> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy < https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Monday, February 10, 2020 5:50 PM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
> Hi Dubravko,
>
> Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
> others.
>
> For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
> CPU number from ACPI table.
>
> Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
> it finds 0 pcpus.
>
> BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
> messages.
> https://github.com/projectacrn/acrn-hypervisor/pull/4381
>
> Best Regards,
> Zide
>
>
>
|
|

Chen, Zide
Hi Dubravko,
Is it possible to post the full log here?
Thanks, Zide
toggle quoted message
Show quoted text
On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hi Zide, Your instructions were clear, unfortunately I'm still having issues. I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT now set to 5 (previously was 6). I've verified that I have all the required files on the EFI partition. I've added a new boot entry with: sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0" Then I've set clr-boot-manager options according to instructions. This all went smoothly. After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these: [21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50. [21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50. [21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50. [21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50. [21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50. When I choose the first option that I believe should start the Service OS, it then continuously prints out these: [34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3 [34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]... [34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3 [34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]... [34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3 [34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]... ... and it never gets to SOS or ACRN console. As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something. Best regards, *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 8:00 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel. You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6. Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged to do that during bring-up. I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list", "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by "vm_console 0", and switch back to HV console by pressing "Ctrl + Space". https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console If you don't see any VM running, you may following this tutorial (of cause you can try other scenario): https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample: misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you installed. Best Regards, Zide On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide, You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs. MADT and DMAR tables seem to be correct. I'm using acrn.efi, RELOC is enabled. Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these: [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001 [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001 [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001 [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001 They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts. RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen. I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway. Thank you and best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 5:50 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others. For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table. Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus. BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages. https://github.com/projectacrn/acrn-hypervisor/pull/4381 Best Regards, Zide
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
The full log is below, with some additional /* comments */. I've noticed now that ACRN console prompt does appear, but it is unusable due to all the debug messages.
ACRN Hypervisor
calibrate_tsc, tsc_khz=1593600
[7621776us][cpu=0][(null)][sev=2][seq=1]:HV version 1.6-unstable-2020-02-11 17:29:55-4b983876-dirty DBG (daily tag:acrn-2020w04.1-140000p) build by clear, start time 7596708us
[7639207us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[7644984us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Atom(TM) Processor E3940 @ 1.60GHz
[7655270us][cpu=0][(null)][sev=1][seq=4]:SECURITY WARNING!!!!!!
[7661731us][cpu=0][(null)][sev=1][seq=5]:Please apply the latest CPU uCode patch!
[7670865us][cpu=0][(null)][sev=5][seq=6]:hide pci uart dev: (0:18:0)
[7720135us][cpu=0][(null)][sev=5][seq=9]:profiling_setup: entering
[7720133us][cpu=1][(null)][sev=5][seq=7]:profiling_setup: entering
[7720136us][cpu=3][(null)][sev=5][seq=10]:profiling_setup: entering
[7720134us][cpu=2][(null)][sev=5][seq=8]:profiling_setup: entering
[7726787us][cpu=0][(null)][sev=5][seq=11]:profiling_setup: calling request_irq
[7733543us][cpu=1][(null)][sev=5][seq=12]:profiling_setup: exiting
[7740402us][cpu=3][(null)][sev=5][seq=13]:profiling_setup: exiting
[7747161us][cpu=2][(null)][sev=5][seq=14]:profiling_setup: exiting
[7755094us][cpu=0][(null)][sev=5][seq=15]:profiling_setup: exiting
[7783951us][cpu=0][(null)][sev=5][seq=16]:System S3/S5 is supported.
[7791335us][cpu=0][(null)][sev=5][seq=17]:Creating VCPU working on PCPU0
[7798597us][cpu=0][(null)][sev=5][seq=18]:Create VM0-VCPU0, Role: PRIMARY
[7806135us][cpu=0][(null)][sev=5][seq=19]:Creating VCPU working on PCPU1
[7813389us][cpu=0][(null)][sev=5][seq=20]:Create VM0-VCPU1, Role: SECONDARY
[7821031us][cpu=0][(null)][sev=5][seq=21]:Creating VCPU working on PCPU2
[7828376us][cpu=0][(null)][sev=5][seq=22]:Create VM0-VCPU2, Role: SECONDARY
[7836012us][cpu=0][(null)][sev=5][seq=23]:Creating VCPU working on PCPU3
[7843359us][cpu=0][(null)][sev=5][seq=24]:Create VM0-VCPU3, Role: SECONDARY
[7850998us][cpu=0][(null)][sev=4][seq=25]:Spurious vector: 0x50.
ACRN:\>[7858255us][cpu=0][(null)][sev=2][seq=26]:Start VM id: 0 name: ACRN SOS VM
[7865814us][cpu=0][vm0:vcpu0][sev=5][seq=27]:VM 0 Starting VCPU 0
[7872492us][cpu=0][vm0:vcpu0][sev=5][seq=28]:VM 0 VCPU 0 successfully launched
[7882501us][cpu=0][vm0:vcpu0][sev=4][seq=29]:Spurious vector: 0x50.
[7899292us][cpu=0][vm0:vcpu0][sev=4][seq=30]:Spurious vector: 0x50.
[7916327us][cpu=0][vm0:vcpu0][sev=4][seq=31]:Spurious vector: 0x50.
/* seq 32..99 are all identical entries like the surrounding ones */
[9075356us][cpu=0][vm0:vcpu0][sev=4][seq=100]:Spurious vector: 0x50.
[9092270us][cpu=0][vm0:vcpu0][sev=4][seq=101]:Spurious vector: 0x50.
[9109179us][cpu=0][vm0:vcpu0][sev=4][seq=102]:Spurious vector: 0x50.
[9126089us][cpu=0][vm0:vcpu0][sev=4][seq=103]:Spurious vector: 0x50.
[9163364us][cpu=0][vm0:vcpu0][sev=2][seq=104]:write_cfg 0:18.0 not found! off: 0x3c, val: 0x4
/* approximately here I've choosed an entry from the boot menu */
[9173239us][cpu=0][vm0:vcpu0][sev=3][seq=105]:vlapic: Start Secondary VCPU1 for VM[0]...
[9182086us][cpu=1][vm0:vcpu1][sev=5][seq=106]:VM 0 Starting VCPU 1
[9182310us][cpu=0][vm0:vcpu0][sev=3][seq=107]:vlapic: Start Secondary VCPU2 for VM[0]...
[9197761us][cpu=2][vm0:vcpu2][sev=5][seq=108]:VM 0 Starting VCPU 2
[9197985us][cpu=0][vm0:vcpu0][sev=3][seq=109]:vlapic: Start Secondary VCPU3 for VM[0]...
[9213427us][cpu=3][vm0:vcpu3][sev=5][seq=110]:VM 0 Starting VCPU 3
[9220288us][cpu=0][vm0:vcpu0][sev=3][seq=111]:vlapic: Start Secondary VCPU3 for VM[0]...
[9229124us][cpu=3][vm0:vcpu3][sev=5][seq=112]:VM 0 Starting VCPU 3
[9235992us][cpu=0][vm0:vcpu0][sev=3][seq=113]:vlapic: Start Secondary VCPU3 for VM[0]...
[9244836us][cpu=3][vm0:vcpu3][sev=5][seq=114]:VM 0 Starting VCPU 3
[9251701us][cpu=0][vm0:vcpu0][sev=3][seq=115]:vlapic: Start Secondary VCPU3 for VM[0]...
[9260536us][cpu=3][vm0:vcpu3][sev=5][seq=116]:VM 0 Starting VCPU 3
[9267402us][cpu=0][vm0:vcpu0][sev=3][seq=117]:vlapic: Start Secondary VCPU3 for VM[0]...
[9276237us][cpu=3][vm0:vcpu3][sev=5][seq=118]:VM 0 Starting VCPU 3
[9283102us][cpu=0][vm0:vcpu0][sev=3][seq=119]:vlapic: Start Secondary VCPU3 for VM[0]...
[9291941us][cpu=3][vm0:vcpu3][sev=5][seq=120]:VM 0 Starting VCPU 3
[9298810us][cpu=0][vm0:vcpu0][sev=3][seq=121]:vlapic: Start Secondary VCPU3 for VM[0]...
[9307648us][cpu=3][vm0:vcpu3][sev=5][seq=122]:VM 0 Starting VCPU 3
[9314511us][cpu=0][vm0:vcpu0][sev=3][seq=123]:vlapic: Start Secondary VCPU3 for VM[0]...
[9323355us][cpu=3][vm0:vcpu3][sev=5][seq=124]:VM 0 Starting VCPU 3
[9330216us][cpu=0][vm0:vcpu0][sev=3][seq=125]:vlapic: Start Secondary VCPU3 for VM[0]...
[9339041us][cpu=3][vm0:vcpu3][sev=5][seq=126]:VM 0 Starting VCPU 3
/* this continues indefinitely... */
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Tuesday, February 11, 2020 5:31 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Is it possible to post the full log here?
Thanks,
Zide
On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi Zide,
>
> Your instructions were clear, unfortunately I'm still having issues.
>
> I have followed the instructions from
https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded
> Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT
> now set to 5 (previously was 6).
>
> I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
>
> sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u
> "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
>
> Then I've set clr-boot-manager options according to instructions. This all went smoothly.
>
> After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
>
> [21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50.
> [21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50.
> [21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50.
> [21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50.
> [21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.
>
>
> When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
>
> [34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3
> [34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]...
> [34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3
> [34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]...
> [34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3
> [34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
>
> ... and it never gets to SOS or ACRN console.
>
> As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux
> shows that all 4 cores are alive and occasionally doing something.
>
> Best regards,
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
> w: exorint.com < https://exorint.com/>
>
>
>
>
> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy < https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Monday, February 10, 2020 8:00 PM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
> Hi Dubravko,
>
> If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.
>
> You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.
>
> Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged
> to do that during bring-up.
>
> I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list",
> "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by
> "vm_console 0", and switch back to HV console by pressing "Ctrl + Space".
>
>
https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console
>
> If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
> https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html
>
> The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample:
> misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you
> installed.
>
>
> Best Regards,
> Zide
>
>
> On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hi Zide,
>>
>> You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
>>
>> MADT and DMAR tables seem to be correct.
>> I'm using acrn.efi, RELOC is enabled.
>>
>> Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number
>> of printouts like these:
>>
>> [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
>> [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
>> [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
>> [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
>>
>> They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also
>> nothing else happens, just these printouts.
>>
>> RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
>>
>> I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just
>> doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if
>> that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
>>
>> Thank you and best regards,
>> Dubravko
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
>> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w: exorint.com < https://exorint.com/>
>>
>>
>>
>>
>> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>
>> Privacy < https://www.exorint.com/it/privacy>
>> --------------------------------------------------------------------------------------------------------------------------
>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>> *Sent:* Monday, February 10, 2020 5:50 PM
>> *To:* acrn-users@... <acrn-users@...>
>> *Subject:* Re: [acrn-users] Getting ACRN to work
>> Hi Dubravko,
>>
>> Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
>> others.
>>
>> For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
>> CPU number from ACPI table.
>>
>> Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
>> it finds 0 pcpus.
>>
>> BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
>> messages.
>> https://github.com/projectacrn/acrn-hypervisor/pull/4381
>>
>> Best Regards,
>> Zide
>>
>>
>>
>
>
>
>
|
|

Chen, Zide
Hi Dubravko,
"Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now.
For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it.
I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage?
Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor.
Hopefully the full logs can give us some clues.
Best Regards, Zide
zide.chen@...
toggle quoted message
Show quoted text
On 2/11/20 9:46 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hi Zide, The full log is below, with some additional /* comments */. I've noticed now that ACRN console prompt does appear, but it is unusable due to all the debug messages. ACRN Hypervisor calibrate_tsc, tsc_khz=1593600 [7621776us][cpu=0][(null)][sev=2][seq=1]:HV version 1.6-unstable-2020-02-11 17:29:55-4b983876-dirty DBG (daily tag:acrn-2020w04.1-140000p) build by clear, start time 7596708us [7639207us][cpu=0][(null)][sev=2][seq=2]:API version 1.0 [7644984us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Atom(TM) Processor E3940 @ 1.60GHz [7655270us][cpu=0][(null)][sev=1][seq=4]:SECURITY WARNING!!!!!! [7661731us][cpu=0][(null)][sev=1][seq=5]:Please apply the latest CPU uCode patch! [7670865us][cpu=0][(null)][sev=5][seq=6]:hide pci uart dev: (0:18:0) [7720135us][cpu=0][(null)][sev=5][seq=9]:profiling_setup: entering [7720133us][cpu=1][(null)][sev=5][seq=7]:profiling_setup: entering [7720136us][cpu=3][(null)][sev=5][seq=10]:profiling_setup: entering [7720134us][cpu=2][(null)][sev=5][seq=8]:profiling_setup: entering [7726787us][cpu=0][(null)][sev=5][seq=11]:profiling_setup: calling request_irq [7733543us][cpu=1][(null)][sev=5][seq=12]:profiling_setup: exiting [7740402us][cpu=3][(null)][sev=5][seq=13]:profiling_setup: exiting [7747161us][cpu=2][(null)][sev=5][seq=14]:profiling_setup: exiting [7755094us][cpu=0][(null)][sev=5][seq=15]:profiling_setup: exiting [7783951us][cpu=0][(null)][sev=5][seq=16]:System S3/S5 is supported. [7791335us][cpu=0][(null)][sev=5][seq=17]:Creating VCPU working on PCPU0 [7798597us][cpu=0][(null)][sev=5][seq=18]:Create VM0-VCPU0, Role: PRIMARY [7806135us][cpu=0][(null)][sev=5][seq=19]:Creating VCPU working on PCPU1 [7813389us][cpu=0][(null)][sev=5][seq=20]:Create VM0-VCPU1, Role: SECONDARY [7821031us][cpu=0][(null)][sev=5][seq=21]:Creating VCPU working on PCPU2 [7828376us][cpu=0][(null)][sev=5][seq=22]:Create VM0-VCPU2, Role: SECONDARY [7836012us][cpu=0][(null)][sev=5][seq=23]:Creating VCPU working on PCPU3 [7843359us][cpu=0][(null)][sev=5][seq=24]:Create VM0-VCPU3, Role: SECONDARY [7850998us][cpu=0][(null)][sev=4][seq=25]:Spurious vector: 0x50. ACRN:\>[7858255us][cpu=0][(null)][sev=2][seq=26]:Start VM id: 0 name: ACRN SOS VM [7865814us][cpu=0][vm0:vcpu0][sev=5][seq=27]:VM 0 Starting VCPU 0 [7872492us][cpu=0][vm0:vcpu0][sev=5][seq=28]:VM 0 VCPU 0 successfully launched [7882501us][cpu=0][vm0:vcpu0][sev=4][seq=29]:Spurious vector: 0x50. [7899292us][cpu=0][vm0:vcpu0][sev=4][seq=30]:Spurious vector: 0x50. [7916327us][cpu=0][vm0:vcpu0][sev=4][seq=31]:Spurious vector: 0x50. /* seq 32..99 are all identical entries like the surrounding ones */ [9075356us][cpu=0][vm0:vcpu0][sev=4][seq=100]:Spurious vector: 0x50. [9092270us][cpu=0][vm0:vcpu0][sev=4][seq=101]:Spurious vector: 0x50. [9109179us][cpu=0][vm0:vcpu0][sev=4][seq=102]:Spurious vector: 0x50. [9126089us][cpu=0][vm0:vcpu0][sev=4][seq=103]:Spurious vector: 0x50. [9163364us][cpu=0][vm0:vcpu0][sev=2][seq=104]:write_cfg 0:18.0 not found! off: 0x3c, val: 0x4 /* approximately here I've choosed an entry from the boot menu */ [9173239us][cpu=0][vm0:vcpu0][sev=3][seq=105]:vlapic: Start Secondary VCPU1 for VM[0]... [9182086us][cpu=1][vm0:vcpu1][sev=5][seq=106]:VM 0 Starting VCPU 1 [9182310us][cpu=0][vm0:vcpu0][sev=3][seq=107]:vlapic: Start Secondary VCPU2 for VM[0]... [9197761us][cpu=2][vm0:vcpu2][sev=5][seq=108]:VM 0 Starting VCPU 2 [9197985us][cpu=0][vm0:vcpu0][sev=3][seq=109]:vlapic: Start Secondary VCPU3 for VM[0]... [9213427us][cpu=3][vm0:vcpu3][sev=5][seq=110]:VM 0 Starting VCPU 3 [9220288us][cpu=0][vm0:vcpu0][sev=3][seq=111]:vlapic: Start Secondary VCPU3 for VM[0]... [9229124us][cpu=3][vm0:vcpu3][sev=5][seq=112]:VM 0 Starting VCPU 3 [9235992us][cpu=0][vm0:vcpu0][sev=3][seq=113]:vlapic: Start Secondary VCPU3 for VM[0]... [9244836us][cpu=3][vm0:vcpu3][sev=5][seq=114]:VM 0 Starting VCPU 3 [9251701us][cpu=0][vm0:vcpu0][sev=3][seq=115]:vlapic: Start Secondary VCPU3 for VM[0]... [9260536us][cpu=3][vm0:vcpu3][sev=5][seq=116]:VM 0 Starting VCPU 3 [9267402us][cpu=0][vm0:vcpu0][sev=3][seq=117]:vlapic: Start Secondary VCPU3 for VM[0]... [9276237us][cpu=3][vm0:vcpu3][sev=5][seq=118]:VM 0 Starting VCPU 3 [9283102us][cpu=0][vm0:vcpu0][sev=3][seq=119]:vlapic: Start Secondary VCPU3 for VM[0]... [9291941us][cpu=3][vm0:vcpu3][sev=5][seq=120]:VM 0 Starting VCPU 3 [9298810us][cpu=0][vm0:vcpu0][sev=3][seq=121]:vlapic: Start Secondary VCPU3 for VM[0]... [9307648us][cpu=3][vm0:vcpu3][sev=5][seq=122]:VM 0 Starting VCPU 3 [9314511us][cpu=0][vm0:vcpu0][sev=3][seq=123]:vlapic: Start Secondary VCPU3 for VM[0]... [9323355us][cpu=3][vm0:vcpu3][sev=5][seq=124]:VM 0 Starting VCPU 3 [9330216us][cpu=0][vm0:vcpu0][sev=3][seq=125]:vlapic: Start Secondary VCPU3 for VM[0]... [9339041us][cpu=3][vm0:vcpu3][sev=5][seq=126]:VM 0 Starting VCPU 3 /* this continues indefinitely... */ *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Tuesday, February 11, 2020 5:31 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, Is it possible to post the full log here? Thanks, Zide On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide, Your instructions were clear, unfortunately I'm still having issues. I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT now set to 5 (previously was 6). I've verified that I have all the required files on the EFI partition. I've added a new boot entry with: sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0" Then I've set clr-boot-manager options according to instructions. This all went smoothly. After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these: [21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50. [21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50. [21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50. [21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50. [21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50. When I choose the first option that I believe should start the Service OS, it then continuously prints out these: [34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3 [34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]... [34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3 [34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]... [34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3 [34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]... ... and it never gets to SOS or ACRN console. As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something. Best regards, *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 8:00 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel. You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6. Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged to do that during bring-up. I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list", "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by "vm_console 0", and switch back to HV console by pressing "Ctrl + Space". https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console If you don't see any VM running, you may following this tutorial (of cause you can try other scenario): https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample: misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you installed. Best Regards, Zide On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide, You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs. MADT and DMAR tables seem to be correct. I'm using acrn.efi, RELOC is enabled. Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these: [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001 [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001 [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001 [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001 They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts. RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen. I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway. Thank you and best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 5:50 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others. For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table. Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus. BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages. https://github.com/projectacrn/acrn-hypervisor/pull/4381 Best Regards, Zide
|
|
Yin, Fengwei <fengwei.yin@...>
On 2/12/2020 8:04 AM, Chen, Zide wrote: Hi Dubravko, "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now. For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it. I highly suspect this endless "Starting VCPU" thing is not from SOS kernel. It's more like from BIOS. It could be related with some BIOS setting. I suggest checking the BIOS setting and disable mwait for AP offline and see any difference. Regards Yin, Fengwei I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage? Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor. Hopefully the full logs can give us some clues. Best Regards, Zide zide.chen@... On 2/11/20 9:46 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
The full log is below, with some additional /* comments */. I've noticed now that ACRN console prompt does appear, but it is unusable due to all the debug messages.
ACRN Hypervisor calibrate_tsc, tsc_khz=1593600 [7621776us][cpu=0][(null)][sev=2][seq=1]:HV version 1.6-unstable-2020-02-11 17:29:55-4b983876-dirty DBG (daily tag:acrn-2020w04.1-140000p) build by clear, start time 7596708us [7639207us][cpu=0][(null)][sev=2][seq=2]:API version 1.0 [7644984us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Atom(TM) Processor E3940 @ 1.60GHz [7655270us][cpu=0][(null)][sev=1][seq=4]:SECURITY WARNING!!!!!! [7661731us][cpu=0][(null)][sev=1][seq=5]:Please apply the latest CPU uCode patch! [7670865us][cpu=0][(null)][sev=5][seq=6]:hide pci uart dev: (0:18:0) [7720135us][cpu=0][(null)][sev=5][seq=9]:profiling_setup: entering [7720133us][cpu=1][(null)][sev=5][seq=7]:profiling_setup: entering [7720136us][cpu=3][(null)][sev=5][seq=10]:profiling_setup: entering [7720134us][cpu=2][(null)][sev=5][seq=8]:profiling_setup: entering [7726787us][cpu=0][(null)][sev=5][seq=11]:profiling_setup: calling request_irq [7733543us][cpu=1][(null)][sev=5][seq=12]:profiling_setup: exiting [7740402us][cpu=3][(null)][sev=5][seq=13]:profiling_setup: exiting [7747161us][cpu=2][(null)][sev=5][seq=14]:profiling_setup: exiting [7755094us][cpu=0][(null)][sev=5][seq=15]:profiling_setup: exiting [7783951us][cpu=0][(null)][sev=5][seq=16]:System S3/S5 is supported. [7791335us][cpu=0][(null)][sev=5][seq=17]:Creating VCPU working on PCPU0 [7798597us][cpu=0][(null)][sev=5][seq=18]:Create VM0-VCPU0, Role: PRIMARY [7806135us][cpu=0][(null)][sev=5][seq=19]:Creating VCPU working on PCPU1 [7813389us][cpu=0][(null)][sev=5][seq=20]:Create VM0-VCPU1, Role: SECONDARY [7821031us][cpu=0][(null)][sev=5][seq=21]:Creating VCPU working on PCPU2 [7828376us][cpu=0][(null)][sev=5][seq=22]:Create VM0-VCPU2, Role: SECONDARY [7836012us][cpu=0][(null)][sev=5][seq=23]:Creating VCPU working on PCPU3 [7843359us][cpu=0][(null)][sev=5][seq=24]:Create VM0-VCPU3, Role: SECONDARY [7850998us][cpu=0][(null)][sev=4][seq=25]:Spurious vector: 0x50. ACRN:\>[7858255us][cpu=0][(null)][sev=2][seq=26]:Start VM id: 0 name: ACRN SOS VM [7865814us][cpu=0][vm0:vcpu0][sev=5][seq=27]:VM 0 Starting VCPU 0 [7872492us][cpu=0][vm0:vcpu0][sev=5][seq=28]:VM 0 VCPU 0 successfully launched [7882501us][cpu=0][vm0:vcpu0][sev=4][seq=29]:Spurious vector: 0x50. [7899292us][cpu=0][vm0:vcpu0][sev=4][seq=30]:Spurious vector: 0x50. [7916327us][cpu=0][vm0:vcpu0][sev=4][seq=31]:Spurious vector: 0x50. /* seq 32..99 are all identical entries like the surrounding ones */ [9075356us][cpu=0][vm0:vcpu0][sev=4][seq=100]:Spurious vector: 0x50. [9092270us][cpu=0][vm0:vcpu0][sev=4][seq=101]:Spurious vector: 0x50. [9109179us][cpu=0][vm0:vcpu0][sev=4][seq=102]:Spurious vector: 0x50. [9126089us][cpu=0][vm0:vcpu0][sev=4][seq=103]:Spurious vector: 0x50. [9163364us][cpu=0][vm0:vcpu0][sev=2][seq=104]:write_cfg 0:18.0 not found! off: 0x3c, val: 0x4 /* approximately here I've choosed an entry from the boot menu */
[9173239us][cpu=0][vm0:vcpu0][sev=3][seq=105]:vlapic: Start Secondary VCPU1 for VM[0]... [9182086us][cpu=1][vm0:vcpu1][sev=5][seq=106]:VM 0 Starting VCPU 1 [9182310us][cpu=0][vm0:vcpu0][sev=3][seq=107]:vlapic: Start Secondary VCPU2 for VM[0]... [9197761us][cpu=2][vm0:vcpu2][sev=5][seq=108]:VM 0 Starting VCPU 2 [9197985us][cpu=0][vm0:vcpu0][sev=3][seq=109]:vlapic: Start Secondary VCPU3 for VM[0]... [9213427us][cpu=3][vm0:vcpu3][sev=5][seq=110]:VM 0 Starting VCPU 3 [9220288us][cpu=0][vm0:vcpu0][sev=3][seq=111]:vlapic: Start Secondary VCPU3 for VM[0]... [9229124us][cpu=3][vm0:vcpu3][sev=5][seq=112]:VM 0 Starting VCPU 3 [9235992us][cpu=0][vm0:vcpu0][sev=3][seq=113]:vlapic: Start Secondary VCPU3 for VM[0]... [9244836us][cpu=3][vm0:vcpu3][sev=5][seq=114]:VM 0 Starting VCPU 3 [9251701us][cpu=0][vm0:vcpu0][sev=3][seq=115]:vlapic: Start Secondary VCPU3 for VM[0]... [9260536us][cpu=3][vm0:vcpu3][sev=5][seq=116]:VM 0 Starting VCPU 3 [9267402us][cpu=0][vm0:vcpu0][sev=3][seq=117]:vlapic: Start Secondary VCPU3 for VM[0]... [9276237us][cpu=3][vm0:vcpu3][sev=5][seq=118]:VM 0 Starting VCPU 3 [9283102us][cpu=0][vm0:vcpu0][sev=3][seq=119]:vlapic: Start Secondary VCPU3 for VM[0]... [9291941us][cpu=3][vm0:vcpu3][sev=5][seq=120]:VM 0 Starting VCPU 3 [9298810us][cpu=0][vm0:vcpu0][sev=3][seq=121]:vlapic: Start Secondary VCPU3 for VM[0]... [9307648us][cpu=3][vm0:vcpu3][sev=5][seq=122]:VM 0 Starting VCPU 3 [9314511us][cpu=0][vm0:vcpu0][sev=3][seq=123]:vlapic: Start Secondary VCPU3 for VM[0]... [9323355us][cpu=3][vm0:vcpu3][sev=5][seq=124]:VM 0 Starting VCPU 3 [9330216us][cpu=0][vm0:vcpu0][sev=3][seq=125]:vlapic: Start Secondary VCPU3 for VM[0]... [9339041us][cpu=3][vm0:vcpu3][sev=5][seq=126]:VM 0 Starting VCPU 3 /* this continues indefinitely... */
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> --------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Tuesday, February 11, 2020 5:31 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
Is it possible to post the full log here?
Thanks, Zide
On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
Your instructions were clear, unfortunately I'm still having issues.
I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT now set to 5 (previously was 6).
I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
Then I've set clr-boot-manager options according to instructions. This all went smoothly.
After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
[21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50. [21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50. [21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50. [21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50. [21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.
When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
[34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3 [34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]... [34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3 [34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]... [34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3 [34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
... and it never gets to SOS or ACRN console.
As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something.
Best regards,
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> --------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 8:00 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.
You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.
Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged to do that during bring-up.
I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list", "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by "vm_console 0", and switch back to HV console by pressing "Ctrl + Space".
https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console
If you don't see any VM running, you may following this tutorial (of cause you can try other scenario): https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html
The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample: misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you installed.
Best Regards, Zide
On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
MADT and DMAR tables seem to be correct. I'm using acrn.efi, RELOC is enabled.
Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these:
[836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001 [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001 [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001 [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts.
RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
Thank you and best regards, Dubravko
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> --------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Monday, February 10, 2020 5:50 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others.
For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table.
Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus.
BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages. https://github.com/projectacrn/acrn-hypervisor/pull/4381
Best Regards, Zide
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hello,
Please find attached the log file and my acrn.conf.
My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same.
New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB.
After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
Best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Wednesday, February 12, 2020 1:04 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
"Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be
ignored for now.
For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and
need to debug it.
I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached
bzImage?
Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in
acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console
is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you
may need to apply the attached patch to ACRN hypervisor.
Hopefully the full logs can give us some clues.
Best Regards,
Zide
zide.chen@...
|
|

Chen, Zide
Not sure why there is no kernel logs come out from serial console. I tried the acrn.efi/bzImage I sent to you, along with the bootargs in your acrn.conf, I was able to see that Linux logs started right after VM0 is launched:
[6931479us][cpu=0][(null)][sev=2][seq=25]:Start VM id: 0 name: ACRN SOS VM [6938866us][cpu=0][vm0:vcpu0][sev=5][seq=26]:VM 0 Starting VCPU 0 [6945396us][cpu=0][vm0:vcpu0][sev=5][seq=27]:VM 0 VCPU 0 successfully launched early console in extract_kernel input_data: 0x00000000025403b1 input_len: 0x0000000000c2416d output: 0x0000000001000000 output_len: 0x0000000002132574 kernel_total_size: 0x0000000001e2c000 trampoline_32bit: 0x000000000008c000 booted via startup_64() Physical KASLR using RDRAND RDTSC... Virtual KASLR using RDRAND RDTSC...
Decompressing Linux... og=2M@0x1FE00000 i915.avail_planes_per_pipe=0x01010F i915.domain_plane_owners=0x011111110000 i915.enable_guc=0 i915.enable_gvt=1 i915.nuclear_pageflip=1 ignore_loglevel init=/usr/lib/systemd/systemd-bootchart intel_iommu=igfx_off memmap=2M$0x1FE00000 no_timer_check no_timer_check noreplace-smp rcu_nocbs=0-64 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs rootwait tsc=reliable rw ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep [ 0.000000] ------------[ cut here ]------------ [ 0.000000] XSAVE consistency problem, dumping leaves [ 0.000000] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/fpu/xstate.c:614 fpu__init_system_xstate+0x464/0x868 [ 0.000000] Modules linked in: ...
[14737156us][cpu=0][vm0:vcpu0][sev=3][seq=30]:vlapic: Start Secondary VCPU1 for VM[0]... [14745830us][cpu=1][vm0:vcpu1][sev=5][seq=31]:VM 0 Starting VCPU 1 [14761070us][cpu=0][vm0:vcpu0][sev=3][seq=32]:vlapic: Start Secondary VCPU2 for VM[0]...
Best Regards, Zide
toggle quoted message
Show quoted text
On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hello, Please find attached the log file and my acrn.conf. My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same. New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB. After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything). Best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 1:04 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now. For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it. I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage? Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor. Hopefully the full logs can give us some clues. Best Regards, Zide zide.chen@...
|
|

Chen, Zide
Hi Dubravko,
In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference?
Best Regards, Zide
toggle quoted message
Show quoted text
On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hello, Please find attached the log file and my acrn.conf. My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same. New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB. After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything). Best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 1:04 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now. For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it. I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage? Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor. Hopefully the full logs can give us some clues. Best Regards, Zide zide.chen@...
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the mentioned setting or similar or related to it.
Best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Wednesday, February 12, 2020 6:47 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference?
Best Regards,
Zide
On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hello,
>
> Please find attached the log file and my acrn.conf.
>
> My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and
> I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall
> behavior is the same.
>
> New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to
> reference Leaf Hill CRB.
>
> After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and
> the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
>
> Best regards,
> Dubravko
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
> w: exorint.com < https://exorint.com/>
>
>
>
>
> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy < https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Wednesday, February 12, 2020 1:04 AM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
> Hi Dubravko,
>
> "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be
> ignored for now.
>
> For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and
> need to debug it.
>
> I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached
> bzImage?
>
> Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in
> acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console
> is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you
> may need to apply the attached patch to ACRN hypervisor.
>
> Hopefully the full logs can give us some clues.
>
> Best Regards,
> Zide
>
> zide.chen@...
>
>
|
|

Chen, Zide
Hi Dubravko,
With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"?
The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS?
Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now.
BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any Linux logs.
From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage?
Best Regards, Zide
toggle quoted message
Show quoted text
On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote: Hi Zide, I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the mentioned setting or similar or related to it. We are using coreboot, built according to instructions from https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added support for our board (the link apparently no longer works). In general it works very well. Best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 6:47 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference? Best Regards, Zide On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello, Please find attached the log file and my acrn.conf. My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same. New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB. After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything). Best regards, Dubravko *Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email. Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 1:04 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko, "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now. For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it. I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage? Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor. Hopefully the full logs can give us some clues. Best Regards, Zide zide.chen@...
|
|

Chen, Zide
Hi Dubravko,
1) Enable X2APIC from guest Linux
Actually ACRN checks X2APIC capability and it goes panic w/o it. So the host CPUID must have X2APIC support.
We met one case where the BIOS DMAR setting has some thing like "do not use x2apic", which prevents Linux from enabling X2APIC. Or you may check why the guest doesn't switch to X2APIC from native Linux dmesg?
2) Need Linux logs to help debugging.
Attached is the ACRN built with the following hack, which directs guest serial prints directly to HV console. Seems Linux goes to the path of numachip_wakeup_secondary() to send SIPIs to secondary CPUs, which we didn't see before. It's good to see Linux early pintk t help address the issue.
diff --git a/hypervisor/dm/vuart.c b/hypervisor/dm/vuart.c index 8771a7d8a15f..be5f0da943b8 100644 --- a/hypervisor/dm/vuart.c +++ b/hypervisor/dm/vuart.c @@ -295,6 +295,7 @@ static void write_reg(struct acrn_vuart *vu, uint16_t reg, uint8_t value_u8) vu->lsr |= LSR_OE; } else { fifo_putchar(&vu->txfifo, (char)value_u8); + printf("%c", value_u8); } vu->thre_int_pending = true; break;
3) "naked" HV is working?
In the meantime, if you want, you can hack ACRN with the following and it won't launch any VMs and the HV console is supposed to be working. It's just a way to check that ACRN is "generally" working on the HW and you may play with some HV console commands.
diff --git a/hypervisor/arch/x86/guest/vm.c b/hypervisor/arch/x86/guest/vm.c index 0471324be065..53ccb6b9acae 100644 --- a/hypervisor/arch/x86/guest/vm.c +++ b/hypervisor/arch/x86/guest/vm.c @@ -873,6 +873,8 @@ void launch_vms(uint16_t pcpu_id) uint16_t vm_id, bsp_id; struct acrn_vm_config *vm_config;
+ return; + for (vm_id = 0U; vm_id < CONFIG_MAX_VM_NUM; vm_id++) { vm_config = get_vm_config(vm_id); if ((vm_config->load_order == SOS_VM) || (vm_config->load_order == PRE_LAUNCHED_VM)) {
Best Regards, Zide
toggle quoted message
Show quoted text
On 2/12/20 11:05 AM, Zide Chen wrote: Hi Dubravko, With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"? The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS? Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now. BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any Linux logs. From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage? Best Regards, Zide On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the mentioned setting or similar or related to it.
We are using coreboot, built according to instructions from https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added support for our board (the link apparently no longer works). In general it works very well.
Best regards, Dubravko
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 6:47 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference?
Best Regards, Zide
On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello,
Please find attached the log file and my acrn.conf.
My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same.
New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB.
After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
Best regards, Dubravko
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 1:04 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
"Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now.
For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it.
I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage?
Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor.
Hopefully the full logs can give us some clues.
Best Regards, Zide
zide.chen@...
|
|

Chen, Zide
One more thing you may try: add maxcpus=1 to acrn.conf options, so we can skip the SIPI issue and see how it goes.
toggle quoted message
Show quoted text
On 2/12/20 12:04 PM, Chen, Zide wrote: Hi Dubravko, 1) Enable X2APIC from guest Linux Actually ACRN checks X2APIC capability and it goes panic w/o it. So the host CPUID must have X2APIC support. We met one case where the BIOS DMAR setting has some thing like "do not use x2apic", which prevents Linux from enabling X2APIC. Or you may check why the guest doesn't switch to X2APIC from native Linux dmesg? 2) Need Linux logs to help debugging. Attached is the ACRN built with the following hack, which directs guest serial prints directly to HV console. Seems Linux goes to the path of numachip_wakeup_secondary() to send SIPIs to secondary CPUs, which we didn't see before. It's good to see Linux early pintk t help address the issue. diff --git a/hypervisor/dm/vuart.c b/hypervisor/dm/vuart.c index 8771a7d8a15f..be5f0da943b8 100644 --- a/hypervisor/dm/vuart.c +++ b/hypervisor/dm/vuart.c @@ -295,6 +295,7 @@ static void write_reg(struct acrn_vuart *vu, uint16_t reg, uint8_t value_u8) vu->lsr |= LSR_OE; } else { fifo_putchar(&vu->txfifo, (char)value_u8); + printf("%c", value_u8); } vu->thre_int_pending = true; break; 3) "naked" HV is working? In the meantime, if you want, you can hack ACRN with the following and it won't launch any VMs and the HV console is supposed to be working. It's just a way to check that ACRN is "generally" working on the HW and you may play with some HV console commands. diff --git a/hypervisor/arch/x86/guest/vm.c b/hypervisor/arch/x86/guest/vm.c index 0471324be065..53ccb6b9acae 100644 --- a/hypervisor/arch/x86/guest/vm.c +++ b/hypervisor/arch/x86/guest/vm.c @@ -873,6 +873,8 @@ void launch_vms(uint16_t pcpu_id) uint16_t vm_id, bsp_id; struct acrn_vm_config *vm_config; + return; + for (vm_id = 0U; vm_id < CONFIG_MAX_VM_NUM; vm_id++) { vm_config = get_vm_config(vm_id); if ((vm_config->load_order == SOS_VM) || (vm_config->load_order == PRE_LAUNCHED_VM)) { Best Regards, Zide On 2/12/20 11:05 AM, Zide Chen wrote:
Hi Dubravko,
With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"?
The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS?
Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now.
BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any Linux logs.
From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage?
Best Regards, Zide
On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the mentioned setting or similar or related to it.
We are using coreboot, built according to instructions from https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added support for our board (the link apparently no longer works). In general it works very well.
Best regards, Dubravko
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> -------------------------------------------------------------------------------------------------------------------------- *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 6:47 PM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any difference?
Best Regards, Zide
On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hello,
Please find attached the log file and my acrn.conf.
My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall behavior is the same.
New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to reference Leaf Hill CRB.
After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
Best regards, Dubravko
*Dubravko Moravski* /SW engineering/ *Exor Embedded S.r.l.* p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413> a: Slavonska avenija, 50, Zagreb, Croatia, 10000 w: exorint.com <https://exorint.com/>
Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy> --------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...> *Sent:* Wednesday, February 12, 2020 1:04 AM *To:* acrn-users@... <acrn-users@...> *Subject:* Re: [acrn-users] Getting ACRN to work Hi Dubravko,
"Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be ignored for now.
For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and need to debug it.
I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached bzImage?
Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you may need to apply the attached patch to ACRN hypervisor.
Hopefully the full logs can give us some clues.
Best Regards, Zide
zide.chen@...
|
|
Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
Hi Zide,
1) X2APIC
I see that it's not mentioned anywhere in ACRN log (and I suppose it should be), but it seems to be enabled somehow in non-hypervised Linux:
clear@clr-bcdcb0fadf6546d7b0984fe80030eee8~ $ sudo dmesg | grep -i x2apic
[ 0.165205] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.167422] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.167426] x2apic enabled
[ 0.167448] Switched APIC routing to cluster x2apic.
clear@clr-bcdcb0fadf6546d7b0984fe80030eee8~ $
Our BIOS does not have a GUI setting to manually enable it, and we couldn't find where to explicitly enable it in source code. It's mentioned at many places; though not in DMAR table.
Can we conclude from the log above that it's ok, or we still need to find a way to enable it in ACRN or BIOS?
2) Linux logs
I'm working on this and will report later.
By the way, I think my last logs might be inconsistent with earlier ones (with endless SIPI messages) because earlier I might have in a hurry just pressed enter in the boot menu to choose whichever option was selected, instead of selecting ACRN. Now I'm more
careful.
Removing 'quiet' parameter didn't help.
3) Naked HV
... is working. With your previous acrn.efi and bzImage from around Wednesday noon Central European Time, after serial boot log stops, I can indeed switch to ACRN console with ctrl+space. If I switch back to SOS, nothing happens. I can only return back to ACRN
console. Here are some printouts:
[13133696us][cpu=0][vm0:vcpu0][sev=3][seq=322]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13146659us][cpu=0][vm0:vcpu0][sev=3][seq=323]:vcpu1 paused, new state: 4
[13154097us][cpu=0][vm0:vcpu0][sev=3][seq=324]:vcpu1 reset
[13160078us][cpu=0][vm0:vcpu0][sev=3][seq=325]:vlapic: vlapic_reset 1693 vm0 vcpu1 initial lapic_id: 2000000
[13170960us][cpu=0][vm0:vcpu0][sev=3][seq=326]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13183985us][cpu=0][vm0:vcpu0][sev=3][seq=327]:vcpu2 paused, new state: 4
[13191429us][cpu=0][vm0:vcpu0][sev=3][seq=328]:vcpu2 reset
[13197408us][cpu=0][vm0:vcpu0][sev=3][seq=329]:vlapic: vlapic_reset 1693 vm0 vcpu2 initial lapic_id: 4000000
[13208295us][cpu=0][vm0:vcpu0][sev=3][seq=330]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13221324us][cpu=0][vm0:vcpu0][sev=3][seq=331]:vcpu3 paused, new state: 4
[13228769us][cpu=0][vm0:vcpu0][sev=3][seq=332]:vcpu3 reset
[13234748us][cpu=0][vm0:vcpu0][sev=3][seq=333]:vlapic: vlapic_reset 1693 vm0 vcpu3 initial lapic_id: 6000000
[13255651us][cpu=0][vm0:vcpu0][sev=3][seq=334]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13268597us][cpu=0][vm0:vcpu0][sev=3][seq=335]:vlapic: Start Secondary VCPU1 for VM[0]...
[13277605us][cpu=0][vm0:vcpu0][sev=3][seq=336]:vcpu1 scheduled on pcpu1 vcpu->state 1
[13286228us][cpu=0][vm0:vcpu0][sev=3][seq=337]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13299256us][cpu=0][vm0:vcpu0][sev=3][seq=338]:vlapic: Start Secondary VCPU2 for VM[0]...
[13308274us][cpu=0][vm0:vcpu0][sev=3][seq=339]:vcpu2 scheduled on pcpu2 vcpu->state 1
[13316902us][cpu=0][vm0:vcpu0][sev=3][seq=340]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13329919us][cpu=0][vm0:vcpu0][sev=3][seq=341]:vlapic: Start Secondary VCPU3 for VM[0]...
[13338926us][cpu=0][vm0:vcpu0][sev=3][seq=342]:vcpu3 scheduled on pcpu3 vcpu->state 1
[13347768us][cpu=0][vm0:vcpu0][sev=3][seq=343]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
[13360707us][cpu=0][vm0:vcpu0][sev=3][seq=344]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
[13373751us][cpu=0][vm0:vcpu0][sev=3][seq=345]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
/* ctrl+space */
---Entering ACRN SHELL---
ACRN:\>
ACRN:\>version
HV version 1.6-unstable-2020-02-11 15:28:54-4990d8574159 DBG (daily tag: acrn-2020w03.4-140000p) build by zide
API version 1.0
ACRN:\>vm_list
VM_UUID VM_ID VM_NAME VM_STATE
================================ ===== ================================ ========
dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Started
ACRN:\>vcpu_list
VM ID PCPU ID VCPU ID VCPU ROLE VCPU STATE THREAD STATE
===== ======= ======= ========= ========== ==========
0 0 0 PRIMARY Running RUNNING
0 1 1 SECONDARY Running BLOCKED
0 2 2 SECONDARY Running BLOCKED
0 3 3 SECONDARY Running BLOCKED
ACRN:\>vioapic
Error: Invalid parameters.
ACRN:\>vioapic 0
PIN VEC DM DEST TM DELM IRR MASK
0 0x0 phys 0x0 edge 0 0 1
1 0x0 phys 0x0 edge 0 0 1
2 0x0 phys 0x0 edge 0 0 1
3 0x0 phys 0x0 edge 0 0 1
4 0x0 phys 0x0 edge 0 0 1
5 0x0 phys 0x0 edge 0 0 1
6 0x0 phys 0x0 edge 0 0 1
7 0x0 phys 0x0 edge 0 0 1
8 0x0 phys 0x0 edge 0 0 1
9 0x0 phys 0x0 edge 0 0 1
10 0x0 phys 0x0 edge 0 0 1
11 0x0 phys 0x0 edge 0 0 1
12 0x0 phys 0x0 edge 0 0 1
13 0x0 phys 0x0 edge 0 0 1
14 0x0 phys 0x0 edge 0 0 1
15 0x0 phys 0x0 edge 0 0 1
16 0x0 phys 0x0 edge 0 0 1
17 0x0 phys 0x0 edge 0 0 1
18 0x0 phys 0x0 edge 0 0 1
19 0x0 phys 0x0 edge 0 0 1
20 0x0 phys 0x0 edge 0 0 1
21 0x0 phys 0x0 edge 0 0 1
22 0x0 phys 0x0 edge 0 0 1
23 0x0 phys 0x0 edge 0 0 1
24 0x0 phys 0x0 edge 0 0 1
25 0x0 phys 0x0 edge 0 0 1
26 0x0 phys 0x0 edge 0 0 1
27 0x0 phys 0x0 edge 0 0 1
28 0x0 phys 0x0 edge 0 0 1
29 0x0 phys 0x0 edge 0 0 1
30 0x0 phys 0x0 edge 0 0 1
31 0x0 phys 0x0 edge 0 0 1
32 0x0 phys 0x0 edge 0 0 1
33 0x0 phys 0x0 edge 0 0 1
34 0x0 phys 0x0 edge 0 0 1
35 0x0 phys 0x0 edge 0 0 1
36 0x0 phys 0x0 edge 0 0 1
37 0x0 phys 0x0 edge 0 0 1
38 0x0 phys 0x0 edge 0 0 1
39 0x0 phys 0x0 edge 0 0 1
40 0x0 phys 0x0 edge 0 0 1
41 0x0 phys 0x0 edge 0 0 1
42 0x0 phys 0x0 edge 0 0 1
43 0x0 phys 0x0 edge 0 0 1
44 0x0 phys 0x0 edge 0 0 1
45 0x0 phys 0x0 edge 0 0 1
46 0x0 phys 0x0 edge 0 0 1
47 0x0 phys 0x0 edge 0 0 1
48 0x0 phys 0x0 edge 0 0 1
49 0x0 phys 0x0 edge 0 0 1
50 0x0 phys 0x0 edge 0 0 1
51 0x0 phys 0x0 edge 0 0 1
52 0x0 phys 0x0 edge 0 0 1
53 0x0 phys 0x0 edge 0 0 1
54 0x0 phys 0x0 edge 0 0 1
55 0x0 phys 0x0 edge 0 0 1
56 0x0 phys 0x0 edge 0 0 1
57 0x0 phys 0x0 edge 0 0 1
58 0x0 phys 0x0 edge 0 0 1
59 0x0 phys 0x0 edge 0 0 1
60 0x0 phys 0x0 edge 0 0 1
61 0x0 phys 0x0 edge 0 0 1
62 0x0 phys 0x0 edge 0 0 1
63 0x0 phys 0x0 edge 0 0 1
64 0x0 phys 0x0 edge 0 0 1
65 0x0 phys 0x0 edge 0 0 1
66 0x0 phys 0x0 edge 0 0 1
67 0x0 phys 0x0 edge 0 0 1
68 0x0 phys 0x0 edge 0 0 1
69 0x0 phys 0x0 edge 0 0 1
70 0x0 phys 0x0 edge 0 0 1
71 0x0 phys 0x0 edge 0 0 1
72 0x0 phys 0x0 edge 0 0 1
73 0x0 phys 0x0 edge 0 0 1
74 0x0 phys 0x0 edge 0 0 1
75 0x0 phys 0x0 edge 0 0 1
76 0x0 phys 0x0 edge 0 0 1
77 0x0 phys 0x0 edge 0 0 1
78 0x0 phys 0x0 edge 0 0 1
79 0x0 phys 0x0 edge 0 0 1
80 0x0 phys 0x0 edge 0 0 1
81 0x0 phys 0x0 edge 0 0 1
82 0x0 phys 0x0 edge 0 0 1
83 0x0 phys 0x0 edge 0 0 1
84 0x0 phys 0x0 edge 0 0 1
85 0x0 phys 0x0 edge 0 0 1
86 0x0 phys 0x0 edge 0 0 1
87 0x0 phys 0x0 edge 0 0 1
88 0x0 phys 0x0 edge 0 0 1
89 0x0 phys 0x0 edge 0 0 1
90 0x0 phys 0x0 edge 0 0 1
91 0x0 phys 0x0 edge 0 0 1
92 0x0 phys 0x0 edge 0 0 1
93 0x0 phys 0x0 edge 0 0 1
94 0x0 phys 0x0 edge 0 0 1
95 0x0 phys 0x0 edge 0 0 1
96 0x0 phys 0x0 edge 0 0 1
97 0x0 phys 0x0 edge 0 0 1
98 0x0 phys 0x0 edge 0 0 1
99 0x0 phys 0x0 edge 0 0 1
100 0x0 phys 0x0 edge 0 0 1
101 0x0 phys 0x0 edge 0 0 1
102 0x0 phys 0x0 edge 0 0 1
103 0x0 phys 0x0 edge 0 0 1
104 0x0 phys 0x0 edge 0 0 1
105 0x0 phys 0x0 edge 0 0 1
106 0x0 phys 0x0 edge 0 0 1
107 0x0 phys 0x0 edge 0 0 1
108 0x0 phys 0x0 edge 0 0 1
109 0x0 phys 0x0 edge 0 0 1
110 0x0 phys 0x0 edge 0 0 1
111 0x0 phys 0x0 edge 0 0 1
112 0x0 phys 0x0 edge 0 0 1
113 0x0 phys 0x0 edge 0 0 1
114 0x0 phys 0x0 edge 0 0 1
115 0x0 phys 0x0 edge 0 0 1
116 0x0 phys 0x0 edge 0 0 1
117 0x0 phys 0x0 edge 0 0 1
118 0x0 phys 0x0 edge 0 0 1
119 0x0 phys 0x0 edge 0 0 1
ACRN:\>
ACRN:\>
ACRN:\>
ACRN:\>
ACRN:\>vm_console 0
----- Entering VM 0 Shell -----
/* no reaction to any regular keys */
/* ctrl+space */
---Entering ACRN SHELL---
ACRN:\>
Thank you again, I would never get this far without help.
Best regards,
Dubravko
Prima di stampare pensa ai costi ambientali. Please consider the
environment before printing this email. Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Wednesday, February 12, 2020 9:04 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
1) Enable X2APIC from guest Linux
Actually ACRN checks X2APIC capability and it goes panic w/o it. So the host CPUID must have X2APIC support.
We met one case where the BIOS DMAR setting has some thing like "do not use x2apic", which prevents Linux from enabling
X2APIC. Or you may check why the guest doesn't switch to X2APIC from native Linux dmesg?
2) Need Linux logs to help debugging.
Attached is the ACRN built with the following hack, which directs guest serial prints directly to HV console.
Seems Linux goes to the path of numachip_wakeup_secondary() to send SIPIs to secondary CPUs, which we didn't see before.
It's good to see Linux early pintk t help address the issue.
diff --git a/hypervisor/dm/vuart.c b/hypervisor/dm/vuart.c
index 8771a7d8a15f..be5f0da943b8 100644
--- a/hypervisor/dm/vuart.c
+++ b/hypervisor/dm/vuart.c
@@ -295,6 +295,7 @@ static void write_reg(struct acrn_vuart *vu, uint16_t reg, uint8_t value_u8)
vu->lsr |= LSR_OE;
} else {
fifo_putchar(&vu->txfifo, (char)value_u8);
+ printf("%c", value_u8);
}
vu->thre_int_pending = true;
break;
3) "naked" HV is working?
In the meantime, if you want, you can hack ACRN with the following and it won't launch any VMs and the HV console is
supposed to be working. It's just a way to check that ACRN is "generally" working on the HW and you may play with some HV
console commands.
diff --git a/hypervisor/arch/x86/guest/vm.c b/hypervisor/arch/x86/guest/vm.c
index 0471324be065..53ccb6b9acae 100644
--- a/hypervisor/arch/x86/guest/vm.c
+++ b/hypervisor/arch/x86/guest/vm.c
@@ -873,6 +873,8 @@ void launch_vms(uint16_t pcpu_id)
uint16_t vm_id, bsp_id;
struct acrn_vm_config *vm_config;
+ return;
+
for (vm_id = 0U; vm_id < CONFIG_MAX_VM_NUM; vm_id++) {
vm_config = get_vm_config(vm_id);
if ((vm_config->load_order == SOS_VM) || (vm_config->load_order == PRE_LAUNCHED_VM)) {
Best Regards,
Zide
On 2/12/20 11:05 AM, Zide Chen wrote:
> Hi Dubravko,
>
> With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"?
>
> The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS?
>
> Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now.
>
> BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any
> Linux logs.
>
> From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence
> (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT
> and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage?
>
> Best Regards,
> Zide
>
> On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hi Zide,
>>
>> I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the
>> mentioned setting or similar or related to it.
>>
>> We are using coreboot, built according to instructions from
>>
https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added
>> support for our board (the link apparently no longer works). In general it works very well.
>>
>> Best regards,
>> Dubravko
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
>> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w: exorint.com < https://exorint.com/>
>>
>>
>>
>>
>> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>
>> Privacy < https://www.exorint.com/it/privacy>
>> --------------------------------------------------------------------------------------------------------------------------
>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>> *Sent:* Wednesday, February 12, 2020 6:47 PM
>> *To:* acrn-users@... <acrn-users@...>
>> *Subject:* Re: [acrn-users] Getting ACRN to work
>> Hi Dubravko,
>>
>> In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any
>> difference?
>>
>> Best Regards,
>> Zide
>>
>> On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>>> Hello,
>>>
>>> Please find attached the log file and my acrn.conf.
>>>
>>> My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and
>>> I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall
>>> behavior is the same.
>>>
>>> New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to
>>> reference Leaf Hill CRB.
>>>
>>> After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and
>>> the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
>>>
>>> Best regards,
>>> Dubravko
>>>
>>>
>>> *Dubravko Moravski*
>>> /SW engineering/
>>> *Exor Embedded S.r.l.*
>>> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
>>> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
>>> w: exorint.com < https://exorint.com/>
>>>
>>>
>>>
>>> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>>
>>> Privacy < https://www.exorint.com/it/privacy>
>>> --------------------------------------------------------------------------------------------------------------------------
>>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>>> *Sent:* Wednesday, February 12, 2020 1:04 AM
>>> *To:* acrn-users@... <acrn-users@...>
>>> *Subject:* Re: [acrn-users] Getting ACRN to work
>>> Hi Dubravko,
>>>
>>> "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be
>>> ignored for now.
>>>
>>> For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and
>>> need to debug it.
>>>
>>> I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached
>>> bzImage?
>>>
>>> Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in
>>> acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console
>>> is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you
>>> may need to apply the attached patch to ACRN hypervisor.
>>>
>>> Hopefully the full logs can give us some clues.
>>>
>>> Best Regards,
>>> Zide
>>>
>>> zide.chen@...
>>>
>>>
>>
>>
>>
>>
|
|
Hi Dubravko,
I glanced through the discussion and found it might be a similar issue that I hit today with UP2 board.
My problem is that with ACRN enabled the linux kernel cannot bootup APs, without ACRN hypervisor the native kernel works well.
I have a fix that posted on acrn-dev maillist several hours ago. Perhaps you can have a quick try and hope it can help.
I paste it here.
---
On UEFI UP2 board, BIOS startup APs and then they all go into HLT status.
Then Guest OS take over and re-init the APs again. The flows from HV perspective is like:
wait_event(VCPU_EVENT_VIRTUAL_INTERRUPT) -> sleep_thread
-> pause_vcpu(ZOMBIE) -> sleep_thread
-> reset_vcpu
-> launch_vcpu -> wake_vcpu
However, the last wake_vcpu will fail because the cpu event VCPU_EVENT_VIRTUAL_INTERRUPT had not got signaled.
Add signal_event before sleep_thread in pause_vcpu to make sure the vcpu thread is not waiting for event.
Signed-off-by: Shuo A Liu <shuo.a.liu@...>
---
hypervisor/arch/x86/guest/vcpu.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/hypervisor/arch/x86/guest/vcpu.c b/hypervisor/arch/x86/guest/vcpu.c
index 6161889..bd2d2d2 100644
--- a/hypervisor/arch/x86/guest/vcpu.c
+++ b/hypervisor/arch/x86/guest/vcpu.c
@@ -701,6 +701,8 @@ void pause_vcpu(struct acrn_vcpu *vcpu, enum vcpu_state new_state)
vcpu->state = new_state;
if (vcpu->prev_state == VCPU_RUNNING) {
+ /* signal_event will wakeup the vcpu's thread if it's in HLT sleep, else do nothing. */
+ signal_event(&vcpu->events[VCPU_EVENT_VIRTUAL_INTERRUPT]);
sleep_thread(&vcpu->thread_obj);
}
if (pcpu_id != get_pcpu_id()) {
--
2.8.3
Thanks
shuo
toggle quoted message
Show quoted text
From: acrn-users@... <acrn-users@...>
On Behalf Of Dubravko Moravski | Exor Embedded S.r.l.
Sent: Friday, February 14, 2020 23:41
To: acrn-users@...
Subject: Re: [acrn-users] Getting ACRN to work
I see that it's not mentioned anywhere in ACRN log (and I suppose it should be), but it seems to be enabled somehow in non-hypervised Linux:
clear@clr-bcdcb0fadf6546d7b0984fe80030eee8~ $ sudo dmesg | grep -i x2apic
[ 0.165205] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.167422] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.167426] x2apic enabled
[ 0.167448] Switched APIC routing to cluster x2apic.
clear@clr-bcdcb0fadf6546d7b0984fe80030eee8~ $
Our BIOS does not have a GUI setting to manually enable it, and we couldn't find where to explicitly enable it in source code. It's mentioned at many places; though not in DMAR table.
Can we conclude from the log above that it's ok, or we still need to find a way to enable it in ACRN or BIOS?
I'm working on this and will report later.
By the way, I think my last logs might be inconsistent with earlier ones (with endless SIPI messages) because earlier I might have in a hurry just pressed enter in the boot menu to choose whichever
option was selected, instead of selecting ACRN. Now I'm more careful.
Removing 'quiet' parameter didn't help.
... is working. With your previous acrn.efi and bzImage from around Wednesday noon Central European Time, after serial boot log stops, I can indeed switch to ACRN console with ctrl+space. If I
switch back to SOS, nothing happens. I can only return back to ACRN console. Here are some printouts:
[13133696us][cpu=0][vm0:vcpu0][sev=3][seq=322]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13146659us][cpu=0][vm0:vcpu0][sev=3][seq=323]:vcpu1 paused, new state: 4
[13154097us][cpu=0][vm0:vcpu0][sev=3][seq=324]:vcpu1 reset
[13160078us][cpu=0][vm0:vcpu0][sev=3][seq=325]:vlapic: vlapic_reset 1693 vm0 vcpu1 initial lapic_id: 2000000
[13170960us][cpu=0][vm0:vcpu0][sev=3][seq=326]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13183985us][cpu=0][vm0:vcpu0][sev=3][seq=327]:vcpu2 paused, new state: 4
[13191429us][cpu=0][vm0:vcpu0][sev=3][seq=328]:vcpu2 reset
[13197408us][cpu=0][vm0:vcpu0][sev=3][seq=329]:vlapic: vlapic_reset 1693 vm0 vcpu2 initial lapic_id: 4000000
[13208295us][cpu=0][vm0:vcpu0][sev=3][seq=330]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 500 icr_low c4500 sipi cnt 0 vcpu state 2
[13221324us][cpu=0][vm0:vcpu0][sev=3][seq=331]:vcpu3 paused, new state: 4
[13228769us][cpu=0][vm0:vcpu0][sev=3][seq=332]:vcpu3 reset
[13234748us][cpu=0][vm0:vcpu0][sev=3][seq=333]:vlapic: vlapic_reset 1693 vm0 vcpu3 initial lapic_id: 6000000
[13255651us][cpu=0][vm0:vcpu0][sev=3][seq=334]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13268597us][cpu=0][vm0:vcpu0][sev=3][seq=335]:vlapic: Start Secondary VCPU1 for VM[0]...
[13277605us][cpu=0][vm0:vcpu0][sev=3][seq=336]:vcpu1 scheduled on pcpu1 vcpu->state 1
[13286228us][cpu=0][vm0:vcpu0][sev=3][seq=337]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13299256us][cpu=0][vm0:vcpu0][sev=3][seq=338]:vlapic: Start Secondary VCPU2 for VM[0]...
[13308274us][cpu=0][vm0:vcpu0][sev=3][seq=339]:vcpu2 scheduled on pcpu2 vcpu->state 1
[13316902us][cpu=0][vm0:vcpu0][sev=3][seq=340]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 600 icr_low c469f sipi cnt 1 vcpu state 1
[13329919us][cpu=0][vm0:vcpu0][sev=3][seq=341]:vlapic: Start Secondary VCPU3 for VM[0]...
[13338926us][cpu=0][vm0:vcpu0][sev=3][seq=342]:vcpu3 scheduled on pcpu3 vcpu->state 1
[13347768us][cpu=0][vm0:vcpu0][sev=3][seq=343]:vlapic: pcpu0 Sending SIPI to vcpu1, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
[13360707us][cpu=0][vm0:vcpu0][sev=3][seq=344]:vlapic: pcpu0 Sending SIPI to vcpu2, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
[13373751us][cpu=0][vm0:vcpu0][sev=3][seq=345]:vlapic: pcpu0 Sending SIPI to vcpu3, mode 600 icr_low c469f sipi cnt 0 vcpu state 2
---Entering ACRN SHELL---
ACRN:\>
ACRN:\>version
HV version 1.6-unstable-2020-02-11 15:28:54-4990d8574159 DBG (daily tag: acrn-2020w03.4-140000p) build by zide
VM_UUID VM_ID VM_NAME VM_STATE
================================ ===== ================================ ========
dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Started
VM ID PCPU ID VCPU ID VCPU ROLE VCPU STATE THREAD STATE
===== ======= ======= ========= ========== ==========
0 0 0 PRIMARY Running RUNNING
0 1 1 SECONDARY Running BLOCKED
0 2 2 SECONDARY Running BLOCKED
0 3 3 SECONDARY Running BLOCKED
Error: Invalid parameters.
PIN VEC DM DEST TM DELM IRR MASK
0 0x0 phys 0x0 edge 0 0 1
1 0x0 phys 0x0 edge 0 0 1
2 0x0 phys 0x0 edge 0 0 1
3 0x0 phys 0x0 edge 0 0 1
4 0x0 phys 0x0 edge 0 0 1
5 0x0 phys 0x0 edge 0 0 1
6 0x0 phys 0x0 edge 0 0 1
7 0x0 phys 0x0 edge 0 0 1
8 0x0 phys 0x0 edge 0 0 1
9 0x0 phys 0x0 edge 0 0 1
10 0x0 phys 0x0 edge 0 0 1
11 0x0 phys 0x0 edge 0 0 1
12 0x0 phys 0x0 edge 0 0 1
13 0x0 phys 0x0 edge 0 0 1
14 0x0 phys 0x0 edge 0 0 1
15 0x0 phys 0x0 edge 0 0 1
16 0x0 phys 0x0 edge 0 0 1
17 0x0 phys 0x0 edge 0 0 1
18 0x0 phys 0x0 edge 0 0 1
19 0x0 phys 0x0 edge 0 0 1
20 0x0 phys 0x0 edge 0 0 1
21 0x0 phys 0x0 edge 0 0 1
22 0x0 phys 0x0 edge 0 0 1
23 0x0 phys 0x0 edge 0 0 1
24 0x0 phys 0x0 edge 0 0 1
25 0x0 phys 0x0 edge 0 0 1
26 0x0 phys 0x0 edge 0 0 1
27 0x0 phys 0x0 edge 0 0 1
28 0x0 phys 0x0 edge 0 0 1
29 0x0 phys 0x0 edge 0 0 1
30 0x0 phys 0x0 edge 0 0 1
31 0x0 phys 0x0 edge 0 0 1
32 0x0 phys 0x0 edge 0 0 1
33 0x0 phys 0x0 edge 0 0 1
34 0x0 phys 0x0 edge 0 0 1
35 0x0 phys 0x0 edge 0 0 1
36 0x0 phys 0x0 edge 0 0 1
37 0x0 phys 0x0 edge 0 0 1
38 0x0 phys 0x0 edge 0 0 1
39 0x0 phys 0x0 edge 0 0 1
40 0x0 phys 0x0 edge 0 0 1
41 0x0 phys 0x0 edge 0 0 1
42 0x0 phys 0x0 edge 0 0 1
43 0x0 phys 0x0 edge 0 0 1
44 0x0 phys 0x0 edge 0 0 1
45 0x0 phys 0x0 edge 0 0 1
46 0x0 phys 0x0 edge 0 0 1
47 0x0 phys 0x0 edge 0 0 1
48 0x0 phys 0x0 edge 0 0 1
49 0x0 phys 0x0 edge 0 0 1
50 0x0 phys 0x0 edge 0 0 1
51 0x0 phys 0x0 edge 0 0 1
52 0x0 phys 0x0 edge 0 0 1
53 0x0 phys 0x0 edge 0 0 1
54 0x0 phys 0x0 edge 0 0 1
55 0x0 phys 0x0 edge 0 0 1
56 0x0 phys 0x0 edge 0 0 1
57 0x0 phys 0x0 edge 0 0 1
58 0x0 phys 0x0 edge 0 0 1
59 0x0 phys 0x0 edge 0 0 1
60 0x0 phys 0x0 edge 0 0 1
61 0x0 phys 0x0 edge 0 0 1
62 0x0 phys 0x0 edge 0 0 1
63 0x0 phys 0x0 edge 0 0 1
64 0x0 phys 0x0 edge 0 0 1
65 0x0 phys 0x0 edge 0 0 1
66 0x0 phys 0x0 edge 0 0 1
67 0x0 phys 0x0 edge 0 0 1
68 0x0 phys 0x0 edge 0 0 1
69 0x0 phys 0x0 edge 0 0 1
70 0x0 phys 0x0 edge 0 0 1
71 0x0 phys 0x0 edge 0 0 1
72 0x0 phys 0x0 edge 0 0 1
73 0x0 phys 0x0 edge 0 0 1
74 0x0 phys 0x0 edge 0 0 1
75 0x0 phys 0x0 edge 0 0 1
76 0x0 phys 0x0 edge 0 0 1
77 0x0 phys 0x0 edge 0 0 1
78 0x0 phys 0x0 edge 0 0 1
79 0x0 phys 0x0 edge 0 0 1
80 0x0 phys 0x0 edge 0 0 1
81 0x0 phys 0x0 edge 0 0 1
82 0x0 phys 0x0 edge 0 0 1
83 0x0 phys 0x0 edge 0 0 1
84 0x0 phys 0x0 edge 0 0 1
85 0x0 phys 0x0 edge 0 0 1
86 0x0 phys 0x0 edge 0 0 1
87 0x0 phys 0x0 edge 0 0 1
88 0x0 phys 0x0 edge 0 0 1
89 0x0 phys 0x0 edge 0 0 1
90 0x0 phys 0x0 edge 0 0 1
91 0x0 phys 0x0 edge 0 0 1
92 0x0 phys 0x0 edge 0 0 1
93 0x0 phys 0x0 edge 0 0 1
94 0x0 phys 0x0 edge 0 0 1
95 0x0 phys 0x0 edge 0 0 1
96 0x0 phys 0x0 edge 0 0 1
97 0x0 phys 0x0 edge 0 0 1
98 0x0 phys 0x0 edge 0 0 1
99 0x0 phys 0x0 edge 0 0 1
100 0x0 phys 0x0 edge 0 0 1
101 0x0 phys 0x0 edge 0 0 1
102 0x0 phys 0x0 edge 0 0 1
103 0x0 phys 0x0 edge 0 0 1
104 0x0 phys 0x0 edge 0 0 1
105 0x0 phys 0x0 edge 0 0 1
106 0x0 phys 0x0 edge 0 0 1
107 0x0 phys 0x0 edge 0 0 1
108 0x0 phys 0x0 edge 0 0 1
109 0x0 phys 0x0 edge 0 0 1
110 0x0 phys 0x0 edge 0 0 1
111 0x0 phys 0x0 edge 0 0 1
112 0x0 phys 0x0 edge 0 0 1
113 0x0 phys 0x0 edge 0 0 1
114 0x0 phys 0x0 edge 0 0 1
115 0x0 phys 0x0 edge 0 0 1
116 0x0 phys 0x0 edge 0 0 1
117 0x0 phys 0x0 edge 0 0 1
118 0x0 phys 0x0 edge 0 0 1
119 0x0 phys 0x0 edge 0 0 1
----- Entering VM 0 Shell -----
/* no reaction to any regular keys */
---Entering ACRN SHELL---
ACRN:\>
Thank you again, I would never get this far without help.
Prima
di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy
Hi Dubravko,
1) Enable X2APIC from guest Linux
Actually ACRN checks X2APIC capability and it goes panic w/o it. So the host CPUID must have X2APIC support.
We met one case where the BIOS DMAR setting has some thing like "do not use x2apic", which prevents Linux from enabling
X2APIC. Or you may check why the guest doesn't switch to X2APIC from native Linux dmesg?
2) Need Linux logs to help debugging.
Attached is the ACRN built with the following hack, which directs guest serial prints directly to HV console.
Seems Linux goes to the path of numachip_wakeup_secondary() to send SIPIs to secondary CPUs, which we didn't see before.
It's good to see Linux early pintk t help address the issue.
diff --git a/hypervisor/dm/vuart.c b/hypervisor/dm/vuart.c
index 8771a7d8a15f..be5f0da943b8 100644
--- a/hypervisor/dm/vuart.c
+++ b/hypervisor/dm/vuart.c
@@ -295,6 +295,7 @@ static void write_reg(struct acrn_vuart *vu, uint16_t reg, uint8_t value_u8)
vu->lsr |= LSR_OE;
} else {
fifo_putchar(&vu->txfifo, (char)value_u8);
+ printf("%c", value_u8);
}
vu->thre_int_pending = true;
break;
3) "naked" HV is working?
In the meantime, if you want, you can hack ACRN with the following and it won't launch any VMs and the HV console is
supposed to be working. It's just a way to check that ACRN is "generally" working on the HW and you may play with some HV
console commands.
diff --git a/hypervisor/arch/x86/guest/vm.c b/hypervisor/arch/x86/guest/vm.c
index 0471324be065..53ccb6b9acae 100644
--- a/hypervisor/arch/x86/guest/vm.c
+++ b/hypervisor/arch/x86/guest/vm.c
@@ -873,6 +873,8 @@ void launch_vms(uint16_t pcpu_id)
uint16_t vm_id, bsp_id;
struct acrn_vm_config *vm_config;
+ return;
+
for (vm_id = 0U; vm_id < CONFIG_MAX_VM_NUM; vm_id++) {
vm_config = get_vm_config(vm_id);
if ((vm_config->load_order == SOS_VM) || (vm_config->load_order == PRE_LAUNCHED_VM)) {
Best Regards,
Zide
On 2/12/20 11:05 AM, Zide Chen wrote:
> Hi Dubravko,
>
> With my acrn.efi, it switches to guest console by default. Does it switch back to ACRN console by pressing "Ctrl + Space"?
>
> The logs show that X2APIC is not enabled by Linux. Is it possible to enable X2APIC from BIOS?
>
> Without X2APIC, ACRN doesn't work. Though I'm expecting ACRN should go beyond what it is now.
>
> BTW, can you remove "quiet" from the Linux bootargs? This may not really help, but I'm puzzled why it doesn't print any
> Linux logs.
>
> From the logs, one big difference with my images (acrn.efi + bzImage) is that now it doesn't repeat the SIPI sequence
> (INIT -> Start up -> INIT -> Start up) endless, instead, after launching 3 VCPUs, the guest sends broadcasting SIPI INIT
> and Start Up only. This is a good news, but not sure that's because of the new acrn.efi, or the bzImage?
>
> Best Regards,
> Zide
>
> On 2/12/20 10:36 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hi Zide,
>>
>> I've checked all the menus and settings in our BIOS, unfortunately I couldn't find anything that looks like the
>> mentioned setting or similar or related to it.
>>
>> We are using coreboot, built according to instructions from
>>
https://software.intel.com/en-us/firmware/sites/default/files/intelatome3900-0.71-buildinstructions.txt, plus we added
>> support for our board (the link apparently no longer works). In general it works very well.
>>
>> Best regards,
>> Dubravko
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
>> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w: exorint.com <https://exorint.com/>
>>
>>
>>
>>
>> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>> --------------------------------------------------------------------------------------------------------------------------
>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>> *Sent:* Wednesday, February 12, 2020 6:47 PM
>> *To:* acrn-users@... <acrn-users@...>
>> *Subject:* Re: [acrn-users] Getting ACRN to work
>> Hi Dubravko,
>>
>> In the meantime, is there something like "AP Thread Idle Manner" in your BIOS? And changing its setting makes any
>> difference?
>>
>> Best Regards,
>> Zide
>>
>> On 2/12/20 8:02 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>>> Hello,
>>>
>>> Please find attached the log file and my acrn.conf.
>>>
>>> My old BIOS didn't have any settings related to Mwait, so I've installed newer BIOS which has "Monitor Mwait Enable" and
>>> I've ensured it's set to "Disabled". Changing the BIOS seems to have also changed some data in logs, but the overall
>>> behavior is the same.
>>>
>>> New acrn.efi seems to work reasonably well. Our custom board is obviously specific, but in general it is similar to
>>> reference Leaf Hill CRB.
>>>
>>> After the last printout from the log, everything stops working (there is nothing on the screen attached to the board and
>>> the keyboard doesn't do anything; also typing in the serial terminal doesn't do anything).
>>>
>>> Best regards,
>>> Dubravko
>>>
>>>
>>> *Dubravko Moravski*
>>> /SW engineering/
>>> *Exor Embedded S.r.l.*
>>> p: +38 512455659 <tel:+38 512455659> m:+38 5915402413 <tel:+38 5915402413>
>>> a: Slavonska avenija, 50, Zagreb, Croatia, 10000
>>> w: exorint.com <https://exorint.com/>
>>>
>>>
>>>
>>> Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>>
>>> Privacy <https://www.exorint.com/it/privacy>
>>> --------------------------------------------------------------------------------------------------------------------------
>>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>>> *Sent:* Wednesday, February 12, 2020 1:04 AM
>>> *To:* acrn-users@... <acrn-users@...>
>>> *Subject:* Re: [acrn-users] Getting ACRN to work
>>> Hi Dubravko,
>>>
>>> "Spurious vector" could because of external interrupts not registered in hypervisor. This may not be an issue and can be
>>> ignored for now.
>>>
>>> For the repeating "Starting VCPU 3" issue, it seems some thing related to SIPI delivery. But nothing is obvious to me and
>>> need to debug it.
>>>
>>> I built a Clear Linux kernel with additional logs showing how it's bringing up APs. Could you please try the attached
>>> bzImage?
>>>
>>> Also please enable early printk by adding "ignore_loglevel earlyprintk=serial,ttyS0,115200n8,keep" to the options in
>>> acrn.conf; And use the attached acrn.efi (I enabled some additional logs and print out Linux logs even before ACRN console
>>> is operational). If acrn.efi doesn't boot, which is highly possibly since it's not built against your custom board, you
>>> may need to apply the attached patch to ACRN hypervisor.
>>>
>>> Hopefully the full logs can give us some clues.
>>>
>>> Best Regards,
>>> Zide
>>>
>>> zide.chen@...
>>>
>>>
>>
>>
>>
>>
|
|