Date   

Re: Getting ACRN to work

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi Zide,

The full log is below, with some additional /* comments */. I've noticed now that ACRN console prompt does appear, but it is unusable due to all the debug messages.

ACRN Hypervisor
calibrate_tsc, tsc_khz=1593600
[7621776us][cpu=0][(null)][sev=2][seq=1]:HV version 1.6-unstable-2020-02-11 17:29:55-4b983876-dirty DBG (daily tag:acrn-2020w04.1-140000p) build by clear, start time 7596708us
[7639207us][cpu=0][(null)][sev=2][seq=2]:API version 1.0
[7644984us][cpu=0][(null)][sev=2][seq=3]:Detect processor: Intel(R) Atom(TM) Processor E3940 @ 1.60GHz
[7655270us][cpu=0][(null)][sev=1][seq=4]:SECURITY WARNING!!!!!!
[7661731us][cpu=0][(null)][sev=1][seq=5]:Please apply the latest CPU uCode patch!
[7670865us][cpu=0][(null)][sev=5][seq=6]:hide pci uart dev: (0:18:0)
[7720135us][cpu=0][(null)][sev=5][seq=9]:profiling_setup: entering
[7720133us][cpu=1][(null)][sev=5][seq=7]:profiling_setup: entering
[7720136us][cpu=3][(null)][sev=5][seq=10]:profiling_setup: entering
[7720134us][cpu=2][(null)][sev=5][seq=8]:profiling_setup: entering
[7726787us][cpu=0][(null)][sev=5][seq=11]:profiling_setup: calling request_irq
[7733543us][cpu=1][(null)][sev=5][seq=12]:profiling_setup: exiting
[7740402us][cpu=3][(null)][sev=5][seq=13]:profiling_setup: exiting
[7747161us][cpu=2][(null)][sev=5][seq=14]:profiling_setup: exiting
[7755094us][cpu=0][(null)][sev=5][seq=15]:profiling_setup: exiting
[7783951us][cpu=0][(null)][sev=5][seq=16]:System S3/S5 is supported.
[7791335us][cpu=0][(null)][sev=5][seq=17]:Creating VCPU working on PCPU0
[7798597us][cpu=0][(null)][sev=5][seq=18]:Create VM0-VCPU0, Role: PRIMARY
[7806135us][cpu=0][(null)][sev=5][seq=19]:Creating VCPU working on PCPU1
[7813389us][cpu=0][(null)][sev=5][seq=20]:Create VM0-VCPU1, Role: SECONDARY
[7821031us][cpu=0][(null)][sev=5][seq=21]:Creating VCPU working on PCPU2
[7828376us][cpu=0][(null)][sev=5][seq=22]:Create VM0-VCPU2, Role: SECONDARY
[7836012us][cpu=0][(null)][sev=5][seq=23]:Creating VCPU working on PCPU3
[7843359us][cpu=0][(null)][sev=5][seq=24]:Create VM0-VCPU3, Role: SECONDARY
[7850998us][cpu=0][(null)][sev=4][seq=25]:Spurious vector: 0x50.
ACRN:\>[7858255us][cpu=0][(null)][sev=2][seq=26]:Start VM id: 0 name: ACRN SOS VM
[7865814us][cpu=0][vm0:vcpu0][sev=5][seq=27]:VM 0 Starting VCPU 0
[7872492us][cpu=0][vm0:vcpu0][sev=5][seq=28]:VM 0 VCPU 0 successfully launched
[7882501us][cpu=0][vm0:vcpu0][sev=4][seq=29]:Spurious vector: 0x50.
[7899292us][cpu=0][vm0:vcpu0][sev=4][seq=30]:Spurious vector: 0x50.
[7916327us][cpu=0][vm0:vcpu0][sev=4][seq=31]:Spurious vector: 0x50.
/* seq 32..99 are all identical entries like the surrounding ones */
[9075356us][cpu=0][vm0:vcpu0][sev=4][seq=100]:Spurious vector: 0x50.
[9092270us][cpu=0][vm0:vcpu0][sev=4][seq=101]:Spurious vector: 0x50.
[9109179us][cpu=0][vm0:vcpu0][sev=4][seq=102]:Spurious vector: 0x50.
[9126089us][cpu=0][vm0:vcpu0][sev=4][seq=103]:Spurious vector: 0x50.
[9163364us][cpu=0][vm0:vcpu0][sev=2][seq=104]:write_cfg 0:18.0 not found! off: 0x3c, val: 0x4
/* approximately here I've choosed an entry from the boot menu */

[9173239us][cpu=0][vm0:vcpu0][sev=3][seq=105]:vlapic: Start Secondary VCPU1 for VM[0]...
[9182086us][cpu=1][vm0:vcpu1][sev=5][seq=106]:VM 0 Starting VCPU 1
[9182310us][cpu=0][vm0:vcpu0][sev=3][seq=107]:vlapic: Start Secondary VCPU2 for VM[0]...
[9197761us][cpu=2][vm0:vcpu2][sev=5][seq=108]:VM 0 Starting VCPU 2
[9197985us][cpu=0][vm0:vcpu0][sev=3][seq=109]:vlapic: Start Secondary VCPU3 for VM[0]...
[9213427us][cpu=3][vm0:vcpu3][sev=5][seq=110]:VM 0 Starting VCPU 3
[9220288us][cpu=0][vm0:vcpu0][sev=3][seq=111]:vlapic: Start Secondary VCPU3 for VM[0]...
[9229124us][cpu=3][vm0:vcpu3][sev=5][seq=112]:VM 0 Starting VCPU 3
[9235992us][cpu=0][vm0:vcpu0][sev=3][seq=113]:vlapic: Start Secondary VCPU3 for VM[0]...
[9244836us][cpu=3][vm0:vcpu3][sev=5][seq=114]:VM 0 Starting VCPU 3
[9251701us][cpu=0][vm0:vcpu0][sev=3][seq=115]:vlapic: Start Secondary VCPU3 for VM[0]...
[9260536us][cpu=3][vm0:vcpu3][sev=5][seq=116]:VM 0 Starting VCPU 3
[9267402us][cpu=0][vm0:vcpu0][sev=3][seq=117]:vlapic: Start Secondary VCPU3 for VM[0]...
[9276237us][cpu=3][vm0:vcpu3][sev=5][seq=118]:VM 0 Starting VCPU 3
[9283102us][cpu=0][vm0:vcpu0][sev=3][seq=119]:vlapic: Start Secondary VCPU3 for VM[0]...
[9291941us][cpu=3][vm0:vcpu3][sev=5][seq=120]:VM 0 Starting VCPU 3
[9298810us][cpu=0][vm0:vcpu0][sev=3][seq=121]:vlapic: Start Secondary VCPU3 for VM[0]...
[9307648us][cpu=3][vm0:vcpu3][sev=5][seq=122]:VM 0 Starting VCPU 3
[9314511us][cpu=0][vm0:vcpu0][sev=3][seq=123]:vlapic: Start Secondary VCPU3 for VM[0]...
[9323355us][cpu=3][vm0:vcpu3][sev=5][seq=124]:VM 0 Starting VCPU 3
[9330216us][cpu=0][vm0:vcpu0][sev=3][seq=125]:vlapic: Start Secondary VCPU3 for VM[0]...
[9339041us][cpu=3][vm0:vcpu3][sev=5][seq=126]:VM 0 Starting VCPU 3
/* this continues indefinitely... */



Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Tuesday, February 11, 2020 5:31 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
 
Hi Dubravko,

Is it possible to post the full log here?

Thanks,
Zide


On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi Zide,
>
> Your instructions were clear, unfortunately I'm still having issues.
>
> I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded
> Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT
> now set to 5 (previously was 6).
>
> I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
>
>     sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u
>     "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
>
> Then I've set clr-boot-manager options according to instructions. This all went smoothly.
>
> After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
>
>     [21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50.
>     [21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50.
>     [21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50.
>     [21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50.
>     [21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.
>
>
> When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
>
>     [34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3
>     [34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]...
>     [34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3
>     [34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]...
>     [34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3
>     [34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
>
> ... and it never gets to SOS or ACRN console.
>
> As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux
> shows that all 4 cores are alive and occasionally doing something.
>
> Best regards,
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Monday, February 10, 2020 8:00 PM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
> Hi Dubravko,
>
> If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.
>
> You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.
>
> Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged
> to do that during bring-up.
>
> I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list",
> "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by
> "vm_console 0", and switch back to HV console by pressing "Ctrl + Space".
>
> https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console
>
> If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
> https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html
>
> The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample:
> misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you
> installed.
>
>
> Best Regards,
> Zide
>
>
> On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
>> Hi Zide,
>>
>> You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
>>
>> MADT and DMAR tables seem to be correct.
>> I'm using acrn.efi, RELOC is enabled.
>>
>> Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number
>> of printouts like these:
>>
>>     [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
>>     [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
>>     [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
>>     [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
>>
>> They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also
>> nothing else happens, just these printouts.
>>
>> RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
>>
>> I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just
>> doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if
>> that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
>>
>> Thank you and best regards,
>> Dubravko
>>
>>
>>
>> *Dubravko Moravski*
>> /SW engineering/
>> *Exor Embedded S.r.l.*
>> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
>> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
>> w:     exorint.com <https://exorint.com/>
>>
>>       
>>
>>
>>   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>>
>> Privacy <https://www.exorint.com/it/privacy>
>> --------------------------------------------------------------------------------------------------------------------------
>> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
>> Lists.Projectacrn.Org <zide.chen=intel.com@...>
>> *Sent:* Monday, February 10, 2020 5:50 PM
>> *To:* acrn-users@... <acrn-users@...>
>> *Subject:* Re: [acrn-users] Getting ACRN to work
>> Hi Dubravko,
>>
>> Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
>> others.
>>
>> For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
>> CPU number from ACPI table.
>>
>> Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
>> it finds 0 pcpus.
>>
>> BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
>> messages.
>> https://github.com/projectacrn/acrn-hypervisor/pull/4381
>>
>> Best Regards,
>> Zide
>>
>>
>>
>
>
>
>




Re: Getting ACRN to work

Chen, Zide
 

Hi Dubravko,

Is it possible to post the full log here?

Thanks,
Zide

On 2/11/20 6:19 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
Your instructions were clear, unfortunately I'm still having issues.
I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT now set to 5 (previously was 6).
I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u
"bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
Then I've set clr-boot-manager options according to instructions. This all went smoothly.
After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
[21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50.
[21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50.
[21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50.
[21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50.
[21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.
When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
[34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting CPU 3
[34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]...
[34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3
[34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]...
[34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3
[34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
... and it never gets to SOS or ACRN console.
As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something.
Best regards,
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Monday, February 10, 2020 8:00 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.
You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.
Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged
to do that during bring-up.
I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list",
"vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by
"vm_console 0", and switch back to HV console by pressing "Ctrl + Space".
https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console
If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html
The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample:
misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you
installed.
Best Regards,
Zide
On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
MADT and DMAR tables seem to be correct.
I'm using acrn.efi, RELOC is enabled.
Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number
of printouts like these:
     [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
     [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
     [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
     [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also
nothing else happens, just these printouts.
RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just
doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if
that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
Thank you and best regards,
Dubravko
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
w:     exorint.com <https://exorint.com/>
   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Monday, February 10, 2020 5:50 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
others.
For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
CPU number from ACPI table.
Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
it finds 0 pcpus.
BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
messages.
https://github.com/projectacrn/acrn-hypervisor/pull/4381
Best Regards,
Zide


Re: Getting ACRN to work

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi Zide,

Your instructions were clear, unfortunately I'm still having issues.

I have followed the instructions from https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html, and first upgraded Clear Linux to the same version as used there, just in case. Also I've rebuilt ACRN with CONFIG_CONSOLE_LOGLEVEL_DEFAULT now set to 5 (previously was 6).

I've verified that I have all the required files on the EFI partition. I've added a new boot entry with:
sudo efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/mmcblk0 -p 1 -L "ACRN hypervisor" -u "bootloader=\EFI\org.clearlinux\bootloaderx64.efi uart=bdf@0:18.0"
Then I've set clr-boot-manager options according to instructions. This all went smoothly.

After the reboot, ACRN loads. While it's waiting in the boot menu it continuously prints out a lot of these:
[21481383us][cpu=0][vm0:vcpu0][sev=4][seq=827]:Spurious vector: 0x50.
[21498377us][cpu=0][vm0:vcpu0][sev=4][seq=828]:Spurious vector: 0x50.
[21515373us][cpu=0][vm0:vcpu0][sev=4][seq=829]:Spurious vector: 0x50.
[21532360us][cpu=0][vm0:vcpu0][sev=4][seq=830]:Spurious vector: 0x50.
[21549353us][cpu=0][vm0:vcpu0][sev=4][seq=831]:Spurious vector: 0x50.

When I choose the first option that I believe should start the Service OS, it then continuously prints out these:
[34072934us][cpu=3][vm0:vcpu3][sev=5][seq=1645]:VM 0 Starting VCPU 3
[34079998us][cpu=0][vm0:vcpu0][sev=3][seq=1646]:vlapic: Start Secondary VCPU3 for VM[0]...
[34089037us][cpu=3][vm0:vcpu3][sev=5][seq=1647]:VM 0 Starting VCPU 3
[34096100us][cpu=0][vm0:vcpu0][sev=3][seq=1648]:vlapic: Start Secondary VCPU3 for VM[0]...
[34105131us][cpu=3][vm0:vcpu3][sev=5][seq=1649]:VM 0 Starting VCPU 3
[34112188us][cpu=0][vm0:vcpu0][sev=3][seq=1650]:vlapic: Start Secondary VCPU3 for VM[0]...
... and it never gets to SOS or ACRN console.

As far as I know, there is nothing wrong with any of the CPU cores; all are enabled in BIOS; "top" in regular Clear Linux shows that all 4 cores are alive and occasionally doing something.

Best regards,
Dubravko


Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Monday, February 10, 2020 8:00 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
 
Hi Dubravko,

If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.

You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.

Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged
to do that during bring-up.

I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list",
"vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by
"vm_console 0", and switch back to HV console by pressing "Ctrl + Space".

https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console

If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html

The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample:
misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you
installed.


Best Regards,
Zide


On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
> Hi Zide,
>
> You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
>
> MADT and DMAR tables seem to be correct.
> I'm using acrn.efi, RELOC is enabled.
>
> Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number
> of printouts like these:
>
>     [836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
>     [836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
>     [836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
>     [836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
>
> They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also
> nothing else happens, just these printouts.
>
> RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
>
> I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just
> doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if
> that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
>
> Thank you and best regards,
> Dubravko
>
>
>
> *Dubravko Moravski*
> /SW engineering/
> *Exor Embedded S.r.l.*
> p:     +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
> a:     Slavonska avenija, 50, Zagreb, Croatia, 10000
> w:     exorint.com <https://exorint.com/>
>
>       
>
>
>   Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
>
> Privacy <https://www.exorint.com/it/privacy>
> --------------------------------------------------------------------------------------------------------------------------
> *From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via
> Lists.Projectacrn.Org <zide.chen=intel.com@...>
> *Sent:* Monday, February 10, 2020 5:50 PM
> *To:* acrn-users@... <acrn-users@...>
> *Subject:* Re: [acrn-users] Getting ACRN to work
> Hi Dubravko,
>
> Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
> others.
>
> For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
> CPU number from ACPI table.
>
> Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
> it finds 0 pcpus.
>
> BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
> messages.
> https://github.com/projectacrn/acrn-hypervisor/pull/4381
>
> Best Regards,
> Zide
>
>
>




Canceled: 2020 ACRN Project Technical Community Meeting (2020/1~2020/7): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

<No topic for this week, let’s skip for 2/12>
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
  • Meets every Wednesday, Starting Nov 07, 2018: 11AM-12AM (China-Shanghai), 7PM-8PM (US-West Coast), 3AM-4AM (Europe-London)
  • Chairperson: Hongbo Wang, hongbo.wang@... (Intel)
  • Online conference link: https://zoom.us/j/457171121
  • Zoom Meeting ID: 457-171-121
  • Online conference phone:
  • US: +1 669 900 6833  or +1 646 558 8656   or +1 877 369 0926 (Toll Free) or +1 855 880 1246 (Toll Free)
  • China: +86 010 87833177  or 400 669 9381 (Toll Free)
  • Germany: +49 (0) 30 3080 6188  or +49 800 724 3138 (Toll Free)
  • Additional international phone numbers
  • Meeting Notes:
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 
 
 


Re: Getting ACRN to work

Chen, Zide
 

Hi Dubravko,

If you see endless vmexit messages, it's possibly that you have successfully loaded the guest kernel.

You may remove the vmexit logs by configuring CONFIG_CONSOLE_LOGLEVEL_DEFAULT to any number < 6.

Build it with "RELEASE=1" disables the hypervisor serial console so you won't be able to see anything. So it's discouraged to do that during bring-up.

I'm expecting you can see the console prompt: "ACRN:\>", and you can try some basic commands like "help", "vm_list", "vcpu_list", etc. If you find any guest VM is running from "vm_list" command, you can switch to the guest console by "vm_console 0", and switch back to HV console by pressing "Ctrl + Space".

https://projectacrn.github.io/latest/tutorials/debug.html?highlight=console

If you don't see any VM running, you may following this tutorial (of cause you can try other scenario):
https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html

The key point is at the end you need to install a clear Linux bzImage and acrn.efi (sample: misc/efi-stub/clearlinux/acrn.conf) in the EFI partition, and the "Linux" line in acrn.efi points to the bzImage you installed.


Best Regards,
Zide

On 2/10/20 10:39 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.
MADT and DMAR tables seem to be correct.
I'm using acrn.efi, RELOC is enabled.
Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these:
[836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
[836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
[836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
[836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001
They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts.
RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.
I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.
Thank you and best regards,
Dubravko
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Monday, February 10, 2020 5:50 PM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
others.
For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
CPU number from ACPI table.
Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
it finds 0 pcpus.
BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
messages.
https://github.com/projectacrn/acrn-hypervisor/pull/4381
Best Regards,
Zide


Re: Getting ACRN to work

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi Zide,

You were right again, I don't know how it happened but my MAX_PCPU_NUM was set to 3, while there are 4 CPUs.

MADT and DMAR tables seem to be correct.
I'm using acrn.efi, RELOC is enabled.

Now the hypervisor seems to be almost working... The initialization seems to complete fine, then I'm getting a huge number of printouts like these:
[836682306us][cpu=0][vm0:vcpu0][sev=6][seq=31854]:Exit Reason: 0x0000000000000001
[836722306us][cpu=0][vm0:vcpu0][sev=6][seq=31855]:Exit Reason: 0x0000000000000001
[836762306us][cpu=0][vm0:vcpu0][sev=6][seq=31856]:Exit Reason: 0x0000000000000001
[836802306us][cpu=0][vm0:vcpu0][sev=6][seq=31857]:Exit Reason: 0x0000000000000001

They continue indefinitely. As far as I can tell, there are no errors (unless these indicate some errors), but also nothing else happens, just these printouts.

RELEASE=1 build doesn't display the above printouts, but also nothing seems to happen.

I think I might not have configured loading of the next stage (Linux loader) correctly, so it's possible that ACRN just doesn't have anything to do. The tutorials aren't quite clear regarding doing that, for new custom boards. Though if that's the problem, I would expect at least some warning message to be displayed. I'll check that first anyway.

Thank you and best regards,
Dubravko



Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Monday, February 10, 2020 5:50 PM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
 
Hi Dubravko,

Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not
others.

For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical
CPU number from ACPI table.

Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that
it finds 0 pcpus.

BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err()
messages.
https://github.com/projectacrn/acrn-hypervisor/pull/4381

Best Regards,
Zide



Re: Getting ACRN to work

Chen, Zide
 

Hi Dubravko,

Did you boot ACRN from acrn.efi (UEFI) or acrn.bin (SBL), or acrn.32.out (Grub)? RELOC is enabled for acrn.efi, but not others.

For the init_percpu_lapic_init() issue, it's highly possibly that the user defined MAX_PCPU_NUM is less than the physical CPU number from ACPI table.

Other than that, probably you need to debug the parse_madt() and see if it can't find MADT table or corrupted MADT so that it finds 0 pcpus.

BTW, you can pull this PR to your code which may help your bring-up since it enables early panic() and other pr_err() messages.
https://github.com/projectacrn/acrn-hypervisor/pull/4381

Best Regards,
Zide

On 2/10/20 5:13 AM, Dubravko Moravski | Exor Embedded S.r.l. wrote:
Hi Zide,
Thank you very much, you were right. I've had RELOC already enabled, but apparently one also need to have CONFIG_HV_RAM_START 2MB aligned. I've set it to 0x1400 0000 and now my hypervisor manages to execute entire init_paging() function.
... but now it crashes in cpu.c, init_percpu_lapic_id(). A block of code from init_pcpu_pre():
printf("aaaaaaa7\n");
if (!init_percpu_lapic_id()) {
panic("failed to init_percpu_lapic_id!");
}
printf("aaaaaaa8\n");
"7" gets printed out, neither "8" nor panic does.
I guess something is wrong with the interrupts? If it means anything, I've made no particular modifications related to interrupts, and our board generally works well with Linux.
Best regards,
Dubravko
*Dubravko Moravski*
/SW engineering/
*Exor Embedded S.r.l.*
p: +38 512455659 <tel:+38 512455659>  m:+38 5915402413 <tel:+38 5915402413>
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com <https://exorint.com/>

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.
Privacy <https://www.exorint.com/it/privacy>
--------------------------------------------------------------------------------------------------------------------------
*From:* acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
*Sent:* Saturday, February 8, 2020 6:33 AM
*To:* acrn-users@... <acrn-users@...>
*Subject:* Re: [acrn-users] Getting ACRN to work
Hi Dubravko,
Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned.
Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue.
In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820() in misc/efi-stub/boot.c, before emalloc_reserved_aligned().
Best Regards,
Zide


Re: Getting ACRN to work

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hi Zide,

Thank you very much, you were right. I've had RELOC already enabled, but apparently one also need to have CONFIG_HV_RAM_START 2MB aligned. I've set it to 0x1400 0000 and now my hypervisor manages to execute entire init_paging() function.

... but now it crashes in cpu.c, init_percpu_lapic_id(). A block of code from init_pcpu_pre():

printf("aaaaaaa7\n");

if (!init_percpu_lapic_id()) {
panic("failed to init_percpu_lapic_id!");
}
printf("aaaaaaa8\n");

"7" gets printed out, neither "8" nor panic does.

I guess something is wrong with the interrupts? If it means anything, I've made no particular modifications related to interrupts, and our board generally works well with Linux.

Best regards,
Dubravko


Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy
From: acrn-users@... <acrn-users@...> on behalf of Chen, Zide via Lists.Projectacrn.Org <zide.chen=intel.com@...>
Sent: Saturday, February 8, 2020 6:33 AM
To: acrn-users@... <acrn-users@...>
Subject: Re: [acrn-users] Getting ACRN to work
 

Hi Dubravko,

 

Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned. 

Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue. 

In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820() in misc/efi-stub/boot.c, before emalloc_reserved_aligned().

 

Best Regards,

Zide


Re: Getting ACRN to work

Chen, Zide
 

Hi Dubravko,

 

Host MMU page size is 2MB in ACRN hypervisor. It seems to me that the direct cause for crash could be CONFIG_HV_RAM_START(0x13151000) is not 2MB aligned. 

Did you disable relocation (RELOC in Kconfig)? By default RELOC is enabled and it allocates memory from EFI and overwrites the user defined CONFIG_HV_RAM_START, which is supposed to be able to avoid the 2MB alignment issue. 

In case of you prefer to disable hypervisor relocation, you may need to make sure that CONFIG_HV_RAM_START is configured to the memory that is of EfiConventionalMemory(7) type. You may find out the available memory by calling dump_e820() in misc/efi-stub/boot.c, before emalloc_reserved_aligned().

 

Best Regards,

Zide


Getting ACRN to work

Dubravko Moravski | Exor Embedded S.r.l. <dubravko.moravski@...>
 

Hello,

My schedule finally allows me to continue working on getting ACRN to work on our board. I have downloaded the newest version few days ago, added a configuration for our board, used the python/web tool to generate code patches, menuconfig, and built acrn.efi. Everything went well so far. Unfortunately the hypervisor crashes quite early, but at least I've got debug prints to work.

Here is a part from mmu.c from hypervisor/arch/x86/, debug printouts added by me:

/*
* remove 'NX' bit for pages that contain hv code section, as by default XD bit is set for
* all pages, including pages for guests.
*/
printf("paging 6\n");
mmu_modify_or_del((uint64_t *)ppt_mmu_pml4_addr, round_pde_down(hv_hpa),
round_pde_up(text_end) - round_pde_down(hv_hpa), 0UL,
PAGE_NX, &ppt_mem_ops, MR_MODIFY);

mmu_modify_or_del((uint64_t *)ppt_mmu_pml4_addr, (uint64_t)get_reserve_sworld_memory_base(),
TRUSTY_RAM_SIZE * (CONFIG_MAX_VM_NUM - 1U), PAGE_USER, 0UL, &ppt_mem_ops, MR_MODIFY);

printf("paging 7\n");
/* Enable paging */
enable_paging();
"paging 6" gets printed out, "7" doesn't. That's the last sign of life from ACRN, nothing happens afterwards. What should I do to make it work?

My board has 8 GB of RAM, verified with multiple passes of MemTest to work correctly. CPU is Apollo Lake E3940. Some configuration parameters from menuconfig that might be of relevance:
  • LOW_RAM_SIZE = 0x10000 (default value)
  • HV_RAM_START = 0x13151000 (default value)
  • HV_RAM_SIZE = 0x7800000 (default value)
  • PLATFORM_RAM_SIZE = 0x200000000 (8 GB)
  • SOS_RAM_SIZE = 0x200000000 (8 GB)
  • UOS_RAM_SIZE = 0x100000000 (4 GB)
I suspect, since it seems to crash while adjusting memory parameters, that some of the parameter values above are bad. I think all the initial values were set for some 16 or 32 GB board, so perhaps I didn't adjust everything that I should have? Unfortunately I'm not familiar with inner workings of x86 CPUs at this low level.

Best regards,
Dubravko Moravski



Dubravko Moravski
SW engineering
Exor Embedded S.r.l.
p: +38 512455659  m: +38 5915402413
a: Slavonska avenija, 50, Zagreb, Croatia, 10000
w: exorint.com 

 Prima di stampare pensa ai costi ambientali. Please consider the environment before printing this email.

Privacy


Canceled: 2020 ACRN Project Technical Community Meeting (2020/1~2020/7): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

<Detailed calendar will be published 1 week before meeting>
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
  • Meets every Wednesday, Starting Nov 07, 2018: 11AM-12AM (China-Shanghai), 7PM-8PM (US-West Coast), 3AM-4AM (Europe-London)
  • Chairperson: Hongbo Wang, hongbo.wang@... (Intel)
  • Online conference link: https://zoom.us/j/457171121
  • Zoom Meeting ID: 457-171-121
  • Online conference phone:
  • US: +1 669 900 6833  or +1 646 558 8656   or +1 877 369 0926 (Toll Free) or +1 855 880 1246 (Toll Free)
  • China: +86 010 87833177  or 400 669 9381 (Toll Free)
  • Germany: +49 (0) 30 3080 6188  or +49 800 724 3138 (Toll Free)
  • Additional international phone numbers
  • Meeting Notes:
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 
 
 


Re: Zephyr as UOS slows down when SOS is busy

Yan, Like
 

Hi Alfonso,

 

#1 Did you calibrate the zephyr clock rate? I am not an expert on Zephyr, as I known, clock calibration is needed in Zephyr.

 

#2 Echo to Geoffroy,  heavy workloads on SOS may have significant impact on guest VM if no CAT to do LLC isolation.

 

BRs,

Like

 

From: acrn-users@... <acrn-users@...> On Behalf Of Alfonso Sanchez-Beato
Sent: Tuesday, January 21, 2020 8:08 PM
To: acrn-users@...
Subject: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

Hello,

 

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

 

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

 

Although the problem runs and I get output from the serial port, I found two problems:

 

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

 

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

 

ACRN version is:

HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

 

Any ideas on what might be going on?

 

Thanks

Alfonso

 


Re: Zephyr as UOS slows down when SOS is busy

Alfonso Sanchez-Beato
 



On Wed, Jan 22, 2020 at 2:28 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:
Hi Geoffroy,

On Wed, Jan 22, 2020 at 9:32 AM Geoffroy Van Cutsem <geoffroy.vancutsem@...> wrote:

Hi Alfonso,

 

I’m assuming you have been using the standard launch script (from /usr/share/acrn/samples/nuc/launch_zephyr.sh) to test your Zephyr app. If so, can you try to use this script instead to see if it makes any difference? https://paste.ubuntu.com/p/rCjfYv3QV4/

 

I have not tried it yet with your app but it’s able to launch the Zephyr “Hello World! Acrn” app.


I have tried with the new script, and I think it helps a bit, although it
does not remove completely the increment in time when something is running
in the SOS. But it is a bit difficult to say tbh.

It turns out that disabling Intel Turbo Boost in the BIOS makes the times
spent in the calculation on Zephyr almost constant regardless of the load
on the SOS. Which makes sense after reading that Turbo Boost changes
dynamically the CPU frequency.

Of course, the drawback is that now the calculations take more time.
 
 

 

Thanks,

Geoffroy


Thanks
Alfonso
 

 

 

From: VanCutsem, Geoffroy
Sent: Tuesday, January 21, 2020 4:18 PM
To: acrn-users@...
Subject: RE: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

Hi Alfonso,

 

My best guess as to what’s happening is that there is some contention between the Service VM and the Zephyr VM to use the cache, Zephyr would typically stay in cache but now that there is some heavy load in the Service VM, it may get evicted at times. This is where Cache Allocation Technology (CAT) can help further isolate the VMs, especially against so-called “noisy neighbours. Unfortunately, Kaby Lake does not support CAT. Here is more about CAT and how to use it with ACRN: https://projectacrn.github.io/latest/getting-started/rt_industry.html?#configure-cat

 

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Alfonso Sanchez-Beato
Sent: Tuesday, January 21, 2020 1:09 PM
To: acrn-users@...
Subject: Re: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

 

 

On Tue, Jan 21, 2020 at 1:07 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:

Hello,

 

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

 

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

 

Although the problem runs and I get output from the serial port, I found two problems:

 

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

 

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

 

ACRN version is:

HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

 

Any ideas on what might be going on?

 

I forgot to mention that I am testing on a NUC7i7DNH.

 

 

Thanks

Alfonso

 


Re: Zephyr as UOS slows down when SOS is busy

Alfonso Sanchez-Beato
 

Hi Geoffroy,

On Wed, Jan 22, 2020 at 9:32 AM Geoffroy Van Cutsem <geoffroy.vancutsem@...> wrote:

Hi Alfonso,

 

I’m assuming you have been using the standard launch script (from /usr/share/acrn/samples/nuc/launch_zephyr.sh) to test your Zephyr app. If so, can you try to use this script instead to see if it makes any difference? https://paste.ubuntu.com/p/rCjfYv3QV4/

 

I have not tried it yet with your app but it’s able to launch the Zephyr “Hello World! Acrn” app.


I have tried with the new script, and I think it helps a bit, although it
does not remove completely the increment in time when something is running
in the SOS. But it is a bit difficult to say tbh.
 

 

Thanks,

Geoffroy


Thanks
Alfonso
 

 

 

From: VanCutsem, Geoffroy
Sent: Tuesday, January 21, 2020 4:18 PM
To: acrn-users@...
Subject: RE: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

Hi Alfonso,

 

My best guess as to what’s happening is that there is some contention between the Service VM and the Zephyr VM to use the cache, Zephyr would typically stay in cache but now that there is some heavy load in the Service VM, it may get evicted at times. This is where Cache Allocation Technology (CAT) can help further isolate the VMs, especially against so-called “noisy neighbours. Unfortunately, Kaby Lake does not support CAT. Here is more about CAT and how to use it with ACRN: https://projectacrn.github.io/latest/getting-started/rt_industry.html?#configure-cat

 

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Alfonso Sanchez-Beato
Sent: Tuesday, January 21, 2020 1:09 PM
To: acrn-users@...
Subject: Re: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

 

 

On Tue, Jan 21, 2020 at 1:07 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:

Hello,

 

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

 

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

 

Although the problem runs and I get output from the serial port, I found two problems:

 

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

 

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

 

ACRN version is:

HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

 

Any ideas on what might be going on?

 

I forgot to mention that I am testing on a NUC7i7DNH.

 

 

Thanks

Alfonso

 


Re: Zephyr as UOS slows down when SOS is busy

Geoffroy Van Cutsem
 

Hi Alfonso,

 

I’m assuming you have been using the standard launch script (from /usr/share/acrn/samples/nuc/launch_zephyr.sh) to test your Zephyr app. If so, can you try to use this script instead to see if it makes any difference? https://paste.ubuntu.com/p/rCjfYv3QV4/

 

I have not tried it yet with your app but it’s able to launch the Zephyr “Hello World! Acrn” app.

 

Thanks,

Geoffroy

 

 

From: VanCutsem, Geoffroy
Sent: Tuesday, January 21, 2020 4:18 PM
To: acrn-users@...
Subject: RE: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

Hi Alfonso,

 

My best guess as to what’s happening is that there is some contention between the Service VM and the Zephyr VM to use the cache, Zephyr would typically stay in cache but now that there is some heavy load in the Service VM, it may get evicted at times. This is where Cache Allocation Technology (CAT) can help further isolate the VMs, especially against so-called “noisy neighbours. Unfortunately, Kaby Lake does not support CAT. Here is more about CAT and how to use it with ACRN: https://projectacrn.github.io/latest/getting-started/rt_industry.html?#configure-cat

 

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Alfonso Sanchez-Beato
Sent: Tuesday, January 21, 2020 1:09 PM
To: acrn-users@...
Subject: Re: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

 

 

On Tue, Jan 21, 2020 at 1:07 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:

Hello,

 

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

 

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

 

Although the problem runs and I get output from the serial port, I found two problems:

 

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

 

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

 

ACRN version is:

HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

 

Any ideas on what might be going on?

 

I forgot to mention that I am testing on a NUC7i7DNH.

 

 

Thanks

Alfonso

 


Re: Zephyr as UOS slows down when SOS is busy

Geoffroy Van Cutsem
 

Hi Alfonso,

 

My best guess as to what’s happening is that there is some contention between the Service VM and the Zephyr VM to use the cache, Zephyr would typically stay in cache but now that there is some heavy load in the Service VM, it may get evicted at times. This is where Cache Allocation Technology (CAT) can help further isolate the VMs, especially against so-called “noisy neighbours. Unfortunately, Kaby Lake does not support CAT. Here is more about CAT and how to use it with ACRN: https://projectacrn.github.io/latest/getting-started/rt_industry.html?#configure-cat

 

Geoffroy

 

From: acrn-users@... <acrn-users@...> On Behalf Of Alfonso Sanchez-Beato
Sent: Tuesday, January 21, 2020 1:09 PM
To: acrn-users@...
Subject: Re: [acrn-users] Zephyr as UOS slows down when SOS is busy

 

 

 

On Tue, Jan 21, 2020 at 1:07 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:

Hello,

 

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

 

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

 

Although the problem runs and I get output from the serial port, I found two problems:

 

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

 

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

 

ACRN version is:

HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

 

Any ideas on what might be going on?

 

I forgot to mention that I am testing on a NUC7i7DNH.

 

 

Thanks

Alfonso

 


Re: Zephyr as UOS slows down when SOS is busy

Alfonso Sanchez-Beato
 



On Tue, Jan 21, 2020 at 1:07 PM Alfonso Sanchez-Beato <alfonso.sanchez-beato@...> wrote:
Hello,

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

Although the problem runs and I get output from the serial port, I found two problems:

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

ACRN version is:
HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

Any ideas on what might be going on?

I forgot to mention that I am testing on a NUC7i7DNH.
 

Thanks
Alfonso


Zephyr as UOS slows down when SOS is busy

Alfonso Sanchez-Beato
 

Hello,

I have been playing with using Zephyr as UOS on top of Ubuntu as SOS, using the Industry scenario and launching the UOS with launch_zephyr.sh.

I created this small program to prove that Zephyr was not affected by what happened in the SOS: https://paste.ubuntu.com/p/c3j9XwvrT6/ (with config https://paste.ubuntu.com/p/sPhwTRrRTZ/). The program simply calculates the number of primes below a number and prints the time spent.

Although the problem runs and I get output from the serial port, I found two problems:

1. gettimeofday is not returning good times, the elapsed time is ~20 times more than the real one. I see the same problem when using k_uptime_get and k_uptime_delta. It looks related to handling of the HW clock, but the pre-defined frequency (100) seems correct.

2. More importantly, I see that the time spent in the calculation varies when I do something CPU intensive on the SOS. For instance, if I run the same calculation on the SOS, I see that Zephyr is expending 5% more time in the calculation, which should not happen and defeats the purpose of this set-up.

ACRN version is:
HV version 1.4-unstable-2020-01-09 09:45:06-bcefd673 DBG (daily tag: acrn-2019w47.1-140000p) build by ubuntu
API version 1.0

Any ideas on what might be going on?

Thanks
Alfonso


Canceled: 2020 ACRN Project Technical Community Meeting (2020/1~2020/7): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

Cancel meeting this week. Happy Chinese New Year Holiday~
 
<Detailed calendar will be published 1 week before meeting>
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
  • Meets every Wednesday, Starting Nov 07, 2018: 11AM-12AM (China-Shanghai), 7PM-8PM (US-West Coast), 3AM-4AM (Europe-London)
  • Chairperson: Hongbo Wang, hongbo.wang@... (Intel)
  • Online conference link: https://zoom.us/j/457171121
  • Zoom Meeting ID: 457-171-121
  • Online conference phone:
  • US: +1 669 900 6833  or +1 646 558 8656   or +1 877 369 0926 (Toll Free) or +1 855 880 1246 (Toll Free)
  • China: +86 010 87833177  or 400 669 9381 (Toll Free)
  • Germany: +49 (0) 30 3080 6188  or +49 800 724 3138 (Toll Free)
  • Additional international phone numbers
  • Meeting Notes:
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 
 
 


Canceled: 2020 ACRN Project Technical Community Meeting (2020/1~2020/7): @ Weekly Wednesday 11AM (China-Shanghai), Tuesday 7PM (US-West Coast), Wednesday 3AM (Europe-London)

Wang, Hongbo
 

Cancel meeting this week. Happy Chinese New Year Holiday~
 
<Detailed calendar will be published 1 week before meeting>
 
 
 
Project ACRN: A flexible, light-weight, open source reference hypervisor for IoT devices
 
We're still in the early stages of forming this TSC, so instead we invite you to attend a weekly "Technical Community" meeting where we'll meet community members and talk about the ACRN project and plans. As we explore community interest and involvement opportunities, we'll (re)schedule these meetings at a time convenient to most attendees:
  • Meets every Wednesday, Starting Nov 07, 2018: 11AM-12AM (China-Shanghai), 7PM-8PM (US-West Coast), 3AM-4AM (Europe-London)
  • Chairperson: Hongbo Wang, hongbo.wang@... (Intel)
  • Online conference link: https://zoom.us/j/457171121
  • Zoom Meeting ID: 457-171-121
  • Online conference phone:
  • US: +1 669 900 6833  or +1 646 558 8656   or +1 877 369 0926 (Toll Free) or +1 855 880 1246 (Toll Free)
  • China: +86 010 87833177  or 400 669 9381 (Toll Free)
  • Germany: +49 (0) 30 3080 6188  or +49 800 724 3138 (Toll Free)
  • Additional international phone numbers
  • Meeting Notes:
 
 
Best regards.
Hongbo
Tel: +86-21-6116 7445
MP: +86-1364 1793 689
 
 
 
 
 
 

701 - 720 of 1228