How to get the 2nd vCPU running in the Industry scenario?


Tzeng, Tonny <tonny.tzeng@...>
 

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Victor Sun
 

Hi Tonny,

This is a known issue. In current implementation, SOS will offline all vcpus except BSP before launch any user VM(i.e. offline pCPU1/2/3 in your case), and your launched user VM2 only init pCPU2/3 to run, this cause pCPU1 always be offlined until you launch user VM1.

In our new implementation, acrn-config tool will do a modification in launch script, that only pCPUs belong to the launched VM could be offlined, then it will resolve your issue.

or you can modify the launch script by manual:

  for i in `ls -d /sys/devices/system/cpu/cpu[1-99]`; do

+ # do not offline the CPU which not belong to the UOS

+ if [ "i" = "/sys/devices/system/cpu/cpu1" ]; then continue; fi

        online=`cat $i/online`

BR,
Victor

On 11/27/2019 5:15 PM, Tzeng, Tonny wrote:

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Liu, Fuzhong
 

Hi Tonny

About the secondary vCPU is in “Init” mode after I launch  user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

I think you add maxcpus=1 in your kernel comline; Could you please share the user VM kernel comline?

 

ACRN:\>vcpu_list(without maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Running

 

ACRN:\>vcpu_list(with maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Victor Sun
Sent: Wednesday, November 27, 2019 8:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny,

This is a known issue. In current implementation, SOS will offline all vcpus except BSP before launch any user VM(i.e. offline pCPU1/2/3 in your case), and your launched user VM2 only init pCPU2/3 to run, this cause pCPU1 always be offlined until you launch user VM1.

In our new implementation, acrn-config tool will do a modification in launch script, that only pCPUs belong to the launched VM could be offlined, then it will resolve your issue.

or you can modify the launch script by manual:

  for i in `ls -d /sys/devices/system/cpu/cpu[1-99]`; do

+ # do not offline the CPU which not belong to the UOS

+ if [ "i" = "/sys/devices/system/cpu/cpu1" ]; then continue; fi

        online=`cat $i/online`

BR,

Victor

 

On 11/27/2019 5:15 PM, Tzeng, Tonny wrote:

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Tzeng, Tonny <tonny.tzeng@...>
 

Hi Fuzhong,

 

You are right! I just copied this comment line to my kernel command line without noticing it limits the vCPU to 1. Both vCPU are running now, though you may consider to remove this misleading parameter in the sample launch script. 😊

 

Regards,

Tonny

 

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Thursday, November 28, 2019 9:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny

About the secondary vCPU is in “Init” mode after I launch  user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

I think you add maxcpus=1 in your kernel comline; Could you please share the user VM kernel comline?

 

ACRN:\>vcpu_list(without maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Running

 

ACRN:\>vcpu_list(with maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Victor Sun
Sent: Wednesday, November 27, 2019 8:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny,

This is a known issue. In current implementation, SOS will offline all vcpus except BSP before launch any user VM(i.e. offline pCPU1/2/3 in your case), and your launched user VM2 only init pCPU2/3 to run, this cause pCPU1 always be offlined until you launch user VM1.

In our new implementation, acrn-config tool will do a modification in launch script, that only pCPUs belong to the launched VM could be offlined, then it will resolve your issue.

or you can modify the launch script by manual:

  for i in `ls -d /sys/devices/system/cpu/cpu[1-99]`; do

+ # do not offline the CPU which not belong to the UOS

+ if [ "i" = "/sys/devices/system/cpu/cpu1" ]; then continue; fi

        online=`cat $i/online`

BR,

Victor

 

On 11/27/2019 5:15 PM, Tzeng, Tonny wrote:

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Victor Sun
 

Thanks Fuzhong to correct my misunderstanding!

 

BR,

Victor

 

From: acrn-users@... <acrn-users@...> On Behalf Of Tzeng, Tonny
Sent: Thursday, November 28, 2019 9:21 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Fuzhong,

 

You are right! I just copied this comment line to my kernel command line without noticing it limits the vCPU to 1. Both vCPU are running now, though you may consider to remove this misleading parameter in the sample launch script. 😊

 

Regards,

Tonny

 

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Thursday, November 28, 2019 9:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny

About the secondary vCPU is in “Init” mode after I launch  user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

I think you add maxcpus=1 in your kernel comline; Could you please share the user VM kernel comline?

 

ACRN:\>vcpu_list(without maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Running

 

ACRN:\>vcpu_list(with maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Victor Sun
Sent: Wednesday, November 27, 2019 8:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny,

This is a known issue. In current implementation, SOS will offline all vcpus except BSP before launch any user VM(i.e. offline pCPU1/2/3 in your case), and your launched user VM2 only init pCPU2/3 to run, this cause pCPU1 always be offlined until you launch user VM1.

In our new implementation, acrn-config tool will do a modification in launch script, that only pCPUs belong to the launched VM could be offlined, then it will resolve your issue.

or you can modify the launch script by manual:

  for i in `ls -d /sys/devices/system/cpu/cpu[1-99]`; do

+ # do not offline the CPU which not belong to the UOS

+ if [ "i" = "/sys/devices/system/cpu/cpu1" ]; then continue; fi

        online=`cat $i/online`

BR,

Victor

 

On 11/27/2019 5:15 PM, Tzeng, Tonny wrote:

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Liu, Fuzhong
 

Hi Tonny

You should copy  https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/samples/nuc/launch_hard_rt_vm.sh for RTVM; Not maxcpus parameter in it.

Not https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/samples/nuc/launch_uos.sh

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Tzeng, Tonny
Sent: Thursday, November 28, 2019 9:21 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Fuzhong,

 

You are right! I just copied this comment line to my kernel command line without noticing it limits the vCPU to 1. Both vCPU are running now, though you may consider to remove this misleading parameter in the sample launch script. 😊

 

Regards,

Tonny

 

From: acrn-users@... <acrn-users@...> On Behalf Of Liu, Fuzhong
Sent: Thursday, November 28, 2019 9:12 AM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny

About the secondary vCPU is in “Init” mode after I launch  user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

I think you add maxcpus=1 in your kernel comline; Could you please share the user VM kernel comline?

 

ACRN:\>vcpu_list(without maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Running

 

ACRN:\>vcpu_list(with maxcpus=1)

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

 

BR.

Fuzhong

From: acrn-users@... <acrn-users@...> On Behalf Of Victor Sun
Sent: Wednesday, November 27, 2019 8:17 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

 

Hi Tonny,

This is a known issue. In current implementation, SOS will offline all vcpus except BSP before launch any user VM(i.e. offline pCPU1/2/3 in your case), and your launched user VM2 only init pCPU2/3 to run, this cause pCPU1 always be offlined until you launch user VM1.

In our new implementation, acrn-config tool will do a modification in launch script, that only pCPUs belong to the launched VM could be offlined, then it will resolve your issue.

or you can modify the launch script by manual:

  for i in `ls -d /sys/devices/system/cpu/cpu[1-99]`; do

+ # do not offline the CPU which not belong to the UOS

+ if [ "i" = "/sys/devices/system/cpu/cpu1" ]; then continue; fi

        online=`cat $i/online`

BR,

Victor

 

On 11/27/2019 5:15 PM, Tzeng, Tonny wrote:

Hi,

 

It looks to me two cores are now assigned to the VM2 in the Industry scenario, but only the primary vCPU is shown in “Running” state, the secondary vCPU is in “Init” mode after I launch my user VM. The HV console indicates:

 

ACRN:\>vcpu_list

 

VM ID    PCPU ID    VCPU ID    VCPU ROLE    VCPU STATE

=====    =======    =======    =========    ==========

  0         0          0       PRIMARY      Running

  2         2          0       PRIMARY      Running

  2         3          1       SECONDARY    Init

 

Is it a bug or how can I move the secondary vCPU to “Running” mode? Thanks.

 

Regards,

Tonny


Geoffroy Van Cutsem
 

On Thu, 2019-11-28 at 02:04 +0000, Liu, Fuzhong wrote:
Hi Tonny
You should copy
https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/samples/nuc/launch_hard_rt_vm.sh
for RTVM; Not maxcpus parameter in it.
Not
https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/samples/nuc/launch_uos.sh
Is there any value at all in having this maxcpus parameter *anywhere*?
My thinking is that it's ACRN that determines how many vCPUs are
visible to any User VM, and we'd want our User VM to always use as many
as are available, regardless of whether this is an RTVM or a standard
User VM, wouldn't we?

Thanks,
Geoffroy


Victor Sun
 

Agree, we should avoid maxcpus setting.
Currently only SOS cmdline of hybrid scenario might need this setting as a workaround, to avoid enumerating all cpus when bootup APs.
When SOS MADT table could be customized, then this workaround is valueless also.

BR,
Victor

-----Original Message-----
From: acrn-users@... <acrn-users@...> On Behalf Of Geoffroy Van Cutsem
Sent: Thursday, November 28, 2019 3:23 PM
To: acrn-users@...
Subject: Re: [acrn-users] How to get the 2nd vCPU running in the Industry scenario?

On Thu, 2019-11-28 at 02:04 +0000, Liu, Fuzhong wrote:
Hi Tonny
You should copy
https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel
/samples/nuc/launch_hard_rt_vm.sh for RTVM; Not maxcpus parameter in
it.
Not
https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel
/samples/nuc/launch_uos.sh
Is there any value at all in having this maxcpus parameter *anywhere*?
My thinking is that it's ACRN that determines how many vCPUs are visible to any User VM, and we'd want our User VM to always use as many as are available, regardless of whether this is an RTVM or a standard User VM, wouldn't we?

Thanks,
Geoffroy