cancel
Showing results for 
Search instead for 
Did you mean: 
mehmetoren1
Cadet
Cadet
  • 677 Views

Re: RHA Labs as VMs?

My entire lab environment is on a VMware ESXi host. I've updated most of the student kiosk machines from RHEL80 to RHEL82 and CPU consumption is almost the same with the old version. Right now the CPU usage of each student kiosk is approximately 400MHz and all vms are up and running on the kiosks.

I can't directly talk about the VMware Workstation and AMD Ryzen 7 because the CPU on the host is an Intel Xeon but here are a few steps to check.

1. Verify that your hardware supports VT-d and that it has been enabled in the BIOS.

2. Make sure that you have enabled hardware assisted virtualization to the guest from the vm config. Search for an option something like that if there is: "Expose hardware assisted virtualization to the guest"

3. If there is an option, enable IOMMU in the vmconfig 

4. After installation of the kiosk system, add the following options to grub configuration, regenerate grub configuration and reboot. 

For AMD: iommu=pt iommu=1

For Intel: intel_iommu=on

5. As a final step to verify if everything is fine, run the following command in a terminal on the kiosk:

virt-host-validate

Actually if you haven't done yet, you can run "virt-host-validate" command as a first step to see what's going wrong Smiley Wink.

 

0 Kudos
Reply
Loading...
atopa
Cadet
Cadet
  • 660 Views

Re: RHA Labs as VMs?

Thanks for reply.

I tested more my system. I found reason for increased CPU and memory usage. There is new utility named VM which takes lot of CPU resources in starting. Later CPU usage drops but memory usage is over 40%.

Is this Utility named VM needed for RH124 or RH134 ? I haven't taught new RH134 yet.

I checked your checklist.

1. AMD-V and IOMMU are enabled in my BIOS setup (Asus ROG Strix B550-F motherboard with newest BIOS)

2. It is enabled

3. It is not available (Even my old Intel i7-2700k have IOMMU available).

iommu-missing.png

4. These options are not working because missing IOMMU support.

5. These warnings are listed:

QEMU: Checking for device assignment IOMMU support : WARN (No ACPI IVRS table found, IOMMU either disabled in BIOS or not supported by this hardware platform)
QEMU: Checking for secure guest support : WARN (Unknown if this platform has Secure Guest support)

 

 

0 Kudos
Reply
Loading...
mehmetoren
Mission Specialist
Mission Specialist
  • 649 Views

Re: RHA Labs as VMs?

It seems that you are on the right track. But it's really weird because as I remember in VMware workstation 15.5, there was an option for the IOMMU under processor settings. Actually vms can work without this option, but it causes some inconsistent behavior especially when booting or rebooting.

Anyway I haven't check RH134 yet, but in RH124 with the version 8.2 I didn't see any lab or practice related to utility vm. Maybe we need an update for the course documents as well.

0 Kudos
Reply
Loading...
Tracy_Baker
Starfighter Starfighter
Starfighter
  • 503 Views

Re: RHA Labs as VMs?

@atopa and @mehmetoren

Is this Utility named VM needed for RH124 or RH134 ? I haven't taught new RH134 yet.

utility is the VM used for the new containers chapter in the RH134 class. If I recall correctly, it is the container registry.

Program Lead at Arizona's first Red Hat Academy, est. 2005
Estrella Mountain Community College
Reply
Loading...
atopa
Cadet
Cadet
  • 631 Views

Re: RHA Labs as VMs?

I tested also Virtualbox with Windowss 10 and KVM/QEMU with Fedora 33.

Virtualbox and KVM/QEMU gave same warnings.

I entered following command in my Fedora 33 host.

dmesg | grep -i IOMMU

[ 0.407487] iommu: Default domain type: Translated
[ 0.612466] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.612504] pci 0000:00:01.0: Adding to iommu group 0

.

.

.

Until 21.

[ 0.615128] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.615885] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

 

I have enabled all IOMMU settings in BIOS settings which I have found in BIOS setup.

0 Kudos
Reply
Loading...
mehmetoren
Mission Specialist
Mission Specialist
  • 612 Views

Re: RHA Labs as VMs?

Sorry, I couldn't check my e-mail for a while. As I mentioned before vms can live without this option but with some inconsistent behaviors especially if you are using nested virtualization. But let's try to find a solution at least for the Fedora host.

You've already enabled VT-x and IOMMU in the BIOS and it seems that your hardware supports IOMMU. After that you need also to enable it in the host's kernel.

To do this, you need to add the iommu option to kernel parameters.

First edit /etc/default/grub

Here is the default entry for my Fedora workstation 33 installed laptop.

GRUB_CMDLINE_LINUX="rhgb quiet"

It should be:

GRUB_CMDLINE_LINUX="rhgb quiet amd_iommu=on iommu=pt"

For AMD, as I remember, iommu=on option also works. But first try the option given above.

Regenerate  grub config.

If your system is using UEFI:

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

If your system is using legacy boot:

grub2-mkconfig -o /boot/grub2/grub.cfg

These commands will regenerate grub config. After that you need to restart the host.

But you need to be very careful because any typo or miss entry will cause boot loops. 

If everything goes well, and you have your system up and running issue the virt-host-validate on Fedora host. If IOMMU is ok after the check, then you would be able to enable and expose IOMMU in the virtualization software's settings.

Just for fun and testing, I prepared 3 layered nested virtualization environment  with these settings and had no issue. 

BR.

0 Kudos
Reply
Loading...
atopa
Cadet
Cadet
  • 596 Views

Re: RHA Labs as VMs?

I tested adding parameters to FC33 host kernel without success. Command virt-host-validate and dmesg output was same as before parameters.

My FC33 host system have passed IOMMU-test.

QEMU: Checking if IOMMU is enabled by kernel : PASS

Foudation0 haven't passed IOMMU-test. I tested parameters also with Foundation0 without success.

I have enabled CPU host-passthrough in Virt Machine Manager.

 

0 Kudos
Reply
Loading...
mehmetoren
Mission Specialist
Mission Specialist
  • 484 Views

Re: RHA Labs as VMs?

Well, this is where nested virtualization starts. Your bare metal hypervisor (FC33) had passed the test with success. And now your virtualized hypervisor has to pass with success. Remember that, kiosk machine is also a hypervisor and that's why you need to apply those kernel settings for the kiosk as well. If those settings are not available in the grub config then virt-host-validate will throw warnings again.

0 Kudos
Reply
Loading...
atopa
Cadet
Cadet
  • 478 Views

Re: RHA Labs as VMs?

I have tested same parameters in Foundation0 and in my FC33 host computer.

0 Kudos
Reply
Loading...
mehmetoren
Mission Specialist
Mission Specialist
  • 470 Views

Re: RHA Labs as VMs?

Sorry that I couldn't reply your questions with a complete answer (but for me, it's hard to remember everything in a row, especially in these days), but as I remember also there is one more option to seek in BIOS settings.

Is there an option to enable SVM in the BIOS of your FC33 workstation? If yes, please enable it.

After that you can check it if it's enabled or not by issuing lscpu|grep svm.

For now that's all that I could suggest for you and maybe it's a hardware problem or virtualization software problem. I can't tell you more about what's going wrong because rest of the setup of my lab environment is same as yours except hardware and virtualization layer. 

0 Kudos
Reply
Loading...
Join the discussion
You must log in to join this conversation.