Warm Migration from VMware to O-Virt

[odurasler]

Hello All! First post here.

I just stood up my first OCP cluster (4.20) and have configured MTV/VDDK.  I was able to successfully create a cold migration plan which completed with no issues. However, when i created a new "warm" migration plan, it failed in that there was "no route to host" even though ports 902 and 443 were allowed. Both vCenter and ESXi are on different networks. While I can curl to vCenter, I can't on the ESXi networks. Because I can easily add a NIC on my nodes (Cisco UCS), I'm thinking on adding a NIC that is configured on the same network as the ESXi network. So here are my questions:

1. Would creating a new NNCP (ethernet) for the new NIC and put it on the same network as the ESXi network, solve my issue?
2. Is NNCP enough or do I need to creat a NAD for the openshift-mtv to use?
3. Is there a better way to attack this or a simple way?

Thanks for your time.

odurasler

[Sunnykumar1]

Hello @odurasler 

1. Adding a NIC on the same network as the ESXi hosts can definitely help resolve that "no route to host" issue. This ensures that the vCenter and the ESXi hosts can communicate over the same network segment, and that should help the warm migration proceed smoothly. So, in most cases, that should solve the connectivity problem.

2. Typically, an NNCP can handle the networking needs for OpenShift, especially when it's set up correctly. However, in some scenarios, creating a NAD can give you more flexibility, like defining custom network configurations or supporting multiple networks.

3. Usually, the best approach really depends on your environment and the complexity you're dealing with. If you're looking for simplicity and quick deployment, sticking with the default NCP configuration and ensuring your networking is aligned might be the easiest route. But if you want more advanced features or fine-tuned networking, then exploring NADs or even other network plugins could be worthwhile. It’s really about balancing simplicity with the level of control you need.

Thank you!

[Chetan_Tiwary_]

@odurasler Thanks for reaching out and great job passing the first post barrier!

For warm migrations, MTV relies on VMware's Changed Block Tracking (CBT).

vCenter handles the metadata coordination over port 443, but the MTV pods pull the actual VM data blocks directly from ESXi hosts using VDDK over port 902, and that's why you need direct ESXi connectivity. Adding a physical NIC with a NNCP to have an IP on the interface for the ESXi network is definitely the solution to the your network isolation issue as evident from "no route to host" error.

https://knowledge.broadcom.com/external/article?legacyId=1020128 

https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.0/html/installing_an... 

NNCP is the physical/host layer  whereas the NAD is the container layer. 

If you only had NNCP, the node would have a connection to ESXi, but the migration software wouldn't be allowed to touch it. If you only had a NAD, the migration software would be looking for a door that the operating system hasn't built yet.

By using both, you ensure the migration data travels over your dedicated high-speed link rather than getting throttled by the standard pod network.

So, technically you need both of them.

Simpler way could be if your existing OpenShift MC or "default" network has a gateway/router that can reach the ESXi network which will prevent the need of adding extra NICs. However, the better way actually is to add dedicated NIC for performance and security. 

[Priyanshu20041]

Hello @odurasler 
Your thinking is on the right track. For warm migrations, MTV needs direct, continuous connectivity to the ESXi hosts, not just vCenter. Cold migrations rely more heavily on vCenter APIs, which is why those can succeed even when ESXi networks aren’t reachable.

To your questions:

1. Will adding a NIC and creating an NNCP help?
Yes—if that NIC places the OpenShift nodes on the same routable network as the ESXi hosts, this will resolve the “no route to host” issue. Warm migrations require direct ESXi access (ports 443/902).

2. Is NNCP enough, or is a NAD required?

An NNCP is sufficient. MTV uses the node network, not a pod-attached network, so a NAD is not required for this use case.

3. Is there a better or simpler approach?

The cleanest solution is to ensure L3 routing between the OpenShift node network and the ESXi management network. If routing can’t be added, attaching a dedicated NIC/VLAN to the nodes (as you suggested) is a valid and commonly used approach.