DO188 - ch03s02 Guided Exercise Lab - getting "tls: failed to verify certificate" error message

[ric]

Hi, everyone!

I'm now doing the "Section 3.2: Guided Exercise: Container Image Registries" Lab for ch03s02 - https://rol.redhat.com/rol/app/courses/do188-4.14/pages/ch03s02 - of the DO188 course (for the course version based on OpenShift Container Platform 4.14).

Everything seems to work well for all the "substeps" of the first step:

[student@workstation ~]$ lab start images-basics
SUCCESS Verifying cluster state
SUCCESS Verifying if Podman is installed
SUCCESS Verifying your OpenShift API URL

[student@workstation ~]$ oc login -u admin -p redhatocp https://api.ocp4.example.com:6443
Login successful.

You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects'

Using project "default".
Welcome! See 'oc help' to get started.

[student@workstation ~]$ podman login -u $(oc whoami) -p $(oc whoami -t) default-route-openshift-image-registry.apps.ocp4.example.com
Login Succeeded!

[student@workstation ~]$ podman login -u developer -p developer registry.ocp4.example.com:8443
Login Succeeded!


[student@workstation ~]$ RHOCP_REGISTRY="default-route-openshift-image.registry.apps.ocp4.example.com"

[student@workstation ~]$ ping $RHOCP_REGISTRY
PING default-route-openshift-image.registry.apps.ocp4.example.com (192.168.50.254) 56(84) bytes of data.
64 bytes from 192.168.50.254 (192.168.50.254): icmp_seq=1 ttl=64 time=0.553 ms
(...)

BUT then, step 2 fails with following error message:

[student@workstation ~]$ skopeo copy --dest-tls-verify=false docker://${RHOCP_REGISTRY}/default/python:3.9-ubi8 docker://registry.ocp4.example.com/default/python:3.9-ubi8
FATA[0000] initializing source docker://default-route-openshift-image.registry.apps.ocp4.example.com/default/python:3.9-ubi8: pinging container registry default-route-openshift-image.registry.apps.ocp4.example.com: Get "https://default-route-openshift-image.registry.apps.ocp4.example.com/v2/": tls: failed to verify certificate: x509: certificate is valid for api.ocp4.example.com, *.apps.ocp4.example.com, not default-route-openshift-image.registry.apps.ocp4.example.com 

As it seemed to me that this error was referring to the source registry and NOT to the destination registry, I tried to add  --src-tls-verify=false to the skopeo copy command, but then I get a "received unexpected HTTP status: 503 Service Unavailable" error message:

[student@workstation ~]$ skopeo copy --src-tls-verify=false --dest-tls-verify=false docker://${RHOCP_REGISTRY}/default/python:3.9-ubi8 docker://registry.ocp4.example.com/default/python:3.9-ubi8
FATA[0000] initializing source docker://default-route-openshift-image.registry.apps.ocp4.example.com/default/python:3.9-ubi8: pinging container registry default-route-openshift-image.registry.apps.ocp4.example.com: received unexpected HTTP status: 503 Service Unavailable 

Am I doing something wrong or is there a problem with this exercise?

Thanks in advance!

[ARoumiantsev · Accepted solution]

Hi@ric 

Please be accurate,  It is typo here: 

RHOCP_REGISTRY="default-route-openshift-image.registry.apps.ocp4.example.com"

 Should be 

RHOCP_REGISTRY="default-route-openshift-image-registry.apps.ocp4.example.com"

Good luck

[ric · Accepted solution]

Hi, @ARoumiantsev 

Thank you very much! You're quite right: I had a typo in the FQDN in the RHOCP_REGISTRY value, namely I had a dot between image and registry instead of an hyphen. I've now corrected that, but now I'm getting a different error message when doing the skopeo copy command (even after logging out from oc and podman and logging back in), namely the error message "FATA[0000] trying to reuse blob sha256:de1a4526c3d6cdf0e04f24b1888f5ef31425209f1c26e5a6ae7694cdad6e8688 at destination: Requesting bearer token: invalid status code from registry 403 (Forbidden)":

[student@workstation ~]$ oc logout
Logged "admin" out on "https://api.ocp4.example.com:6443"

[student@workstation ~]$ echo $RHOCP_REGISTRY

default-route-openshift-image-registry.apps.ocp4.example.com

[student@workstation ~]$ podman logout $RHOCP_REGISTRY 

Removed login credentials for default-route-openshift-image-registry.apps.ocp4.example.com

[student@workstation ~]$ podman logout registry.ocp4.example.com

Error: not logged into registry.ocp4.example.com

[student@workstation ~]$ oc login -u admin -p redhatocp https://api.ocp4.example.com:6443

Login successful.

You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects'

Using project "default".

[student@workstation ~]$ podman login -u $(oc whoami) -p $(oc whoami -t) $RHOCP_REGISTRY

Login Succeeded!

[student@workstation ~]$ podman login -u developer -p developer registry.ocp4.example.com:8443

Login Succeeded!

[student@workstation ~]$ skopeo copy --dest-tls-verify=false docker://${RHOCP_REGISTRY}/default/python:3.9-ubi8 docker://registry.ocp4.example.com/default/python:3.9-ubi8

Getting image source signatures

FATA[0000] trying to reuse blob sha256:de1a4526c3d6cdf0e04f24b1888f5ef31425209f1c26e5a6ae7694cdad6e8688 at destination: Requesting bearer token: invalid status code from registry 403 (Forbidden) 

Could you (or anyone) please tell me if is there a way to troubleshoot / solve this other error? Thanks again!

EDIT: Never mind! Now the problem was that I was missing the :8443 port in the destination registry. It's now working, with the following command and output:

[student@workstation ~]$ skopeo copy --dest-tls-verify=false docker://${RHOCP_REGISTRY}/default/python:3.9-ubi8 docker://registry.ocp4.example.com:8443/developer/python:3.9-ubi8
Getting image source signatures
Copying blob dbefd0ecfdfe done
Copying blob 0c7a1eac6f18 skipped: already exists
Copying blob d921314675de skipped: already exists
Copying blob de1a4526c3d6 skipped: already exists
Copying config c69930c6f7 done
Writing manifest to image destination
Storing signatures

[Chetan_Tiwary_]

@ric glad that you  figured it out yourself. Kudos!

[Chetan_Tiwary_]

@ARoumiantsev spot on!

@ric refer the screenshot :

[ric]

Right. Problem solved. Thank you very much, @ARoumiantsev and @Chetan_Tiwary_ !

[Chetan_Tiwary_]

@ric wonderful !