Lab: Troubleshooting Application Issues

[oblanco]

https://role.rhu.redhat.com/rol-rhu/app/courses/rh342-8.4/pages/ch08s09

The lab need to be corrected because if you run the grade script, it is going to give you an error like this

* Checking containerized application ...................... FAIL

despite the application is responding with the correct data. If you look at the answers, it is peformining an extra step which is exposing the port 3306. I don't think this is specified in the instructions and it is scary if the real exam is going to be like that 

[Travis]

@oblanco -

So exam instructions can be more specific at times, but often they can also be somewhat cryptic and make assumptions. So the solutions do show opening the port up to the database, but the scenario in the instructions gives clues that you will most likely need to open a port to the DataBase.

Another system administrator deployed the showview application, but it is not working.. The application uses a web service with a containerized database service. Resolve the issue and verify that the containerized service is working.

So there are web services as containers and databases as containers and they must talk to each other. This implies that there may need to be some network ports open. There might not even be steps 1-4 at a high level or those could be represented as substeps. What you need to do is figure out what is "needed" and test this.

So if it is substeps, then Step 3 about connecting and getting information from a database implies that the network port should be open. In order to test something like that you would need a client to see if you could retrieve information from a database as you might not know everything the application does, so you could use something like the MySQL client to see if you could connect and retrieve data. So when inspecting the container, you should see the password information and you should also see that a port is supposed to be exposed.

[Wasim_Raja]

+1 @Travis 

[oblanco]

Step 3 says "Resolve the issues that prevent the application from
obtaining the database information."

---> I assume that if the information is displaying correctly in the
screen without exposing the port from the database that's it , Why do
I need to open the port?
Again, You don't need to expose port 3306 in this lab and the command
# curl localhost/showviews.php displays the information correctly. In the exam, you will
assume that the information is displaying correctly and you will move
on to the next question but in reality, when the grade script run, it
will be marked as FAIL even if the information was displayed correctly
on the screen. this is scary

[Chetan_Tiwary_]

@oblanco I agree that     1. Instruction can be more clear    2. If we skip the port exposure - it should still give the error - given the port is not opened , and  3. The grading should also check if the curl is working fine or not apart from checking the port ( if both are basically the same thing - then how is the grading verifying that the  application is working ?) 

Also as @Travis mentioned - checking the port is part of basic application troubleshooting where we always check the ports, firewall, network connectivity to rule out the possible app not working issues - 3306 port  is the default port for the classic MySQL protocol. 

I have raised an internal ticket for this as explained in your other post on the same topic. 

[Travis]

@Chetan_Tiwary_ -

I'm not sure a ticket needs to be raised. I just went through it with the solutions and reading the questions carefully. I believe everything is functioning as it should. As this isn't a GE, the solution and process displayed just has to "work" which it does. The OCD pieces in me though would want to show the curl command and the show database (or lack thereof because you can't connect), and then once you fixed the MySQL container, show the curl again and the show database and that the data is different ... now what is broken ... then you look at the PHP app and see that it has an incorrect IP address it is attempting to pull data from and fix it, finally showing CURL again and SHOW DATABASE and the values match. 

The final step could then be adding one more item to the table and doing the curl/show database and show that the application is working correctly and the database is dynamically updating the information returned to the webapp.

[Travis]

@oblanco -

 In the exam, you will assume that the information is displaying correctly and you will move on to the next question but in reality, when the grade script run, it will be marked as FAIL even if the information was displayed correctly on the screen. this is scary.

That would be a very incorrect assumption. The question clearly states that the web application gets data from a containerized database. The CURL command you are running just verifies that the webserver is working and responding. It does not return any data from a database. In a real-world scenario if you had a web application getting data from a database, you would check that the web application is up and that the database is up and responding (providing the data to the web application). Another clue here, is that you are told there is a web application using a containerized database. 

There are multiple curl commands used to check and verify, but sometimes that confuses people, so I will show Firefox with the site ...

The first CURL command shows that the HTTPD (Apache webserver) is online and proving the PHP page, but notice, there is no data being returned from a database. You only have the headers that are builtin as part of the PHP file.

So I will concede that currently, there is something wrong with the PHP website as even with the database able to connect, it doesn't display and return the data. However, when looking at things it is clear that no data is coming from the container (database) and it is impossible for something external to connect to a database without the port being available.

The other part that is thrown in here as a curveball is troubleshooting the web application itself. Because even once the port has been exposed, things aren't working. The reason is that the webapp was looking at an incorrect IP address for the container. You need to also update the PHP file on the webapp.

Once the Webserver is up and running and the PHP application has been modified with the container port forwarding traffic to port 3306, the application can get the data that is needed. The database conainter is supposed to be providing data to other services, so a quick podman ps showing that the application doesn't have exposed ports lets you know that external applications cannot access data within the database. If the webserver would have been on ServerB (for example), there would have been no way for data to leave (granted at this point you would also need to play with firewalls and such o make it work).

Again, the solutions and steps are for one way. As for the grading script, it should fail right off on the first item because fixing HTTPD and Apache has nothing to do with the containerized application.

So not only does the script check that the database container is up, running, and accessible, it also then goes and looks to ensure the data is coming back. Being extremely OCD on checking things and in cases of a REAL WORLD scenario or the EXAM, I would take things a step further and login to the MySQL database and verify what is there. We are given information through troubleshooting on what should be coming out so you could do a connection to the database and do a "select * from <table_name>" to see the data that is returned and verify it matches the listing from the PHP website.

By completing this process, you are showing that the data is coming out of the database container and the PHP website matches the contents that are in the Database.

Finally, one thing to keep in mind, the classroom environment, labs, and sometimes the GEs can have some slight issues with checks and verifications in terms of gradining and implementation as the classes are quite large and lots of things can happen. The team writing the exams and creating the evaluations and problems for the exams are a completely different team and way way way more care has been put into the development and testing of the exams and evaluation process. The courses having Guided Exercises and then End of Chapter labs with the solutions, sometimes the grading on the EoC labs can be less robust in catching all the different aspects and may "accidentally" look for something that was done in a solution. This is "not" the way exams are designed.

Again, the larger concern here is that just checking an HTTP website isn't sufficient in this case and the solution provided guides you through steps. However, to fully understand, you should be able to work this without a solution and realize the steps to take to troubleshoot a scenario like this, because you are right, if there is an assumption the when a website returns data things are good and everything is OK, that is very scary indeed as you should know if what is being returned is correct and valid. The only way to do that in this case would be to connect to the database and view the data and when attempting to connect with a client, you would immediately know that MySQL client couldn't connect to the database.

[Chetan_Tiwary_]

@Travis just enlarged the images for better visibility - hope you won't mind. 

[Travis]

Nope ... that's perfect. I don't always think about that as I normally click on images and it enlarges them when I look at them. I run high resolution, so  things oftten look small to me and I don't pay attention to if it is actually small. 

[Travis]

@oblanco -

I will give you some POINTS and CREDIT in that you are correct as if the "port" isn't forwarded, because things are running on SERVERA and because the network is running on SERVERA and it has access to all the container networks, that HTTPD can connect to the MYSQL database and return information. So yes, in this instance, because you weren't sepcifically told to open the port and just told that the application had to work, things are unfair. I think what we could do is in the instructions and scenario be a little more specific about a "containerized database which is exposed to external sources both on ServerA and outside of ServerA" that would have been enough.

However, in troubleshooting, as stated in my answer, the thing we are missing is actually verifying the data we are getting is valid. Just because we are getting data back doesn't mean it is correct or good, so a natural thing to do is check to see if the source matches and in checking the data source, you would have needed to open the port.

Again, as mentioned the grading for exams is a completely different team and we look at things differently there. For the course environment, often we use the scripts to check what we've done to get to a specific solution and that isn't necessarily the "ONLY" way to do things.

@bonnevil - What do you think about making the scenario a little bit more detailed here as if we truly go through the lab solutions and change the IP, it does connect to the database because we are using the containerized network on ServerA and because it is bound to the container IP and not the NAT address, ServerA can reach things without port forward. The other thing we could do is make the MySQL/MARIADB a "rootless" container and then it would be forced to use the NAT which would 100% require the port forward on ServerA.

@Chetan_Tiwary_ and @Wasim_Raja  ^^^^ thoughts on above?