how to resolve OOM issue on JBOSS

[jeesshnasree]

how to resolve OOM issue on JBOSS 

[Scott · Accepted solution]

@jeesshnasree,

The short answer is "Use less memory" or "Get more memory".  Not having any specifics in your post, I'm just going to guess that this is because a java JVM is consuming more memory than is expected, likely because the application running in it is doing something weird or allocating too much and not releasing it, or something like that.

If it's consistent, you should be able to see these processes prior to them getting killed by OOM in something like top.  Further, you can attach an strace to the process to try and figure out what it's doing.  From the system administration perspective though, you're probably not going to be able to influence this without a change to the app or additional RAM being added to the hardware.

-STM

[jeesshnasree]

 one JVM process logs contain - java.lang.OutOfMemoryError: Java heap space in in JBOSS EAP .

how to resolve this issue and finally unable to find jstack . how to generate thread dump . kindly suggest . 

[oldbenko]

Yeah, you see, @jeesshnasree, this is where your brevity starts playing into the area where whoever tries to help you asks themselves, "why did I even try?"

An OOM is something completely different from an OutOfMemoryError.

If OutOfMemoryError is what you are getting (rather than a kernel OOM), this can be because of a couple of different reasons, none of which I can reliably guess with the information you have given us.

It may be your MetaSpace (or PermGen) is too small, and the amount of space allocated to permanent objects exceeds it, or it may be because of what @Scott suggested, namely, a memory leak. Or it may be because you are (or someone else is) using a NoOp garbage collector. Or it may be because the aliens have just invaded and the first thing they did was corrupt all JVM garbage collectors.

In summary, this requires way more information than what you provided.

Look into tools such as HeapAnalyzer and GCA (both from IBM's developerworks), and run the application with verbose garbage collection logs (-verbose:gc and other options), so you at least get some idea of what the hell is happening to your JVM.

If you need a stack trace (which won't help you a lot with this problem), you can send a QUIT signal to the JVM - knowing what the process ID of the right JVM is (say $pid), do a "kill -QUIT $pid" (but the -QUIT part here is extremely important).

You may also want to try to connect to the JVM using jconsole (or jvisualvm) and request a heap dump from the HotSpotDiagnostics MBean in the JMX section (if HotSpot is the VM you're running).

The number of guesses I had to make to write any of the above should give you a hint you are really not being very fair by giving us nothing to work with and then requesting help.

All of us are here because we like to help. But if you keep asking questions the way you do, I'm afraid we'll eventually stop because it requires way more effort on our side than you're willing to put in.

I suggest re-reading my advice on how to ask smart questions the other day. If you don't believe me, try googling for "how to ask good questions when debugging" or something similar.

Kind regards,
Grega

[oldbenko · Accepted solution]

Hey, @jeesshnasree,

I'll just add some Java-specific notes here.

From what little you say, it seems that the JVM is somehow trying to use more RAM than it should, which can only be for two reasons:

• you are telling it to do so (by setting maximum heap size too high using -Xmx option)
• it can't figure out correctly how much RAM it can use, which would imply a memory cgroup and an old JVM (pre-9)

The Java Virtual Machine would not try to exceed the amount of available physical memory in any other case.

In either case, what happens is that the amount of heap allocated for live objects exceeds the amount of physical memory in the system, and the kernel has no other choice but to kill the offending application.

What I suggest is that you carefully review what kind of memory constraints the application and the system have and why, and then adjust the -Xmx setting to roughly 150-250MB less than that (again, completely depending on your permanent generation / metaspace, and stack settings).

Next time before you ask questions, please refer back to the reminder I have already given you and tell us more about the nature of your problem. Provide logs and more context.

Kind regards,
Grega