javaOOM should analyze the dump file instead of looking at the abnormal log.

javaOOM should analyze the dump file instead of looking at the abnormal log.

The application has an OOM exception, do you still troubleshoot the problem by looking at the log (it is only a coincidence that this method is used to locate and solve the problem)? The correct troubleshooting plan is to analyze the dump file. Do you know why?

OOM exception--intsmaze

First of all, the OOM exceptions I encountered during development are basically located by looking at the log log (many OOM exceptions are caused by an infinite loop or the amount of data returned by the query, no paging, etc., through the exception log we can indeed Quickly locate, but this is not the correct posture.) It just happens that the exception stack information printed in the log is the corresponding code problem. Many blogs have also said that locating OOM exceptions is by analyzing dump logs, so I am deeply puzzled, why I have to analyze dump logs, which can be solved by looking at log logs, and I have not retrieved satisfactory answers on the Internet. I asked many developers around me. , It only says that dump performs performance analysis, and log logs perform abnormal investigation. After a few deep thoughts, I suddenly opened my mind, and hereby write down the reasons.

Abnormal OOM will cause the program to crash and the process ends.

Correct posture dump file analysis--intsmaze

Please see the big screen:

public class OOMDump {

    static class OOMIntsmaze {
        public byte[] placeholder = new byte[64 * 1024];
    }

    public static void fillHeap(ArrayList<OOMIntsmaze> list, int num) throws Exception {

        for (int i = 0; i <num; i++) {
            list.add(new OOMIntsmaze());
            System.out.println(i);
        }
    }

    public static void main(String[] args) throws Exception {
        ArrayList<OOMIntsmaze> list = new ArrayList<OOMIntsmaze>();
        fillHeap(list,131);
        Thread.sleep(20000000);
    }
}

We configure the jvm parameters as follows -Xmx10M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=d://When fillHeap(list,131), the program executes normally; when fillHeap(list,132), the program will report an OOM exception:

130
java.lang.OutOfMemoryError: Java heap space
Dumping heap to d://\java_pid10000.hprof ...
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at cn.intsmaze.dump.OOMDump$OOMIntsmaze.<init>(OOMDump.java:27)
    at cn.intsmaze.dump.OOMDump.fillHeap(OOMDump.java:34)
    at cn.intsmaze.dump.OOMDump.main(OOMDump.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
Heap dump file created [10195071 bytes in 0.017 secs]
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

Through the exception log, we can see that the problem is caused by the code at cn.intsmaze.dump.OOMDump.fillHeap(OOMDump.java:34) list.add(new OOMIntsmaze()); through the log WYSIWYG, I The problem was solved immediately, why should I look at the dump log? I'm sick.

Actually not, Sao Nian. If the main method is as follows, execute

    public static void main(String[] args) throws Exception {
        ArrayList<OOMIntsmaze> list = new ArrayList<OOMIntsmaze>();
        fillHeap(list,131);
        Map<String,OOMIntsmaze> map=new HashMap<String,OOMIntsmaze>();
        map.put("LIUYANG",new OOMIntsmaze());
        map.put("intsmaze",new OOMIntsmaze());
        Thread.sleep(20000000);
    }

At this time, we found through the exception log that it was caused by map.put("LIUYANG",new OOMIntsmaze());. When we found the code and found that a piece of data was inserted into the map, there was no infinite loop. How could it cause an OOM exception? See you a long time ago.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at cn.intsmaze.dump.OOMDump$OOMIntsmaze.<init>(OOMDump.java:27)
    at cn.intsmaze.dump.OOMDump.main(OOMDump.java:49)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

Setting 10M, we can find that OOM will occur when 132 elements are added to the list. At this time, we add 131 elements to the list, and then execute the map addition. When we find that the map adds an element, an error is reported. The oom exception log at this time locates the result of adding elements to the map. But the real situation is not, because looking at the code will also find that the map only adds 2 elements, how could it be caused by him. The addition of the map is just at the moment when the jvm memory reaches the upper limit of its capacity. Therefore, to find the fundamental problem, the capacity status of each object needs to be analyzed through the dump file when the OOM is analyzed. And in reality, the code of map.put() will not be put together with the code of list.add() like the above example, but will be located in different packages and different business processes. At this time, it is basically impossible to locate the log by looking at the log. But why everyone travels OOM abnormally by looking at the log and the location is correct. It's just that it has been executing in the loop like list.add, and it is basically triggered by him with a high probability.

Correct posture--intsmaze

Therefore, in order to prevent trouble, when the programmer is developing, we must configure the jvm startup parameter HeapDumpOnOutOfMemoryError. Parameter -XX: +HeapDumpOnOutOfMemoryError can allow the virtual machine to dump the current memory heap dump snapshot when a memory overflow exception occurs for later analysis

dump lost print --intsmaze

Sometimes, our application is down, neither log daily information will be printed nor dump file will be generated. At this time, it is basically the Linux system that killed our application process.

View the/var/log/messages file

The messages log is the core system log file. It contains the boot messages when the system starts, and other status messages when the system is running. The following information will appear in messages

out of memory:kill process 8398(java) score 699 or sacrifice child
killed process 8398,UID 505,(java) total-vm:2572232kB,anno-rss:1431292kB,file-rss:908kB

The oom killer is a protection process of the Linux system. It will be triggered when the remaining memory space of the Linux system is insufficient for the normal operation of the system. When oomkiller executes, it will find the pid with the highest score value of all threads in the system, and then kill it. Here we can see that the JAVA process was indeed killed by the oom killer of LINUX.

Both our applications and logs can only record memory overflows that occur within the JVM. If the heap size set by the JVM exceeds the memory size allowed by the operating system, the operating system will directly kill the process. In this case, the JVM cannot record this operation. Linux has an OOM score for each process, this score is in the/proc/pid/oom_score file. For example,/proc/8398/oom_score, if you do not want to kill this process, change the content of oom_adj to -17. For more information about Linux's oom killer mechanism, please search Baidu by yourself.

The most correct posture: first adjust the heap size of the JVM so that the OOM of the JVM appears prior to the OOM of the operating system, and then set the operating parameters, and output the heapdump file when OOM occurs.