問題描述

當使用 Lucene 索引文檔時，我的 JVM (1.6.0_29) 在密集使用時不斷崩潰.我明白了:

My JVM (1.6.0_29) keeps crashing on intensive use when indexing documents with Lucene. I get:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00002b6b196d767c, pid=26417, tid=1183217984
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J  org.apache.lucene.store.DataInput.readVInt()I
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

環境:

JDK:1.6u29(與 1.6_02 相同的問題)Lucene 版本 3.4.0

JDK: 1.6u29 (same issue with 1.6_02) Lucene Version 3.4.0

vm_info:用于 linux-amd64 JRE (1.6.0_29-b11) 的 Java HotSpot(TM) 64 位服務器 VM (20.4-b02)，由java_re"和 gcc 3.2 于 2011 年 10 月 3 日 01:19:20 構建.2 (SuSE Linux)

vm_info: Java HotSpot(TM) 64-Bit Server VM (20.4-b02) for linux-amd64 JRE (1.6.0_29-b11), built on Oct 3 2011 01:19:20 by "java_re" with gcc 3.2.2 (SuSE Linux)

操作系統:CentOS 5.0 版(最終版)

OS:CentOS release 5.0 (Final)

jvm_args: -Dcatalina.home=/var/local/tomcat-8081 -Dcatalina.base=/var/local/tomcat-8081 -Djava.io.tmpdir=/var/tmp -Dfile.encoding=UTF-8-Xmx1024M -XX:MaxPermSize=96m

jvm_args: -Dcatalina.home=/var/local/tomcat-8081 -Dcatalina.base=/var/local/tomcat-8081 -Djava.io.tmpdir=/var/tmp -Dfile.encoding=UTF-8 -Xmx1024M -XX:MaxPermSize=96m

這似乎是 jdk 1.7 中修復的 jdk 問題，但引入了其他問題.https://issues.apache.org/jira/browse/LUCENE-3335Java 7 包含自 1.6.0_21 以來對 readVInt 問題的修復(大約，LUCENE-2975)"

It seems to be a jdk issue that was fixed in jdk 1.7, but other issues where introduced. https://issues.apache.org/jira/browse/LUCENE-3335 "Java 7 contains a fix to the readVInt issue since 1.6.0_21 (approx, LUCENE-2975)"

那么，如何使用 JDK 1.6 解決這個問題?我應該升級到 jdk 1.7 嗎?

So, how can I fix this issue using JDK 1.6? Should I upgrade to jdk 1.7?

推薦答案

這些 JDK 問題也在 1.6.9_29(不僅僅是 1.7.0u1)中得到修復.ReadVInt 不能再崩潰.因此，您的崩潰與任何著名的 java6/7 錯誤"都無關(vint 錯誤根本不會使您的 JVM 崩潰，它只是通過返回錯誤值來破壞您的索引 - 自 Lucene 3.1 以來這個問題肯定已修復).

these JDK issues are also fixed in 1.6.9_29 (not only 1.7.0u1). ReadVInt can no longer crash. So your crash is not related to any of the "famous java6/7 bugs" (the vint bug does not crash your JVM at all it just corrupts your index by returning wrong values - and this one is definitely fixed since Lucene 3.1).

但是還有另一個可能導致 JVM 崩潰:您在 64 位平臺 (Linux) 上，因此默認目錄實現是 MMapDirectory.Lucene 使用 hack 能夠從虛擬地址空間取消映射映射文件.JVM 本身不允許這樣做，但會使取消映射依賴于垃圾收集器，這對 Lucene 來說是個問題.默認情況下，MMapDirectory 在關閉 IndexInputs 后取消映射文件.MMapDirectory 根本不同步，所以當另一個線程在取消映射后嘗試訪問 IndexInput 時，它將訪問一個未映射的地址并會 SIGSEGV.

But there is another chance you can crash your JVM: You are on a 64 bit platform (Linux), so the default directory implementation is MMapDirectory. Lucene uses a hack to be able to unmap mapped files from virtual address space. This is not allowed by the JVM itsself, but makes unmapping dependent on garbage collector, which is a problem for Lucene. By default MMapDirectory unmaps the files after closing the IndexInputs. MMapDirectory is not synchronized at all, so when another thread tries to access the IndexInput after unmapping it will access an unmapped address and will SIGSEGV.

如果您的代碼正確，這不會發生，但看起來您正在使用已經關閉的 IndexReader/IndexWriter 來訪問索引.在 Lucene 3.5(即將推出)之前，IndexReader 中缺少檢查將導致已關閉的 IndexReader 及其所有已關閉(且未映射)的 IndexInputs 嘗試訪問索引數據和段錯誤.

If your code would be correct this cannot happen, but it looks like you are using an already closed IndexReader/IndexWriter to access the index. Before Lucene 3.5 (will come out soon), missing checks in IndexReader will make it possible that an already closed IndexReader with all its closed (and unmapped) IndexInputs tries to access index data and segfaults.

在 3.5 中，我們添加了額外的安全檢查來防止這種非法訪問，但它不是 100%(因為缺少同步).我會查看代碼并檢查是否沒有任何內容可以訪問已關閉的索引.

In 3.5 we added additional safety checks to prevent this illegal access, but its not 100% (as synchronization is missing). I would review the code and check that nothing accesses closed index.

檢查這是否是您的問題的簡單檢查是使用 NIOFSDirectory(在 Linux 上較慢)而不是 MMapDirectory.如果它沒有崩潰并且可能拋出 AlreadyClosedExceptions，則該錯誤正在訪問已關閉的索引.

A simple check to see if this is your issue would be to use NIOFSDirectory (slower on Linux) instead of MMapDirectory. If it does not crash and possibly throws AlreadyClosedExceptions, the bug is accessing closed indexes.

這篇關于JVM 在 Lucene DataInput.readVInt 上崩潰的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！