問(wèn)題描述

在我的 Java 代碼中，我使用 Guava 的 Multimap (com.google.common.collect.Multimap) 使用這個(gè):

In my Java code, I am using Guava's Multimap (com.google.common.collect.Multimap) by using this:

 Multimap<Integer, Integer> Index = HashMultimap.create()

這里，Multimap 鍵是 URL 的一部分，值是 URL 的另一部分(轉(zhuǎn)換為整數(shù)).現(xiàn)在，我分配我的 JVM 2560 Mb (2.5 GB) 堆空間(通過(guò)使用 Xmx 和 Xms).但是，它只能存儲(chǔ) 900 萬(wàn)個(gè)這樣的(鍵、值)整數(shù)對(duì)(大約 1000 萬(wàn)個(gè)).但是，理論上(根據(jù) int 占用的內(nèi)存)它應(yīng)該存儲(chǔ)更多.

Here, Multimap key is some portion of a URL and value is another portion of the URL (converted into an integer). Now, I assign my JVM 2560 Mb (2.5 GB) heap space (by using Xmx and Xms). However, it can only store 9 millions of such (key,value) pairs of integers (approx 10 million). But, theoretically (according to memory occupied by int) it should store more.

誰(shuí)能幫幫我，

1. 為什么 Multimap 使用大量?jī)?nèi)存?我檢查了我的代碼，沒(méi)有在 Multimap 中插入對(duì)，它只使用了 1/2 MB 的內(nèi)存.
2.

1. Why is Multimap using lots of memory? I checked my code and without inserting pairs into the Multimap, it only uses 1/2 MB of memory.
2.

是否有另一種方法或自制的解決方案來(lái)解決這個(gè)內(nèi)存問(wèn)題?意思是，有沒(méi)有辦法減少這些對(duì)象開(kāi)銷(xiāo)，因?yàn)槲抑幌氪鎯?chǔ) int-int?在任何其他語(yǔ)言?或任何其他解決方案(首選自制)來(lái)解決我面臨的問(wèn)題，意味著基于數(shù)據(jù)庫(kù)或類(lèi)似的解決方案.

Is there another way or home-baked solution to solve this memory issue? Means, Is there any way to reduce those object overheads as I want to store only int-int? In any other language ? Or any other solution (home-baked preferred) to solve issue I faced, means DB based or something like that solution.

推薦答案

與 Multimap 相關(guān)的開(kāi)銷(xiāo)很大.至少:

There's a huge amount of overhead associated with Multimap. At a minimum:

• 每個(gè)鍵和值都是一個(gè) Integer 對(duì)象，它(至少)使每個(gè) int 值的存儲(chǔ)需求翻倍.
• HashMultimap 中的每個(gè)唯一鍵值都與一個(gè) Collection 值相關(guān)聯(lián)(根據(jù) 來(lái)源，Collection是哈希集).
• 每個(gè) Hashset 都使用 8 個(gè)值的默認(rèn)空間創(chuàng)建.

• Each key and value is an Integer object, which (at a minimum) doubles the storage requirements of each int value.
• Each unique key value in the HashMultimap is associated with a Collection of values (according to the source, the Collection is a Hashset).
• Each Hashset is created with default space for 8 values.

因此，每個(gè)鍵/值對(duì)(至少)需要的空間可能比您對(duì)兩個(gè) int 值的預(yù)期多一個(gè)數(shù)量級(jí).(當(dāng)多個(gè)值存儲(chǔ)在一個(gè)鍵下時(shí)會(huì)少一些.)我預(yù)計(jì) 1000 萬(wàn)個(gè)鍵/值對(duì)可能占用 400MB.

So each key/value pair requires (at a minimum) perhaps an order of magnitude more space than you might expect for two int values. (Somewhat less when multiple values are stored under a single key.) I would expect 10 million key/value pairs to take perhaps 400MB.

雖然您有 2.5GB 的堆空間，但如果這還不夠，我也不會(huì)感到驚訝.我認(rèn)為，上述估計(jì)偏低.此外，它僅說(shuō)明地圖構(gòu)建后需要存儲(chǔ)多少.隨著映射的增長(zhǎng)，表需要重新分配和重新散列，這暫時(shí)至少使使用的空間量增加一倍.最后，所有這些都假設(shè) int 值和對(duì)象引用需要 4 個(gè)字節(jié).如果 JVM 使用 64 位尋址，字節(jié)數(shù)可能會(huì)翻倍.

Although you have 2.5GB of heap space, I wouldn't be all that surprised if that's not enough. The above estimate is, I think, on the low side. Plus, it only accounts for how much is needed to store the map once it is built. As the map grows, the table needs to be reallocated and rehashed, which temporarily at least doubles the amount of space used. Finally, all this assumes that int values and object references require 4 bytes. If the JVM is using 64-bit addressing, the byte count probably doubles.

這篇關(guān)于多地圖空間問(wèn)題:番石榴的文章就介紹到這了，希望我們推薦的答案對(duì)大家有所幫助，也希望大家多多支持html5模板網(wǎng)！