問題描述
下面的代碼將三個數(shù)字放在一個隊列中.然后它嘗試從隊列中取回號碼.但它永遠不會.如何從隊列中獲取數(shù)據(jù)?
The code below places three numbers in a queue. Then it attempts to get the numbers back from the queue. But it never does. How to get the data from the queue?
import multiprocessing
queue = multiprocessing.Queue()
for i in range(3):
queue.put(i)
while not queue.empty():
print queue.get()
推薦答案
我最初在閱讀@Martijn Pieters 后刪除了這個答案，因為他更詳細(xì)和更早地描述了為什么這不起作用".然后我意識到，OP 示例中的用例不太適合
I originally deleted this answer after I read @Martijn Pieters', since he decribed the "why this doesn't work" in more detail and earlier. Then I realized, that the use case in OP's example doesn't quite fit to the canonical sounding title of
如何使用 multiprocessing.Queue.get 方法".
"How to use multiprocessing.Queue.get method".
那不是因為有演示中不涉及子進程，但是因為在實際應(yīng)用程序中，幾乎沒有一個隊列是預(yù)先填充的，并且只在之后讀取，但是讀取并且寫入發(fā)生在中間的等待時間之間.Martijn 展示的擴展演示代碼在通常情況下不起作用，因為當(dāng)排隊跟不上讀取速度時，while 循環(huán)會過早中斷.所以這里是重新加載的答案，它能夠處理通常的交錯提要和讀取場景:
That's not because there's no child process involved for demonstration, but because in real applications hardly ever a queue is pre-filled and only read out after, but reading and writing happens interleaved with waiting times in between. The extended demonstration code Martijn showed, wouldn't work in the usual scenarios, because the while loop would break too soon when enqueuing doesn't keep up with reading. So here is the answer reloaded, which is able to deal with the usual interleaved feeds & reads scenarios:
不要依賴 queue.empty 檢查同步.
Don't rely on queue.empty checks for synchronization.
在將對象放入空隊列后，隊列的 empty() 方法返回 False 并且 get_nowait() 可以在不引發(fā) queue.Empty 的情況下返回之前，可能會有一個無限小的延遲....
After putting an object on an empty queue there may be an infinitesimal delay before the queue’s empty() method returns False and get_nowait() can return without raising queue.Empty. ...
empty()
如果隊列為空，則返回 True，否則返回 False.由于多線程/多處理語義，這是不可靠的.docs
Return True if the queue is empty, False otherwise. Because of multithreading/multiprocessing semantics, this is not reliable. docs
從隊列中使用 for msg in iter(queue.get, sentinel): 到 .get()，您可以通過傳遞一個哨兵值...iter(callable, sentinel)?
Either use for msg in iter(queue.get, sentinel): to .get() from the queue, where you break out of the loop by passing a sentinel value...iter(callable, sentinel)?
from multiprocessing import Queue
SENTINEL = None
if __name__ == '__main__':
queue = Queue()
for i in [*range(3), SENTINEL]:
queue.put(i)
for msg in iter(queue.get, SENTINEL):
print(msg)
...如果您需要非阻塞解決方案，請使用 get_nowait() 并處理可能的 queue.Empty 異常.
...or use get_nowait() and handle a possible queue.Empty exception if you need a non-blocking solution.
from multiprocessing import Queue
from queue import Empty
import time
SENTINEL = None
if __name__ == '__main__':
queue = Queue()
for i in [*range(3), SENTINEL]:
queue.put(i)
while True:
try:
msg = queue.get_nowait()
if msg == SENTINEL:
break
print(msg)
except Empty:
# do other stuff
time.sleep(0.1)
如果只有一個進程和該進程中的一個線程正在讀取隊列，也可以將最后一個代碼片段交換為:
In case only one process and only one thread within this process is reading the queue, it would be also possible to exchange the last code snippet with:
while True:
if not queue.empty(): # this is not an atomic operation ...
msg = queue.get() # ... thread could be interrupted in between
if msg == SENTINEL:
break
print(msg)
else:
# do other stuff
time.sleep(0.1)
由于線程可能會在檢查 if not queue 之間刪除 GIL.empty() 和 queue.get()，這不適用于進程中的多線程隊列讀取.如果多個進程正在從隊列中讀取，這同樣適用.
Since a thread could drop the GIL in between checking if not queue.empty() and queue.get(), this wouldn't be suitable for multi-threaded queue-reads in a process. The same applies if multiple processes are reading from the queue.
對于單一生產(chǎn)者/單一消費者的場景，使用 multiprocessing.Pipe 而不是 multiprocessing.Queue 就足夠了，而且性能更高.
For single-producer / single-consumer scenarios, using a multiprocessing.Pipe instead of multiprocessing.Queue would be sufficient and more performant, though.
這篇關(guān)于如何使用 multiprocessing.Queue.get 方法?的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！