問題描述

這段代碼遇到了死鎖問題:

I'm having deadlock problems with this piece of code:

def _entropy_split_parallel(data_train, answers_train, weights):
    CPUS = 1 #multiprocessing.cpu_count()
    NUMBER_TASKS = len(data_train[0])
    processes = []

    multi_list = zip(data_train, answers_train, weights)

    task_queue = multiprocessing.Queue()
    done_queue = multiprocessing.Queue()

    for feature_index in xrange(NUMBER_TASKS):
        task_queue.put(feature_index)

    for i in xrange(CPUS):
        process = multiprocessing.Process(target=_worker, 
                args=(multi_list, task_queue, done_queue))
        processes.append(process)
        process.start()

    min_entropy = None
    best_feature = None
    best_split = None
    for i in xrange(NUMBER_TASKS):
        entropy, feature, split = done_queue.get()
        if (entropy < min_entropy or min_entropy == None) and entropy != None:
            best_feature = feature
            best_split = split

    for i in xrange(CPUS):
        task_queue.put('STOP')

    for process in processes:
        process.join()

    return best_feature, best_split


def _worker(multi_list, task_queue, done_queue):
    feature_index = task_queue.get()
    while feature_index != 'STOP':
        result = _entropy_split3(multi_list, feature_index)
        done_queue.put(result)
        feature_index = task_queue.get()

當我運行我的程序時，它可以通過 _entropy_split_parallel 運行幾次，但最終會死鎖.父進程在 done_queue.get() 上阻塞，工作進程在 done_queue.put() 上阻塞.由于發(fā)生這種情況時隊列始終為空，因此預計會阻塞 get.我不明白為什么工作人員會阻塞 put，因為隊列顯然沒有滿(它是空的！).我試過 block 和 timeout 關鍵字參數(shù)，但得到相同的結果.

When I run my program, it works fine for several runs through _entropy_split_parallel, but eventually deadlocks. The parent process is blocking on done_queue.get(), and the worker process is blocking on done_queue.put(). Since the queue is always empty when this happens, blocking on get is expected. What I don't understand is why the worker is blocking on put, since the queue is obviously not full (it's empty!). I've tried the block and timeout keyword arguments, but get the same result.

我正在使用多處理反向端口，因為我堅持使用 Python 2.5.

I'm using the multiprocessing backport, since I'm stuck with Python 2.5.

看起來我也遇到了多處理模塊提供的示例之一的死鎖問題.這是 here. 底部的第三個示例.如果我調用多次測試方法.例如，將腳本底部更改為:

It looks like I'm also getting deadlock issues with one of the examples provided with the multiprocessing module. It's the third example from the bottom here. The deadlocking only seems to occur if I call the test method many times. For example, changing the bottom of the script to this:

if __name__ == '__main__':
    freeze_support()
    for x in xrange(1000):
        test()

<小時>

我知道這是一個老問題，但測試表明這在使用 Python 2.7 的 Windows 上不再是問題.我將嘗試 Linux 并報告.

I know this is an old question, but testing shows that this is no longer a problem on windows with Python 2.7. I will try Linux and report back.

推薦答案

這個問題在 Python 的新版本中消失了，所以我假設它是 backport 的問題.無論如何，這不再是問題.

This problem went away with newer versions of Python, so I'm assuming it was a problem with the backport. Anyways, it's no longer an issue.

這篇關于Python multiprocessing.Queue 上的 put 和 get 死鎖的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！