問題描述

我需要并行執行同一類的許多實例的方法.為此，我嘗試使用 Process.start() 和 Process.join() 命令來自 multiprocessing 模塊.

I need to execute in parallel a method of many instances of the same class. For doing this I'm trying to use the Process.start() and the Process.join() commands from the multiprocessing module.

例如對于一個類:

class test:
     def __init__(self):
     ...
     ...
     def method(self):
     ...
     ...

where method 修改了一些類變量.如果我創建該類的兩個實例:

where method modifies some of the class variables. If I make two instances of the class:

t1=test()
t2=test()

并執行:

from multiprocessing import Process
pr1=Process(target=t1.method, args=(,))
pr2=Process(target=t2.method, args=(,))
pr1.start()
pr2.start()
pr1.join()
pr2.join()

類實例的變量沒有更新(整個代碼太長，這里貼不上來，不過是這個思路).

the variables of the instances of the class are not updated (the whole code is too long to be pasted here but this is the idea).

有什么方法可以實現嗎?謝謝

Is there any way to achieve this? Thank you

推薦答案

當您在子進程中調用 obj.method 時，子進程將在 <代碼>對象.因此，您在子項中對它們所做的更改不會反映在父項中.您需要通過 multiprocessing.Queue 以使更改生效父:

When you call obj.method in a child process, the child process is getting its own separate copy of each instance variable in obj. So, the changes you make to them in the child will not be reflected in the parent. You'll need to explicitly pass the changed values back to the parent via a multiprocessing.Queue in order to make the changes take effect the parent:

from multiprocessing import Process, Queue
q1 = Queue()
q2 = Queue()
pr1 = Process(target=t1.method, args=(q1,))
pr2 = Process(target=t2.method, args=(q2,))
pr1.start()
pr2.start()
out1 = q1.get()
out2 = q2.get()
t1.blah = out1
t2.blah = out2
pr1.join()
pr2.join()

其他選項是使您需要更改的實例變量 multiprocessing.Value 實例，或 multiprocessing.Manager Proxy 實例.這樣，您在子項中所做的更改會自動反映在父項中.但這是以增加使用父級變量的開銷為代價的.

Other options would be to make the instance variables you need to change multiprocessing.Value instances, or multiprocessing.Manager Proxy instances. That way, the changes you make in the children would be reflected in the parent automatically. But that comes at the cost of adding overhead to using the variables in the parent.

這是一個使用 multiprocessing.Manager 的示例.這不起作用:

Here's an example using multiprocessing.Manager. This doesn't work:

import multiprocessing

class Test(object) :

    def __init__(self):
       self.some_list = []  # Normal list

    def method(self):
        self.some_list.append(123)  # This change gets lost


if __name__ == "__main__":
    t1 = Test()
    t2 = Test()
    pr1 = multiprocessing.Process(target=t1.method)
    pr2 = multiprocessing.Process(target=t2.method)
    pr1.start()
    pr2.start()
    pr1.join()
    pr2.join()
    print(t1.some_list)
    print(t2.some_list)

輸出:

[]
[]

這行得通:

import multiprocessing

class Test(object) :

    def __init__(self):
       self.manager = multiprocessing.Manager()
       self.some_list = self.manager.list()  # Shared Proxy to a list

    def method(self):
        self.some_list.append(123) # This change won't be lost


if __name__ == "__main__":
    t1 = Test()
    t2 = Test()
    pr1 = multiprocessing.Process(target=t1.method)
    pr2 = multiprocessing.Process(target=t2.method)
    pr1.start()
    pr2.start()
    pr1.join()
    pr2.join()
    print(t1.some_list)
    print(t2.some_list)

輸出:

[123]
[123]

請記住，multiprocessing.Manager 會啟動一個子進程來管理您創建的所有共享實例，并且每次您訪問其中一個 Proxy 實例時，您實際上是在對 Manager 進程進行 IPC 調用.

Just keep in mind that a multiprocessing.Manager starts a child process to manage all the shared instances you create, and that every time you access one of the Proxy instances, you're actually making an IPC call to the Manager process.

這篇關于類方法的并行執行的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！