問題描述
所以我試圖在 python 中實現(xiàn)多處理，我希望有一個由 4-5 個進程組成的池并行運行一個方法.這樣做的目的是運行總共一千次 Monte 模擬(每個進程 250-200 次模擬)而不是運行 1000 次.我希望每個進程在處理完一個模擬的結(jié)果，寫入結(jié)果并釋放鎖.所以這應(yīng)該是一個三步過程:
So I'm trying to implement multiprocessing in python where I wish to have a Pool of 4-5 processes running a method in parallel. The purpose of this is to run a total of thousand Monte simulations (250-200 simulations per process) instead of running 1000. I want each process to write to a common shared array by acquiring a lock on it as soon as its done processing the result for one simulation, writing the result and releasing the lock. So it should be a three step process :
- 獲取鎖
- 寫入結(jié)果
- 為等待寫入數(shù)組的其他進程釋放鎖.
每次我將數(shù)組傳遞給進程時，每個進程都會創(chuàng)建一個我不想要的數(shù)組副本，因為我想要一個公共數(shù)組.任何人都可以通過提供示例代碼來幫助我嗎?
Everytime I pass the array to the processes each process creates a copy of that array which I donot want as I want a common array. Can anyone help me with this by providing sample code?
推薦答案
由于您只是將狀態(tài)從子進程返回到父進程，因此使用共享數(shù)組和顯式鎖是多余的.你可以使用 Pool.map 或 Pool.starmap 來完成你所需要的.例如:
Since you're only returning state from the child process to the parent process, then using a shared array and explicity locks is overkill. You can use Pool.map or Pool.starmap to accomplish exactly what you need. For example:
from multiprocessing import Pool
class Adder:
"""I'm using this class in place of a monte carlo simulator"""
def add(self, a, b):
return a + b
def setup(x, y, z):
"""Sets up the worker processes of the pool.
Here, x, y, and z would be your global settings. They are only included
as an example of how to pass args to setup. In this program they would
be "some arg", "another" and 2
"""
global adder
adder = Adder()
def job(a, b):
"""wrapper function to start the job in the child process"""
return adder.add(a, b)
if __name__ == "__main__":
args = list(zip(range(10), range(10, 20)))
# args == [(0, 10), (1, 11), ..., (8, 18), (9, 19)]
with Pool(initializer=setup, initargs=["some arg", "another", 2]) as pool:
# runs jobs in parallel and returns when all are complete
results = pool.starmap(job, args)
print(results) # prints [10, 12, ..., 26, 28]
這篇關(guān)于多處理 - 共享數(shù)組的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網(wǎng)！