問題描述
我有兩段代碼用于了解 Python 3.1 中的多處理.我的目標是使用 100% 的所有可用處理器.但是，這里的代碼片段在所有處理器上僅達到 30% - 50%.
I have two pieces of code that I'm using to learn about multiprocessing in Python 3.1. My goal is to use 100% of all the available processors. However, the code snippets here only reach 30% - 50% on all processors.
無論如何強制"python 100% 使用?操作系統(Windows 7、64 位)是否限制了 Python 對處理器的訪問?當下面的代碼片段正在運行時，我打開任務管理器并觀察處理器的峰值，但從未達到并保持 100%.除此之外，我還可以看到在此過程中創建和銷毀了多個 python.exe 進程.這些過程與處理器有什么關系?例如，如果我生成 4 個進程，則每個進程都沒有使用它自己的核心.相反，這些進程使用的是什么?他們是否共享所有內核?如果是這樣，是操作系統強制進程共享內核嗎?
Is there anyway to 'force' python to use all 100%? Is the OS (windows 7, 64bit) limiting Python's access to the processors? While the code snippets below are running, I open the task manager and watch the processor's spike, but never reach and maintain 100%. In addition to that, I can see multiple python.exe processes created and destroyed along the way. How do these processes relate to processors? For example, if I spawn 4 processes, each process isn't using it's own core. Instead, what are the processes using? Are they sharing all cores? And if so, is it the OS that is forcing the processes to share the cores?
import multiprocessing
def worker():
#worker function
print ('Worker')
x = 0
while x < 1000:
print(x)
x += 1
return
if __name__ == '__main__':
jobs = []
for i in range(50):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()
代碼片段 2
from multiprocessing import Process, Lock
def f(l, i):
l.acquire()
print('worker ', i)
x = 0
while x < 1000:
print(x)
x += 1
l.release()
if __name__ == '__main__':
lock = Lock()
for num in range(50):
Process(target=f, args=(lock, num)).start()
推薦答案
要使用 100% 的所有內核，不要創建和銷毀新進程.
To use 100% of all cores, do not create and destroy new processes.
為每個核心創建幾個進程并將它們與管道鏈接.
Create a few processes per core and link them with a pipeline.
在操作系統級別，所有流水線進程同時運行.
At the OS-level, all pipelined processes run concurrently.
你寫的越少(你委托給操作系統的越多)你就越有可能使用盡可能多的資源.
The less you write (and the more you delegate to the OS) the more likely you are to use as many resources as possible.
python p1.py | python p2.py | python p3.py | python p4.py ...
將最大限度地利用您的 CPU.
Will make maximal use of your CPU.
這篇關于將 100% 的內核與多處理模塊一起使用的文章就介紹到這了，希望我們推薦的答案對大家有所幫助，也希望大家多多支持html5模板網！