python - multiprocessing Programming guidelines unclear -

python - multiprocessing Programming guidelines unclear -

i trying understand next guideline:

better inherit pickle/unpickle

when using spawn or forkserver start methods many types multiprocessing need picklable kid processes can utilize them. however, 1 should avoid sending shared objects other processes using pipes or queues. instead should arrange programme process needs access shared resource created elsewhere can inherit ancestor process.

what mean "arrange program"? how can share resources inheriting?

i'm running windows, new processes spawned, means forked processes can inherit?

1. mean "arrange program"?

it means programme should able run self-contained without external resources. sharing files give locking issues, sharing memory either same or can give corruption due multiple processes modifying info @ same time.

here's illustration of bad idea:

while some_queue_is_not_empty(): run_external_process(some_queue) def external_process(queue): item = queue.pop() # processing here

versus:

while some_queue_is_not_empty(): item = queue.pop() run_external_process(item) def external_process(item): # processing here

this way can avoid locking queue and/or corruption issues due multiple processes getting same item.

2. how can share resources inheriting?

on windows, can't. on linux can utilize file descriptors parent opened, on windows brand new process don't have parent except given.

example copied from: http://rhodesmill.org/brandon/2010/python-multiprocessing-linux-windows/

from multiprocessing import process f = none def child(): print f if __name__ == '__main__': f = open('mp.py', 'r') p = process(target=child) p.start() p.join()

on linux like:

$ python mp.py <open file 'mp.py', mode 'r' @ 0xb7734ac8>

on windows get:

c:\users\brandon\dev>python mp.py none

python multiprocessing