多线程 VS 多进程（三）

线程替代方案

subprocess

完全跳过线程，使用进程

是派生进程的主要替代方案

python2.4后引入

multiprocessiong

使用threading接口派生，使用子进程

允许为多核或者多cpu派生进程，接口跟threading非常相似

python2.6后引入

concurrent.futures

新的异步执行模块

任务级别的操作

python3.2后引入

多进程

进程间通讯(InterprocessCommunication, IPC)

进程之间无任何共享状态

进程的创建

直接生成Process实例对象，案例19

import multiprocessing from time import sleep, ctime def clock(interval): while True: print("The time is %s" % ctime()) sleep(interval) if __name__ == '__main__': p = multiprocessing.Process(target=clock, args=(5, )) p.start() while True: print("sleeping......") sleep(1)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

sleeping...... The time is Tue Aug 13 19:47:41 2019 sleeping...... sleeping...... sleeping...... sleeping...... sleeping...... The time is Tue Aug 13 19:47:46 2019 sleeping...... sleeping...... sleeping...... sleeping...... ...

1

2

3

4

5

6

7

8

9

10

11

12

13

派生子类，案例20

import multiprocessing from time import sleep, ctime class ClockProcess(multiprocessing.Process): ''' 两个函数比较重要 1. init构造函数 2. run ''' def __init__(self, interval): super(ClockProcess, self).__init__() self.interval = interval def run(self): while True: print("The time is %s" % ctime()) sleep(self.interval) if __name__ == '__main__': p = ClockProcess(3) p.start()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

The time is Tue Aug 13 19:48:49 2019 The time is Tue Aug 13 19:48:52 2019 The time is Tue Aug 13 19:48:55 2019 The time is Tue Aug 13 19:48:58 2019 The time is Tue Aug 13 19:49:01 2019 The time is Tue Aug 13 19:49:04 2019 ...

1

2

3

4

5

6

7

在os中查看pid，ppid以及他们的关系

案例21

from multiprocessing import Process import os def info(title): print(title) print("module name:", __name__) # 得到父亲进程的id print("parent process:", os.getppid()) # 得到本身进程的id print("process id:", os.getpid()) def f(name): info('function f') print('hello', name) if __name__ == '__main__': info('main line') p = Process(target=f, args=('bob', )) p.start() p.join()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

main line module name: __main__ parent process: 11900 process id: 18832 function f module name: __mp_main__ parent process: 18832 process id: 20868 hello bob

1

2

3

4

5

6

7

8

9

生产者消费者模型

JoinableQueue

案例22

import multiprocessing from time import ctime def consumer(input_q): print("Into consumer:", ctime) while True: # 处理项 item = input_q.get() print('pull', item, 'out of q') input_q.task_done() # 发出信号通知任务完成 print("Out of consumer:", ctime()) ## 此句未执行，因为q.join()收集到四个task_done()信号后，主进程启动，未等到print此句完成，程序就结束了 def producer(sequence, output_q): print("Into procuder:", ctime()) for item in sequence: output_q.put(item) print('put', item, 'into q') print('Out of procuder', ctime()) # 建立进程 if __name__ == '__main__': q = multiprocessing.JoinableQueue() # 进行消费者进程 cons_p = multiprocessing.Process(target=consumer, args=(q, )) cons_p.daemon = True cons_p.start() # 生产多个项，sequence代表要发送给消费者的项序列 # 在实践中，这可能是生成器的输出或通过一些其他地方生产出来 sequence = [1, 2, 3, 4] producer(sequence, q) # 等待所有项被处理 q.join()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

Into procuder: Tue Aug 13 19:50:38 2019 put 1 into q put 2 into q put 3 into q put 4 into q Out of procuder Tue Aug 13 19:50:38 2019 Into consumer: pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q

1

2

3

4

5

6

7

8

9

10

11

队列中哨兵的使用，案例23

import multiprocessing from time import ctime # 设置哨兵问题 def consumer(input_q): print("Into consumer:", ctime()) while True: item = input_q.get() if item is None: break print("pull", item, "out of q") print ("Out of consumer:", ctime()) ## 此句执行完成，再转入主进程 def producer(sequence, output_q): print ("Into procuder:", ctime()) for item in sequence: output_q.put(item) print ("put", item, "into q") print ("Out of procuder:", ctime()) if __name__ == '__main__': q = multiprocessing.Queue() cons_p = multiprocessing.Process(target = consumer, args = (q,)) cons_p.start() sequence = [1,2,3,4] producer(sequence, q) q.put(None) cons_p.join()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

Into procuder: Tue Aug 13 19:51:23 2019 put 1 into q put 2 into q put 3 into q put 4 into q Out of procuder: Tue Aug 13 19:51:23 2019 Into consumer: Tue Aug 13 19:51:24 2019 pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q Out of consumer: Tue Aug 13 19:51:24 2019

1

2

3

4

5

6

7

8

9

10

11

12

哨兵的改进，案例24

import multiprocessing from time import ctime def consumer(input_q): print ("Into consumer:", ctime()) while True: item = input_q.get() if item is None: break print("pull", item, "out of q") print ("Out of consumer:", ctime()) def producer(sequence, output_q): for item in sequence: print ("Into procuder:", ctime()) output_q.put(item) print ("Out of procuder:", ctime()) if __name__ == '__main__': q = multiprocessing.Queue() cons_p1 = multiprocessing.Process (target = consumer, args = (q,)) cons_p1.start() cons_p2 = multiprocessing.Process (target = consumer, args = (q,)) cons_p2.start() sequence = [1,2,3,4] producer(sequence, q) q.put(None) q.put(None) cons_p1.join() cons_p2.join()

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into consumer: Tue Aug 13 19:52:08 2019 pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q Out of consumer: Tue Aug 13 19:52:08 2019 Into consumer: Tue Aug 13 19:52:08 2019 Out of consumer: Tue Aug 13 19:52:08 2019

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

任务调度 多线程

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。