0.目录

2. threading.Lock() 的必要性
3.观察block
4.threading.RLock() 的应用场景

1.参考

Thread Synchronization Mechanisms in Python

count += 1 不是原子操作，三步操作可能被中断，通过lock将三步操作“封装”为一步操作，要么执行，要么不执行。

counter = 0

def process_item(item):

    global counter

    ... do something with item ...

    counter += 1

# The reason for this is that the increment operation is actually executed in three steps;

#first, the interpreter fetches the current value of the counter,

# then it calculates the new value,

# and finally, it writes the new value back to the variable.

Atomic Operations #

The simplest way to synchronize access to shared variables or other resources is to rely on atomic operations in the interpreter.

An atomic operation is an operation that is carried out in a single execution step, without any chance that another thread gets control.

What kinds of global value mutation are thread-safe?

python的原子操作

A global interpreter lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via sys.setcheckinterval(). Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python program.

In theory, this means an exact accounting requires an exact understanding of the PVM bytecode implementation. In practice, it means that operations on shared variables of built-in data types (ints, lists, dicts, etc) that “look atomic” really are.

For example, the following operations are all atomic (L, L1, L2 are lists, D, D1, D2 are dicts, x, y are objects, i, j are ints):

L.append(x)

L1.extend(L2)

x = L[i]

x = L.pop()

L1[i:j] = L2

L.sort()

x = y

x.field = y

D[x] = y

D1.update(D2)

D.keys()

These aren’t:

i = i+1

L.append(L[-1])

L[i] = L[j]

D[x] = D[x] + 1

Operations that replace other objects may invoke those other objects’ __del__() method when their reference count reaches zero, and that can affect things. This is especially true for the mass updates to dictionaries and lists. When in doubt, use a mutex!

2. threading.Lock()  的必要性

#!usr/bin/env python

#coding:utf-8

import sys

import time

import random

import logging

import threading

import Queue

lock = threading.Lock()     #'function-call ownership'

rlock = threading.RLock()   #thread ownership

logging.basicConfig(level=logging.DEBUG,

                    format = '%(asctime)s - %(threadName)-10s - %(levelname)s - %(message)s')

logger = logging.getLogger()

count = 0

class MyThread(threading.Thread):

    def __init__(self):

        threading.Thread.__init__(self)

    def run(self):

        global count

        for i in range(100):

            count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))     

def main():

    logger.debug('initial count: {}'.format(count))

    thread_list = [MyThread() for i in range(2)]

    for t in thread_list:

        t.start()

    for t in thread_list:

        t.join()   

    logger.debug('final count: {}'.format(count))

if __name__ == '__main__':

    main()

修改run函数代码的不同输出：

    def run(self):

        global count

        for i in range(100):

            count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))

# 在切换线程之前，某一线程已经完成，两个线程顺序完成，结果几乎不会有误

# 2017-08-20 12:19:30,857 - MainThread - DEBUG - initial count: 0

# 2017-08-20 12:19:30,858 - Thread-1   - DEBUG - Thread-1 finished, count is 100

# 2017-08-20 12:19:30,858 - Thread-2   - DEBUG - Thread-2 finished, count is 200

# 2017-08-20 12:19:30,858 - MainThread - DEBUG - final count: 200        

        time.sleep(0.001)

        for i in range(100):

            count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))

# 开头sleep导致两个线程几乎同时启动，结果可能有误

# 2017-08-20 12:24:59,046 - MainThread - DEBUG - initial count: 0

# 2017-08-20 12:24:59,048 - Thread-2   - DEBUG - Thread-2 finished, count is 124

# 2017-08-20 12:24:59,048 - Thread-1   - DEBUG - Thread-1 finished, count is 153

# 2017-08-20 12:24:59,048 - MainThread - DEBUG - final count: 153

        for i in range(10000):

            count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))

# bytecodes足够导致两个线程交替运行，结果大概率有误

# 2017-08-20 12:20:17,719 - MainThread - DEBUG - initial count: 0

# 2017-08-20 12:20:17,723 - Thread-1   - DEBUG - Thread-1 finished, count is 12438

# 2017-08-20 12:20:17,723 - Thread-2   - DEBUG - Thread-2 finished, count is 12616

# 2017-08-20 12:20:17,723 - MainThread - DEBUG - final count: 12616        

        with lock:

            for i in range(10000):

                count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))

# lock直到某一线程完成，结果正确

# 2017-08-20 12:20:37,630 - MainThread - DEBUG - initial count: 0

# 2017-08-20 12:20:37,631 - Thread-1   - DEBUG - Thread-1 finished, count is 10000

# 2017-08-20 12:20:37,632 - Thread-2   - DEBUG - Thread-2 finished, count is 20000

# 2017-08-20 12:20:37,634 - MainThread - DEBUG - final count: 20000

        for i in range(10000):

            with lock:

                count += 1

        logger.debug('{} finished, count is {}'.format(self.name, count))

# 两个线程交替lock，结果正确

# 2017-08-20 12:21:03,921 - MainThread - DEBUG - initial count: 0

# 2017-08-20 12:21:03,973 - Thread-1   - DEBUG - Thread-1 finished, count is 19979

# 2017-08-20 12:21:03,973 - Thread-2   - DEBUG - Thread-2 finished, count is 20000

# 2017-08-20 12:21:03,973 - MainThread - DEBUG - final count: 20000

3.观察block

    def run(self):

        global count

        all = range(10000)  #确保每个线程 +1 的次数

        while all != []:

            if not lock.acquire(False):  #假设没有参数会导致block，则马上返回false当不block；否则返回true且acquire

                logger.debug('{} wait...{}'.format(self.name, len(all)))

            else:

                try:

                    count += 1

                    all.pop()

                except Exception as err:

                    logger.debug('{} err, count is {}'.format(self.name, count))

                finally:

                    # logger.debug('{} release {} {}'.format(self.name, count, len(all)))  #导致两个线程顺序执行？？？

                    lock.release()

        logger.debug('{} finished, count is {}'.format(self.name, count))

输出：

2017-08-20 12:32:55,204 - MainThread - DEBUG - initial count: 0

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,210 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,211 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,213 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,214 - Thread-1   - DEBUG - Thread-1 wait...9925

2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,216 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,217 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,219 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,220 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,221 - Thread-2   - DEBUG - Thread-2 wait...6036

2017-08-20 12:32:55,226 - Thread-1   - DEBUG - Thread-1 finished, count is 13964

2017-08-20 12:32:55,236 - Thread-2   - DEBUG - Thread-2 finished, count is 20000

2017-08-20 12:32:55,236 - MainThread - DEBUG - final count: 20000

4.threading.RLock() 的应用场景

Problems with Simple Locking

lock = threading.Lock()

def get_first_part():

    with lock: # any thread that attempts to acquire the lock will block, even if the same thread is already holding the lock.

        ... fetch data for first part from shared object

    return data    

def get_second_part():

    with lock:

        ... fetch data for second part from shared object

    return data  

def get_both_parts():

    with lock:  # other thread may modify the resource between the two calls

        first = get_first_part()

        # between the two calls

        second = get_second_part()

    return first, second    

# While simple locks will block if the same thread attempts to acquire the same lock twice,

# a re-entrant lock only blocks if another thread currently holds the lock.

rlock = threading.RLock()

python之多线程 threading.Lock() 和 threading.RLock()的更多相关文章

  1. python的threading的使用（join方法，多线程，锁threading.Lock和threading.Condition

    一.开启多线程方法一 import threading,time def write1(): for i in range(1,5): print('1') time.sleep(1) def wri ...

  2. python爬虫——多线程+协程（threading+gevent）

    上一篇博客中我介绍了如何将爬虫改造为多进程爬虫,但是这种方法对爬虫效率的提升不是非常明显,而且占用电脑cpu较高,不是非常适用于爬虫.这篇博客中,我将介绍在爬虫中广泛运用的多线程+协程的解决方案,亲测 ...

  3. 基于Python的多线程模块Threading小结

    步入正题前,先准备下基本知识,线程与进程的概念. 相信作为一个测试人员,如果从理论概念上来说其两者的概念或者区别,估计只会一脸蒙蔽,这里就举个例子来说明下其中的相关概念. 平安夜刚过,你是吃到了苹果还 ...

  4. Python：多线程编程

    1.IO编程 IO(input/output).凡是用到数据交换的地方,都会涉及io编程,例如磁盘,网络的数据传输.在IO编程中,stream(流)是一种重要的概念,分为输入流(input strea ...

  5. 孤荷凌寒自学python第三十九天python 的线程锁Lock

    孤荷凌寒自学python第三十九天python的线程锁Lock (完整学习过程屏幕记录视频地址在文末,手写笔记在文末) 当多个线程同时操作一个文件等需要同时操作某一对象的情况发生时,很有可能发生冲突, ...

  6. Day12- Python基础12 线程、GIL、Lock锁、RLock锁、Semaphore锁、同步条件event

    http://www.cnblogs.com/yuanchenqi/articles/6248025.html  博客地址 本节内容: 1:进程和线程的说明 2:线程的两种调用方式 3:threadi ...

  7. python中多线程相关

    基础知识 进程:进程就是一个程序在一个数据集上的一次动态执行过程 数据集:程序执行过程中需要的资源 进程控制块:完成状态保存的单元 线程:线程是寄托在进程之上,为了提高系统的并发性 线程是进程的实体 ...

  8. python网络-多线程(22)

    一.什么是线程 线程(英语:thread)是操作系统能够进行运算调度的最小单位.它被包含在进程之中,是进程中的实际运作单位.同一进程中的多条线程将共享该进程中的全部系统资源,一个进程可以有很多线程,每 ...

  9. python 使用多线程进行并发编程/互斥锁的使用

    import threading import time """ python的thread模块是比较底层的模块,python的threading模块是对thread做了 ...

随机推荐

  1. MFCWinInet学习

    http://blog.csdn.net/segen_jaa/article/details/6278167 背景: 功能:服务端下载文件 服务端:用Java写Sevlet进行有效性验证 客户端:用C ...

  2. 其他-n个互相独立的连续随机变量中第i小的数值期望

    提出问题 有\(n\)个互相独立的\(0\)至\(1\)之间等概率生成的随机变量,求从小到大排序后第\(i\)个数的数值期望 一个简化的问题 我们先来求解一个简化的问题:最大值的数值期望是多少? 我们 ...

  3. UVALive 8513 lovers 2017 西安区域赛 B 贪心+multiset

    UVALive 8513 有2种人,每个人有自己的权值$A_i$ $B_i$ 当$A_i + B_i >=K$时 两个人可以配对 问最多多少人可以配对 解法 : 把$/{ A_i /}$ 排序 ...

  4. 训练报告 (2014-2015) 2014, Samara SAU ACM ICPC Quarterfinal Qualification Contest

    Solved A Gym 100488A Yet Another Goat in the Garden   B Gym 100488B Impossible to Guess Solved C Gym ...

  5. Winform按键捕获

    参考:http://blog.csdn.net/zhensoft163/article/details/4239796 下载链接 方法1:使用窗体的 KeyDown 事件 private void F ...

  6. Mysql --初识mysql语句

    本节课先对mysql的基本语法初体验. 操作文件夹(库) 增 create database db1 charset utf8;#db1 是数据库的名字 也就是文件夹 查 # 查看当前创建的数据库 s ...

  7. 开通博客的第一天上传我的C#基础笔记。

    1.索引器  string arrStr = "sddfdfgfh";  索引器的目的就是为了方便而已,可以在该类型的对象后面直接写[]访问该对象里面的成员  Console.Wr ...

  8. Tornado学习笔记(一) helloword/多进程/启动参数

    前言 当你觉得你过得很舒服的时候,你肯定没有在进步.所以我想学习新的东西,然后选择了Tornado.因为我觉得Tornado更匹配目前的我的综合素质. Tornado学习笔记系列主要参考<int ...

  9. python-异常处理、元类

    一.异常处理 1.异常处理介绍: 异常是错误发生的信号,一旦程序出错就会产生一个异常,如果该异常没有被应用程序处理,那么该异常就会被抛出来,程序执行随之停止 2.异常通常包含三个部分 1.traceb ...

  10. Elasticsearch入门，这一篇就够了

    实时搜索引擎Elasticsearch Elasticsearch(简称ES)是一个基于Apache Lucene(TM)的开源搜索引擎,无论在开源还是专有领域,Lucene可以被认为是迄今为止最先进 ...