golang schedule crash
golang 起超过100W的goroutine就会crash,这个算不算是个bug?
2036119xxxx
panic: inconsistent poll.fdMutex
goroutine 2041065 [running]:
internal/poll.(*fdMutex).rwlock(0xc420090050, 0xc517305500, 0x1089b36)
/usr/local/Cellar/go/1.10/libexec/src/internal/poll/fd_mutex.go:145 +0x13f
internal/poll.(*FD).writeLock(0xc420090050, 0x1, 0x10c1cee)
/usr/local/Cellar/go/1.10/libexec/src/internal/poll/fd_mutex.go:237 +0x32
internal/poll.(*FD).Write(0xc420090050, 0xc518138ee0, 0x7, 0x8, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.10/libexec/src/internal/poll/fd_unix.go:243 +0x46
os.(*File).write(0xc42000c018, 0xc518138ee0, 0x7, 0x8, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.10/libexec/src/os/file_unix.go:243 +0x4e
os.(*File).Write(0xc42000c018, 0xc518138ee0, 0x7, 0x8, 0x8, 0xc517a3bd50, 0x0)
/usr/local/Cellar/go/1.10/libexec/src/os/file.go:144 +0x6f
fmt.Fprint(0x10d2720, 0xc42000c018, 0xc5173057b8, 0x1, 0x1, 0x109ed40, 0x0, 0xc518138ed8)
/usr/local/Cellar/go/1.10/libexec/src/fmt/print.go:223 +0x8b
fmt.Print(0xc5173057b8, 0x1, 0x1, 0xc518138ed8, 0x0, 0x0)
/usr/local/Cellar/go/1.10/libexec/src/fmt/print.go:232 +0x57
main.test(0x1f24c1)
/Users/guanshuai/go/src/main.go:11 +0x81
created by main.main
/Users/guanshuai/go/src/main.go:19 +0x87
exit status 2
real 0m10.859s
user 0m29.635s
sys 0m10.859s
[guanshuai] src $
mac os
goroutine 15192446 [running]:
internal/poll.(*fdMutex).rwlock(0xc420080050, 0x1000000010bf000, 0x0)
/usr/local/go/src/internal/poll/fd_mutex.go:145 +0x13f
internal/poll.(*FD).writeLock(0xc420080050, 0xc420034800, 0xc4ddd4cc40)
/usr/local/go/src/internal/poll/fd_mutex.go:237 +0x32
internal/poll.(*FD).Write(0xc420080050, 0xc4ddcea480, 0x11, 0x20, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:243 +0x46
os.(*File).write(0xc42000c018, 0xc4ddcea480, 0x11, 0x20, 0x10, 0x10, 0x11)
/usr/local/go/src/os/file_unix.go:243 +0x4e
os.(*File).Write(0xc42000c018, 0xc4ddcea480, 0x11, 0x20, 0x8, 0xc4531795a0, 0x0)
/usr/local/go/src/os/file.go:144 +0x6f
fmt.Fprintln(0x10d09a0, 0xc42000c018, 0xc4ddd7e7a8, 0x2, 0x2, 0x109d300, 0x0, 0xc4531795a0)
/usr/local/go/src/fmt/print.go:255 +0x8b
fmt.Println(0xc4ddd7e7a8, 0x2, 0x2, 0xc4531795a0, 0x0, 0x0)
/usr/local/go/src/fmt/print.go:264 +0x57
// lock adds a reference to mu and locks mu.
// It reports whether mu is available for reading or writing.
func (mu *fdMutex) rwlock(read bool) bool {
var mutexBit, mutexWait, mutexMask uint64
var mutexSema *uint32
if read {
mutexBit = mutexRLock
mutexWait = mutexRWait
mutexMask = mutexRMask
mutexSema = &mu.rsema
} else {
mutexBit = mutexWLock
mutexWait = mutexWWait
mutexMask = mutexWMask
mutexSema = &mu.wsema
}
for {
old := atomic.LoadUint64(&mu.state)
if old&mutexClosed != 0 {
return false
}
var new uint64
if old&mutexBit == 0 {
// Lock is free, acquire it.
new = (old | mutexBit) + mutexRef
if new&mutexRefMask == 0 {
panic("inconsistent poll.fdMutex")
}
} else {
// Wait for lock.
new = old + mutexWait
if new&mutexMask == 0 {
panic("inconsistent poll.fdMutex")
}
}
if atomic.CompareAndSwapUint64(&mu.state, old, new) {
if old&mutexBit == 0 {
return true
}
runtime_Semacquire(mutexSema)
// The signaller has subtracted mutexWait.
}
}
}
// 1 bit - lock for write operations.
// 20 bits - total number of references (read+write+misc).
// 20 bits - number of outstanding read waiters.
// 20 bits - number of outstanding write waiter
// fdMutex.state is organized as follows:
// 1 bit - whether FD is closed, if set all subsequent lock operations will fail.
// 1 bit - lock for read operations.
// 1 bit - lock for write operations.
// 20 bits - total number of references (read+write+misc).
// 20 bits - number of outstanding read waiters.
// 20 bits - number of outstanding write waiters.
const (
mutexClosed = 1 << 0
mutexRLock = 1 << 1
mutexWLock = 1 << 2
mutexRef = 1 << 3
mutexRefMask = (1<<20 - 1) << 3
mutexRWait = 1 << 23
mutexRMask = (1<<20 - 1) << 23
mutexWWait = 1 << 43
mutexWMask = (1<<20 - 1) << 43
)
互斥的变量里面是64位的数值
1.bit 代表是否关闭
2bit 锁的类型是读
3bit 锁的类型是写
4bit 可以递增的单位数值
5bit-- (22位) 掩码 2**22 = 2 * 2 ** 20
>>> import math
>>> pow(2, 20)
1048576
>>>
在一个file 或者socket上最大的锁为1000000, 所以有人提交patch,提示更精细的panic
https://go-review.googlesource.com/c/go/+/119956
internal/poll: better panic message for lock overflow
const overflowMsg = "too many concurrent operations on a single file or socket (max 1000000)"
golang schedule crash的更多相关文章
- Go语言(golang)开源项目大全
转http://www.open-open.com/lib/view/open1396063913278.html内容目录Astronomy构建工具缓存云计算命令行选项解析器命令行工具压缩配置文件解析 ...
- [转]Go语言(golang)开源项目大全
内容目录 Astronomy 构建工具 缓存 云计算 命令行选项解析器 命令行工具 压缩 配置文件解析器 控制台用户界面 加密 数据处理 数据结构 数据库和存储 开发工具 分布式/网格计算 文档 编辑 ...
- Golang优秀开源项目汇总, 10大流行Go语言开源项目, golang 开源项目全集(golang/go/wiki/Projects), GitHub上优秀的Go开源项目
Golang优秀开源项目汇总(持续更新...)我把这个汇总放在github上了, 后面更新也会在github上更新. https://github.com/hackstoic/golang-open- ...
- 面试必问:Golang高阶-Golang协程实现原理
引言 实现并发编程有进程,线程,IO多路复用的方式.(并发和并行我们这里不区分,如果CPU是多核的,可能在多个核同时进行,我们叫并行,如果是单核,需要排队切换,我们叫并发) 进程和线程的区别 进程是计 ...
- Ubuntu14.04+RabbitMQ3.6.3+Golang的最佳实践
目录 [TOC] 1.RabbitMQ介绍 1.1.什么是RabbitMQ? RabbitMQ 是由 LShift 提供的一个 Advanced Message Queuing Protocol ...
- 【GoLang】GoLang 的流程与函数
003.GO流程与函数 1 概述 1.1 Go中流程控制分三大类:条件判断,循环控制和无条件跳转 2 流程 2.1 if 2.1.1 if条件判断语句中不需要括号 2.1.2 条件判断语句里面允许声明 ...
- Golang网络库中socket阻塞调度源码剖析
本文分析了Golang的socket文件描述符和goroutine阻塞调度的原理.代码中大部分是Go代码,小部分是汇编代码.完整理解本文需要Go语言知识,并且用Golang写过网络程序.更重要的是,需 ...
- Golang源码探索(二) 协程的实现原理
Golang最大的特色可以说是协程(goroutine)了, 协程让本来很复杂的异步编程变得简单, 让程序员不再需要面对回调地狱, 虽然现在引入了协程的语言越来越多, 但go中的协程仍然是实现的是最彻 ...
- Golang源码探索(三) GC的实现原理
Golang从1.5开始引入了三色GC, 经过多次改进, 当前的1.9版本的GC停顿时间已经可以做到极短. 停顿时间的减少意味着"最大响应时间"的缩短, 这也让go更适合编写网络服 ...
随机推荐
- x-www-form-urlencoded与multipart/form-data区别
转载声明: http://blog.chinaunix.net/uid-7210505-id-329700.html 在Flex中,UrlRequest中的contentType默认值为 applic ...
- docker容器运行与退出
#下载centos镜像,运行一个名为mycentos的容器,并在容器里运行/bin/bash docker run -ti --name mycentos centos /bin/bash #退出 e ...
- c++ 类的堆成员的声明及使用
_reg = new boost::regex("aoe "); boost::regex_search(line, what, *_reg)
- 【OCP、OCM、高可用等】小麦苗课堂网络班招生简章(从入门到专家)--课程大纲
[OCP.OCM.高可用等]小麦苗课堂网络班招生简章(从入门到专家)--课程大纲 小麦苗信息 我的个人信息 网名:小麦苗 QQ:646634621 QQ群:618766405 我的博客:http:// ...
- python rabbitmq的库,rabbitpy代替pika
之前看网上都是清一色pika包的例子,就用的pika包,最大问题是非多线程安全,改为使用rabbitpy.大幅改善了pika多线程需要加锁,和外网推送延迟又不能开多线程导致推送慢的问题. rabbit ...
- C# StackExchange.Redis 用法总结
安装 StackExchange.Redis 在 NuGet 中搜索 StackExchange.Redis 和 Newtonsoft.Json,直接点击按钮安装即可. StackExchange.R ...
- [Educational Round 3][Codeforces 609E. Minimum spanning tree for each edge]
这题本来是想放在educational round 3的题解里的,但觉得很有意思就单独拿出来写了 题目链接:609E - Minimum spanning tree for each edge 题目大 ...
- Spring 注解配置(2)——@Autowired
版权声明:本文为博主原创文章,如需转载请标注转载地址. 博客地址:http://www.cnblogs.com/caoyc/p/5626365.html @Autowired 注释,它可以对类成员变 ...
- 一、HTML基础学习
1.基本格式<html> <head><title></title></head> <body></body>< ...
- java 三大框架 hibernate部分知识实现增删该查操作
1.三层架构 表现层 web层(MVC是一个表现层的设计模型) 业务层 service层 持久层 dao层2.三大框架和三层架构的关系(建议学习三大框架的顺序:先学习hibernat ...