Flink问题1
flink问题1
报错:
More buffers requested available than totally available
查看源码:
/**
* This method makes sure that at least a certain number of memory segments is in the list of free segments.
* Free memory can be in the list of free segments, or in the return-queue where segments used to write behind are
* put. The number of segments that are in that return-queue, but are actually reclaimable is tracked. This method
* makes sure at least a certain number of buffers is reclaimed.
*
* @param minRequiredAvailable The minimum number of buffers that needs to be reclaimed.
*/
final void ensureNumBuffersReturned(final int minRequiredAvailable) {
if (minRequiredAvailable > this.availableMemory.size() + this.writeBehindBuffersAvailable) {
throw new IllegalArgumentException("More buffers requested available than totally available.");
}
try {
while (this.availableMemory.size() < minRequiredAvailable) {
this.availableMemory.add(this.writeBehindBuffers.take());
this.writeBehindBuffersAvailable--;
}
}
catch (InterruptedException iex) {
throw new RuntimeException("Hash Join was interrupted.");
}
}
原因:
taskmanager.memory.managed.size: 512M
这里设置的过小,恢复默认配置
Flink问题1的更多相关文章
- apache flink 入门
配置环境 包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
- Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
- Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
- Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
- Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...
- Flink – Trigger,Evictor
org.apache.flink.streaming.api.windowing.triggers; Trigger public abstract class Trigger<T, W e ...
- Flink - RocksDBStateBackend
如果要考虑易用性和效率,使用rocksDB来替代普通内存的kv是有必要的 有了rocksdb,可以range查询,可以支持columnfamily,可以各种压缩 但是rocksdb本身是一个库,是跑在 ...
- Flink - state管理
在Flink – Checkpoint 没有描述了整个checkpoint的流程,但是对于如何生成snapshot和恢复snapshot的过程,并没有详细描述,这里补充 StreamOperato ...
- Flink - state
public class StreamTaskState implements Serializable, Closeable { private static final long serial ...
- Stream Processing for Everyone with SQL and Apache Flink
Where did we come from? With the 0.9.0-milestone1 release, Apache Flink added an API to process rela ...
随机推荐
- 总是说spring难学?来看完这些spring的注解及其解释,真香!
前言 用过spring的人都知道,spring简单的通过注解就可以完成很多事情,但这些东西是如何实现的呢以及如何应用到我们自己的代码中?接下来,让我们一起开启注解的旅程. 1. @Controller ...
- ABBYY FineReader 14如何查看PDF文档
使用 ABBYY FineReader,您可以轻松查看和编辑任何类型的 PDF文档,就像是一款功能强大的PDF编辑转换器,不仅如此,它还能够允许您复制其中的文本.图片和表格.本文我们来看看如何从&qu ...
- Guitar Pro吉他指弹入门——特殊调弦
本期文章中,我们将通过吉他打谱软件Guitar Pro 7来向大家讲解指弹曲目中所涉及的特殊调弦. 作为一个吉他手,在练琴的时候总会遇到各种各样的问题,比如说鼓手不肯跟你合作(因为打鼓往往不能露脸), ...
- 无效的HTTP_主机头Invalid HTTP_HOST header: '192.168.56.100:8888'. You may need to add '192.168.56.100' to ALLOWED_HOSTS.
Invalid HTTP_HOST header: '192.168.56.100:8888'. You may need to add '192.168.56.100' to ALLOWED_HOS ...
- Python爬虫实现翻译功能
前言 学了这么久的python理论知识,需要开始实战来练手巩固了. 准备 首先安装爬虫urllib库 pip install urllib 获取有道翻译的链接url 需要发送的参数在form data ...
- Vim常用按键
- Contest 982
A 直接模拟即可,为了方便边界判断建议用 !=. 时间复杂度 \(O\left(n\right)\). B \(w\) 排序来处理内向者,坐人后丢进大根堆来处理外向者. 时间复杂度 \(O\left( ...
- 使用Python过程出现的细节问题:TypeError: not enough arguments for format string
今天使用字符串格式化时,遇到的一点小问题:调用这个方法解释器出错了:TypeError: not enough arguments for format string def ll(name,age) ...
- 为什么不用UUID做主键?
不易于存储:UUID太长,16字节128位,通常以36长度的字符串表示,很多场景不适用. 信息不安全:基于MAC地址生成UUID的算法可能会造成MAC地址泄露,这个漏洞曾被用于寻找梅丽莎病毒的制作者位 ...
- LeetCode 047 Permutations II
题目要求:Permutations II Given a collection of numbers that might contain duplicates, return all possibl ...