flink问题1

报错:

More buffers requested available than totally available

查看源码:

/**
* This method makes sure that at least a certain number of memory segments is in the list of free segments.
* Free memory can be in the list of free segments, or in the return-queue where segments used to write behind are
* put. The number of segments that are in that return-queue, but are actually reclaimable is tracked. This method
* makes sure at least a certain number of buffers is reclaimed.
*
* @param minRequiredAvailable The minimum number of buffers that needs to be reclaimed.
*/
final void ensureNumBuffersReturned(final int minRequiredAvailable) {
if (minRequiredAvailable > this.availableMemory.size() + this.writeBehindBuffersAvailable) {
throw new IllegalArgumentException("More buffers requested available than totally available.");
} try {
while (this.availableMemory.size() < minRequiredAvailable) {
this.availableMemory.add(this.writeBehindBuffers.take());
this.writeBehindBuffersAvailable--;
}
}
catch (InterruptedException iex) {
throw new RuntimeException("Hash Join was interrupted.");
}
}

原因:

taskmanager.memory.managed.size: 512M

这里设置的过小,恢复默认配置

Flink问题1的更多相关文章

  1. apache flink 入门

    配置环境 包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...

  2. Flink 1.1 – ResourceManager

    Flink resource manager的作用如图,   FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...

  3. Apache Flink初接触

    Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...

  4. Flink - InstanceManager

    InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...

  5. Flink – window operator

      参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

  6. Flink – Trigger,Evictor

    org.apache.flink.streaming.api.windowing.triggers;   Trigger public abstract class Trigger<T, W e ...

  7. Flink - RocksDBStateBackend

    如果要考虑易用性和效率,使用rocksDB来替代普通内存的kv是有必要的 有了rocksdb,可以range查询,可以支持columnfamily,可以各种压缩 但是rocksdb本身是一个库,是跑在 ...

  8. Flink - state管理

    在Flink – Checkpoint 没有描述了整个checkpoint的流程,但是对于如何生成snapshot和恢复snapshot的过程,并没有详细描述,这里补充   StreamOperato ...

  9. Flink - state

      public class StreamTaskState implements Serializable, Closeable { private static final long serial ...

  10. Stream Processing for Everyone with SQL and Apache Flink

    Where did we come from? With the 0.9.0-milestone1 release, Apache Flink added an API to process rela ...

随机推荐

  1. 总是说spring难学?来看完这些spring的注解及其解释,真香!

    前言 用过spring的人都知道,spring简单的通过注解就可以完成很多事情,但这些东西是如何实现的呢以及如何应用到我们自己的代码中?接下来,让我们一起开启注解的旅程. 1. @Controller ...

  2. ABBYY FineReader 14如何查看PDF文档

    使用 ABBYY FineReader,您可以轻松查看和编辑任何类型的 PDF文档,就像是一款功能强大的PDF编辑转换器,不仅如此,它还能够允许您复制其中的文本.图片和表格.本文我们来看看如何从&qu ...

  3. Guitar Pro吉他指弹入门——特殊调弦

    本期文章中,我们将通过吉他打谱软件Guitar Pro 7来向大家讲解指弹曲目中所涉及的特殊调弦. 作为一个吉他手,在练琴的时候总会遇到各种各样的问题,比如说鼓手不肯跟你合作(因为打鼓往往不能露脸), ...

  4. 无效的HTTP_主机头Invalid HTTP_HOST header: '192.168.56.100:8888'. You may need to add '192.168.56.100' to ALLOWED_HOSTS.

    Invalid HTTP_HOST header: '192.168.56.100:8888'. You may need to add '192.168.56.100' to ALLOWED_HOS ...

  5. Python爬虫实现翻译功能

    前言 学了这么久的python理论知识,需要开始实战来练手巩固了. 准备 首先安装爬虫urllib库 pip install urllib 获取有道翻译的链接url 需要发送的参数在form data ...

  6. Vim常用按键

  7. Contest 982

    A 直接模拟即可,为了方便边界判断建议用 !=. 时间复杂度 \(O\left(n\right)\). B \(w\) 排序来处理内向者,坐人后丢进大根堆来处理外向者. 时间复杂度 \(O\left( ...

  8. 使用Python过程出现的细节问题:TypeError: not enough arguments for format string

    今天使用字符串格式化时,遇到的一点小问题:调用这个方法解释器出错了:TypeError: not enough arguments for format string def ll(name,age) ...

  9. 为什么不用UUID做主键?

    不易于存储:UUID太长,16字节128位,通常以36长度的字符串表示,很多场景不适用. 信息不安全:基于MAC地址生成UUID的算法可能会造成MAC地址泄露,这个漏洞曾被用于寻找梅丽莎病毒的制作者位 ...

  10. LeetCode 047 Permutations II

    题目要求:Permutations II Given a collection of numbers that might contain duplicates, return all possibl ...