https://github.com/cloudera/flume/blob/master/flume-docs/src/docs/UserGuide/Introduction

=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.
=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.

three supported reliability levels: * End-to-end * Store on failure * Best effort的更多相关文章

  1. SignalR Supported Platforms -摘自网络

    SignalR is supported under a variety of server and client configurations. In addition, each transpor ...

  2. store操作

    store.remove(rs); store.sync({ success: function (e, opt) { this.store.commitChanges(); }, failure: ...

  3. extjs 解决使用store.sync()方法更新item有时不触发后台action的问题

    问题描述: extjs 解决使用store.sync()方法更新item有时不触发后台action,不出发后台action的原因是item的字段值没有变化 解决方法: item.setDirty(tr ...

  4. PMP用语集

    AC actual cost 实际成本 ACWP actual cost of work performed 已完工作实际成本 BAC budget at completion 完工预算 BCWP b ...

  5. Solaris10安装配置LDAP(iPlanet Directory Server )

    Solaris10安装光盘自带了iPlanet Directory Server安装包,系统管理员可以利用iPlanet Directory Server在Solaris系统创建一个LDAP Serv ...

  6. memory ordering 内存排序

    Memory ordering - Wikipedia https://en.wikipedia.org/wiki/Memory_ordering https://zh.wikipedia.org/w ...

  7. Web测试介绍2一 安全测试

            安全测试是在IT软件产品的生命周期中,特别是产品开发基本完成到发布阶段,对产品进行检验以验证产品符合安全需求定义和产品质量标准的过程. 主要安全需求包括: (i) 认证 Authent ...

  8. Python 3.6.0的sqlite3模块无法执行VACUUM语句

    Python 3.6.0的sqlite3模块存在一个bug(见issue 29003),无法执行VACUUM语句. 一执行就出现异常: Traceback (most recent call last ...

  9. Flume1.5.0的安装、部署、简单应用(含伪分布式、与hadoop2.2.0、hbase0.96的案例)

    目录: 一.什么是Flume? 1)flume的特点 2)flume的可靠性 3)flume的可恢复性 4)flume 的 一些核心概念 二.flume的官方网站在哪里? 三.在哪里下载? 四.如何安 ...

随机推荐

  1. 【Visual Studio】工程类型选择win32还是x64 ?

    1. 如果将Visual Studio的工程类型设置为win32,那么我们开发的程序既能在32位操作系统上运行,也能在64位操作系统上运行. 2. 如果将Visual Studio的工程类型设置为x6 ...

  2. YUV和RGB格式分析【转】

    转自:http://www.cnblogs.com/silence-hust/p/4465354.html 做嵌入式项目的时候,涉及到YUV视频格式到RGB图像的转换,虽然之前有接触到RGB到都是基于 ...

  3. andriod多线程

    用ThreadHandle可以实现多线程,然后再主线程更新UI 第二种就是用 AsyncTask 具体看代码 public void onClick(View v) { new DownloadIma ...

  4. 浅谈JS的arguments对象

    在JavaScript中,arguments属于当前对象的一个内置属性,arguments非常类似于Array对象,但是又不是实例的Array.比如: Array.prototype.testArg ...

  5. HDU 2767:Proving Equivalences(强连通)

    题意: 一个有向图,问最少加几条边,能让它强连通 方法: 1:tarjan 缩点 2:采用如下构造法: 缩点后的图找到所有头结点和尾结点,那么,可以这么构造:把所有的尾结点连一条边到头结点,就必然可以 ...

  6. js-classList修改class属性

    classList定义与用法 1)classList属性返回元素的类名,作为DOMTokenList对象 2)该属性用于在元素中添加,移除及切换css类 3)classList属性是只读的,但可以用a ...

  7. 你也可以当面霸-MVC的原理及特点

    MVC是面试中经常被问到问题,如果能把MVC的原理简单清楚的描述出来,肯定会在面试官的心目中加分. 如果在能画图的情况下,画出一张MVC的流程图,无疑能简化不少概念上的术语,如果不能也不要紧,只要把核 ...

  8. Codeforces Gym101502 A.Very Hard Question

    2017 JUST Programming Contest 3.0 昨天的训练赛,打的好难过,因为被暴打了,写了8题,他们有的写了9题,差了一道dp,博客上写7道题的题解. 因为有一道是套板子过的,并 ...

  9. python学习笔记1-numpy/enumerate

    1. np.size和np.prod import numpy as np x = np.zeros((3, 5, 2), dtype=np.complex128) # ndarray.size is ...

  10. Could not change executable permissions on the application

    I could solve it erasing an application that I had previously uploaded using the same Bundle Identif ...