https://github.com/cloudera/flume/blob/master/flume-docs/src/docs/UserGuide/Introduction

=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.
=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.

three supported reliability levels: * End-to-end * Store on failure * Best effort的更多相关文章

  1. SignalR Supported Platforms -摘自网络

    SignalR is supported under a variety of server and client configurations. In addition, each transpor ...

  2. store操作

    store.remove(rs); store.sync({ success: function (e, opt) { this.store.commitChanges(); }, failure: ...

  3. extjs 解决使用store.sync()方法更新item有时不触发后台action的问题

    问题描述: extjs 解决使用store.sync()方法更新item有时不触发后台action,不出发后台action的原因是item的字段值没有变化 解决方法: item.setDirty(tr ...

  4. PMP用语集

    AC actual cost 实际成本 ACWP actual cost of work performed 已完工作实际成本 BAC budget at completion 完工预算 BCWP b ...

  5. Solaris10安装配置LDAP(iPlanet Directory Server )

    Solaris10安装光盘自带了iPlanet Directory Server安装包,系统管理员可以利用iPlanet Directory Server在Solaris系统创建一个LDAP Serv ...

  6. memory ordering 内存排序

    Memory ordering - Wikipedia https://en.wikipedia.org/wiki/Memory_ordering https://zh.wikipedia.org/w ...

  7. Web测试介绍2一 安全测试

            安全测试是在IT软件产品的生命周期中,特别是产品开发基本完成到发布阶段,对产品进行检验以验证产品符合安全需求定义和产品质量标准的过程. 主要安全需求包括: (i) 认证 Authent ...

  8. Python 3.6.0的sqlite3模块无法执行VACUUM语句

    Python 3.6.0的sqlite3模块存在一个bug(见issue 29003),无法执行VACUUM语句. 一执行就出现异常: Traceback (most recent call last ...

  9. Flume1.5.0的安装、部署、简单应用(含伪分布式、与hadoop2.2.0、hbase0.96的案例)

    目录: 一.什么是Flume? 1)flume的特点 2)flume的可靠性 3)flume的可恢复性 4)flume 的 一些核心概念 二.flume的官方网站在哪里? 三.在哪里下载? 四.如何安 ...

随机推荐

  1. es6总结(十一)--class & decorator

  2. 14深入理解C指针之---指针与内存分配

    一.内存的含义 1.定义:物理内存是固定,逻辑内存是非常灵活的 2.特征: 1).逻辑内存最终必须映射到物理内存才能作用 2).逻辑内存每次映射可能到不同的物理内存 3).逻辑内存与物理内存的链接通过 ...

  3. Beyond compare vs kdiff3

    這裡使用的 kdiff3 版本是 0.9.98   基於以下 三點,最終選擇了 beyond compare   1. kdiff3 不能刪檔案,     以下為例,不能刪1   2. kdiff3 ...

  4. Python入门--17--pickle

    pickle模块,实际用途也很简单.单一,主要用于保存列表(list),元祖(Tuple),字典(dictionary) 当然,一定是大的列表.字典什么的,成千上万行的字典.列表,才能凸显出它的用武之 ...

  5. poj 1970(搜索)

    The Game Time Limit: 1000MS   Memory Limit: 30000K Total Submissions: 6247   Accepted: 1601 Descript ...

  6. poj 3281(构图+网络流)

    Dining Time Limit: 2000MS   Memory Limit: 65536K Total Submissions: 14144   Accepted: 6425 Descripti ...

  7. DB2 导出数据文件

    export to c:/frameno2.del of del + [SQL 语句]

  8. vs-react-插件

  9. 最小生成树-prim算法模板

    题目描述 如题,给出一个无向图,求出最小生成树,如果该图不连通,则输出orz 输入输出格式 输入格式: 第一行包含两个整数N.M,表示该图共有N个结点和M条无向边.(N<=5000,M<= ...

  10. 查找系列合集-二叉查找树BST

    一. 二叉树 1. 什么是二叉树? 在计算机科学中,二叉树是每个结点最多有两个子树的树结构. 通常子树被称作“左子树”(left subtree)和“右子树”(right subtree). 二叉树常 ...