https://github.com/cloudera/flume/blob/master/flume-docs/src/docs/UserGuide/Introduction

=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.
=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.

three supported reliability levels: * End-to-end * Store on failure * Best effort的更多相关文章

  1. SignalR Supported Platforms -摘自网络

    SignalR is supported under a variety of server and client configurations. In addition, each transpor ...

  2. store操作

    store.remove(rs); store.sync({ success: function (e, opt) { this.store.commitChanges(); }, failure: ...

  3. extjs 解决使用store.sync()方法更新item有时不触发后台action的问题

    问题描述: extjs 解决使用store.sync()方法更新item有时不触发后台action,不出发后台action的原因是item的字段值没有变化 解决方法: item.setDirty(tr ...

  4. PMP用语集

    AC actual cost 实际成本 ACWP actual cost of work performed 已完工作实际成本 BAC budget at completion 完工预算 BCWP b ...

  5. Solaris10安装配置LDAP(iPlanet Directory Server )

    Solaris10安装光盘自带了iPlanet Directory Server安装包,系统管理员可以利用iPlanet Directory Server在Solaris系统创建一个LDAP Serv ...

  6. memory ordering 内存排序

    Memory ordering - Wikipedia https://en.wikipedia.org/wiki/Memory_ordering https://zh.wikipedia.org/w ...

  7. Web测试介绍2一 安全测试

            安全测试是在IT软件产品的生命周期中,特别是产品开发基本完成到发布阶段,对产品进行检验以验证产品符合安全需求定义和产品质量标准的过程. 主要安全需求包括: (i) 认证 Authent ...

  8. Python 3.6.0的sqlite3模块无法执行VACUUM语句

    Python 3.6.0的sqlite3模块存在一个bug(见issue 29003),无法执行VACUUM语句. 一执行就出现异常: Traceback (most recent call last ...

  9. Flume1.5.0的安装、部署、简单应用(含伪分布式、与hadoop2.2.0、hbase0.96的案例)

    目录: 一.什么是Flume? 1)flume的特点 2)flume的可靠性 3)flume的可恢复性 4)flume 的 一些核心概念 二.flume的官方网站在哪里? 三.在哪里下载? 四.如何安 ...

随机推荐

  1. hdu4183往返经过至多每个点一次/最大流

    题意:从s到t,每个点有f值,只能从f值小的到大的,到T后回来,只能从f值大的到 小的,求可行否. 往返,其实就是俩条路过去(每个点最多一次),所以想到流量为2,跑最大流,看是否满2,又要每个点最多一 ...

  2. 在IOS11中position:fixed弹出框中的input出现光标错位的问题

    问题出现的背景: 在IOS11中position:fixed弹出框中的input出现光标错位的问题 解决方案 一.设计交互方面最好不要让弹窗中出现input输入框: 二.前端处理此兼容性的方案思路: ...

  3. python笔记1:python基础

    1.Python模块的标准文件模板: #!/usr/bin/env python #第1行注释可以让这个 .py 文件直接在Unix/Linux/Mac上运行 # -*- coding: utf-8 ...

  4. vue报错之Duplicate keys detected: '0'. This may cause an update error.

    昨天运行vue项目的时候,出现了[Vue warn]: Duplicate keys detected: '0'. This may cause an update error(错误,检测到重复的ke ...

  5. SQL-基础学习使用的数据库资料

    -- ------------------------ Create Customers table-- ----------------------CREATE TABLE Customers( c ...

  6. 每天进步一点点—SQL优化

    一.           SQL优化 1.   通过show status 命令了解各种SQL的运行频率 mysql>show status like 'Com_%'; +----------- ...

  7. 全方位绕过软WAF攻略

    0×00 前言 现在软waf较为多,就在今年夏天苦逼挖洞的日子里经常遇到360主机卫士,安全狗,云锁之类的软waf进行拦截,经常碰到如下拦截提示: 看到以上三个拦截提示就让人头疼不已,欲罢不能. so ...

  8. SSH 原理和公匙私匙

    先主要介绍了Telnet.SSH 的通信原理,分析了其通信时的工作流程. Telnet 无论Telnet协议连接的是什么类型终端,都会转换为NVT(Net Virtual Terminal)格式进行通 ...

  9. intellij idea 的常用有用快捷键

    ctrl + R:替换(这一点和office 中的ctrl + H不一样) ctrl + alt + L:自动整理代码(不会整理注释文件) ctrl + alt:(自动导入包,不能批量导入,有人评论批 ...

  10. Unity3D:Gizmos画圆(原创)

    Unity3D引擎技术交流QQ群:[21568554] Gizmos是场景视图里的一个可视化调试工具. 在做项目过程中.我们常常会用到它,比如:绘制一条射线等. Unity3D 4.2版本号截至.眼下 ...