https://github.com/cloudera/flume/blob/master/flume-docs/src/docs/UserGuide/Introduction

=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.
=== Reliability
   
  Reliability, the ability to continue delivering events in the face of
  failures without losing data, is a vital feature of Flume. Large
  distributed systems can and do suffer partial failures in many ways -
  physical hardware can fail, resources such as network bandwidth or
  memory can become scarce, or software can crash or run slowly. Flume
  emphasizes fault-tolerance as a core design principle and keeps
  running and collecting data even when many components have failed.
   
  Flume can guarantee that all data received by an agent node will
  eventually make it to the collector at the end of its flow as long as
  the agent node keeps running. That is, data can be *reliably*
  delivered to its eventual destination.
   
  However, reliable delivery can be very resource intensive and is often
  a stronger guarantee than some data sources require. Therefore, Flume
  allows the user to specify, on a per-flow basis, the level of
  reliability required. There are three supported reliability levels:
   
  * End-to-end
  * Store on failure
  * Best effort
   
  .A Note About Reliability
  ******************
  Although Flume is extremely tolerant to machine, network, and software
  failures, there is never any such thing as '100% reliability'. If all
  the machines in a Flume installation were irrevocably destroyed in
  some terrible data center incident, all copies of Flume's data would
  be lost and there would be no way to recover them. Therefore all of
  Flume's reliability levels make guarantees about data delivery 'until
  some maximum number of failures have occurred'. Flume's failure modes
  - in terms of what can fail and what will keep running if they do -
  are described in detail later in this guide.
  ******************
   
  The *end-to-end* reliability level guarantees that once Flume accepts
  an event, that event will make it to the endpoint - as long as the
  agent that accepted the event remains live long enough. The first
  thing the agent does in this setting is write the event to disk in a
  ''write-ahead log'' (WAL) so that, if the agent crashes and restarts,
  knowledge of the event is not lost. After the event has successfully
  made its way to the end of its flow, an acknowledgment is sent back to
  the originating agent so that it knows it no longer needs to store the
  event on disk. This reliability level can withstand any number of
  failures downstream of the initial agent.
   
  The *store on failure* reliability level causes nodes to only require
  an acknowledgement from the node one hop downstream. If the sending
  node detects a failure, it will store data on its local disk until the
  downstream node is repaired, or an alternate downstream destination
  can be selected. While this is effective, data can be lost if a
  compound or silent failure occurs.
   
  The *best-effort* reliability level sends data to the next hop with no
  attempts to confirm or retry delivery. If nodes fail, any data that
  they were in the process of transmitting or receiving can be
  lost. This is the weakest reliability level, but also the most
  lightweight.

three supported reliability levels: * End-to-end * Store on failure * Best effort的更多相关文章

  1. SignalR Supported Platforms -摘自网络

    SignalR is supported under a variety of server and client configurations. In addition, each transpor ...

  2. store操作

    store.remove(rs); store.sync({ success: function (e, opt) { this.store.commitChanges(); }, failure: ...

  3. extjs 解决使用store.sync()方法更新item有时不触发后台action的问题

    问题描述: extjs 解决使用store.sync()方法更新item有时不触发后台action,不出发后台action的原因是item的字段值没有变化 解决方法: item.setDirty(tr ...

  4. PMP用语集

    AC actual cost 实际成本 ACWP actual cost of work performed 已完工作实际成本 BAC budget at completion 完工预算 BCWP b ...

  5. Solaris10安装配置LDAP(iPlanet Directory Server )

    Solaris10安装光盘自带了iPlanet Directory Server安装包,系统管理员可以利用iPlanet Directory Server在Solaris系统创建一个LDAP Serv ...

  6. memory ordering 内存排序

    Memory ordering - Wikipedia https://en.wikipedia.org/wiki/Memory_ordering https://zh.wikipedia.org/w ...

  7. Web测试介绍2一 安全测试

            安全测试是在IT软件产品的生命周期中,特别是产品开发基本完成到发布阶段,对产品进行检验以验证产品符合安全需求定义和产品质量标准的过程. 主要安全需求包括: (i) 认证 Authent ...

  8. Python 3.6.0的sqlite3模块无法执行VACUUM语句

    Python 3.6.0的sqlite3模块存在一个bug(见issue 29003),无法执行VACUUM语句. 一执行就出现异常: Traceback (most recent call last ...

  9. Flume1.5.0的安装、部署、简单应用(含伪分布式、与hadoop2.2.0、hbase0.96的案例)

    目录: 一.什么是Flume? 1)flume的特点 2)flume的可靠性 3)flume的可恢复性 4)flume 的 一些核心概念 二.flume的官方网站在哪里? 三.在哪里下载? 四.如何安 ...

随机推荐

  1. 用node写的一个后台框架

    server.js var http=require('http') var handleUrl=require('./handleUrl') var config = require('./conf ...

  2. linux-3.2.36内核启动1-启动参数(arm平台 启动参数的获取和处理,分析setup_arch)【转】

    转自:http://blog.csdn.net/tommy_wxie/article/details/17093297 最近公司要求调试一个内核,启动时有问题,所以就花了一点时间看看内核启动. 看的过 ...

  3. os.system() 和 os.popen()

    1.os.popen(command[, mode[, bufsize]])  os.system(command) 2.os.popen() 功能强于os.system() , os.popen() ...

  4. AJAX 是一种在无需重新加载整个网页的情况下,能够更新部分网页的技术。

    AJAX = Asynchronous JavaScript and XML(异步的 JavaScript 和 XML). AJAX 不是新的编程语言,而是一种使用现有标准的新方法. AJAX 是与服 ...

  5. IP分段小记

    192.168.0.1 个人电脑:0.2-0.50 硬件开发板:0.51-0.100 机器人工控机:0.101-0.200 激光雷达:192.168.254.51~100 编码器板子:192.168. ...

  6. Java-HashMap原理解析

    本文分析HashMap的实现原理. 数据结构(散列表) HashMap是一个散列表(也叫哈希表),用来存储键值对(key-value)映射.散列表是一种数组和链表的结合体,结构图如下: 简单来说散列表 ...

  7. IntelliJ IDEA中Spring Boot项目使用spring-boot-devtools无法实现热部署/热更新的问题解决

    这个设置真的和Eclipse有很大区别,Eclipse中只要运行之后就可实现修改文件自动重启.但IDEA不太一样,需要做如下配置: 前提: 1.添加spring-boot-devtools到POM. ...

  8. ORACLE MOS 翻译

    http://blog.csdn.net/msdnchina/article/details/53174196

  9. Unity -- 入门教程一

    首先声明一下,我用的Unity版本是4.6.6,编译环境是VS2010,其余的我会慢慢介绍,安装的过程这里我就不做讲解了,度娘那会做的比我详细.安装包可以在最下面的联系方式找我要,现在开始进入主题.  ...

  10. Build FTP Server on Windows

    1. Use the self-ftp component service with windows control panel / program / start or close windows ...