YARN Resource Management
https://www.jianshu.com/p/b9245242472b
https://stackoverflow.com/questions/42637631/what-does-virtual-core-in-yarn-vcore-mean
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_yarn-resource-management/content/enabling_cgroups.html
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_yarn-resource-management/content/using_cgroups.html
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html
https://developer.ibm.com/hadoop/2017/06/30/deep-dive-yarn-cgroups/
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
Chapter 11. Using the YARN CLI to View Logs for Running Applications
This chapter describes how to use the YARN CLI (Command Line Interface) to view log files for running applications. You can access container log files using the YARN ResourceManager web UI, but more options are available when you use the yarn logs
CLI command.
View all Log Files for a Running Application
Use the following command format to view all logs for a running application:
yarn logs -applicationId <Application ID>
View a Specific Log Type for a Running Application
Use the following command format to view all logs of a particular type for a running application:
yarn logs -applicationId <Application ID> -log_files <log_file_type>
For example, to view only the stderr
error logs:
yarn logs -applicationId <Application ID> -log_files stderr
The -logFiles
option also supports Java regular expressions, so the following format would return all types of log files:
yarn logs -applicationId <Application ID> -log_files .*
View ApplicationMaster Log Files
Use the following command format to view all ApplicationMaster container log files for a running application:
yarn logs -applicationId <Application ID> -am ALL
Use the following command format to view only the first ApplicationMaster containter log files:
yarn logs -applicationId <Application ID> -am 1
List Container IDs
Use the following command format to list all container IDs for a running application:
yarn logs -applicationId <Application ID> -show_application_log_info
View Log Files for One Container
Once you have the container IDs, you can use the following command format to list the log files for a particular container:
yarn logs -applicationId <Application ID> -containerId <Container ID>
Show Container Log File Information
Use the following command format to list all of the container log file names (types) for a running application:
yarn logs -applicationId <Application ID> -show_container_log_info
You can then use the -logFiles
option to view a particular log type.
View a Portion of the Log Files for One Container
For large container log files, you can use the following command format to list only a portion of the log files for a particular container:
yarn logs -applicationId <Application ID> -containerId <Container ID> -size <bytes>
To view the first 1000 bytes:
yarn logs -applicationId <Application ID> -containerId <Container ID> -size 1000
To view the last 1000 bytes:
yarn logs -applicationId <Application ID> -containerId <Container ID> -size -1000
Download Logs for a Running Application
Use the following command format to download logs to a local folder:
yarn logs -applicationId <Application ID> -out <path_to_local_folder>
The container log files are organized in parent folders labeled with the applicable node ID.
Display Help for YARN Logs
To display Help for yarn logs
, run the following commmand:
yarn logs -help
Commands for testing¶
These operations are here primarily for testing.
kill-container <name> --id container-id
Kill a YARN container belong to the application. This is useful primarily for testing the resilience to failures.
Container IDs can be determined from the application instance status JSON document.
YARN Resource Management的更多相关文章
- 【IOS笔记】Resource Management in View Controllers
Resource Management in View Controllers 视图控制器的资源管理 View controllers are an essential part of managin ...
- Linux设备模型(9)_device resource management ---devm申请空间【转】
转自:http://www.wowotech.net/linux_kenrel/device_resource_management.html . 前言 蜗蜗建议,每一个Linux驱动工程师,都能瞄一 ...
- View Controller Programming Guide for iOS---(五)---Resource Management in View Controllers
Resource Management in View Controllers View controllers are an essential part of managing your app’ ...
- 资源管理(Resource Management),知识点
资料 网址 资源管理(Resource Management)服务 包含一系列支持企业IT治理的资源管理产品集合,主要包括资源组和资源目录.通过资源管理服务,您可以按照业务需要搭建合适的资源组织关系, ...
- Hadoop 管理工具HUE配置-Yarn Resource Manager HA配置
安装HUE之后,需要配置很多东西才能将这个系统的功能发挥出来,因为Yarn是配置的HA模式,所以在配置HUE的时候,会有些不用,下面一段文字是官网拿来的 # Configuration for YAR ...
- Resource Management in View Controllers
UIViewController生命周期 UIViewControl是IOS程序中的一个重要组成部分,扮演者一个大管家的身份,管理着程序中的众多视图,今天看看了官方文档并做了如下一些简单的记录: 何时 ...
- Configuring Apache Kafka for Performance and Resource Management
Apache Kafka is optimized for small messages. According to benchmarks, the best performance occurs w ...
- <Effective C++>读书摘要--Resource Management<二>
<Item 15> Provide access to raw resources in resource-managing classes 1.You need a way to con ...
- <Effective C++>读书摘要--Resource Management<一>
1.除了内存资源以外,Other common resources include file descriptors, mutex locks, fonts and brushes in graphi ...
随机推荐
- Vue项目History模式404问题解决
本文主要解决Vue项目使用History模式发布到服务器Nginx上刷新页面404问题.(由于每个项目的情况都不尽相同,本方案已经完美解决本在所使用项目,具体情况可能还需要修改.) 1.项目背景分析 ...
- MySQL安装目录修改
- XamarinSQLite教程在Xamarin.Android项目中使用数据库
XamarinSQLite教程在Xamarin.Android项目中使用数据库 在Xamarin.Android项目中使用预设数据库的具体操作步骤如下: (1)创建一个Xamarin.Android项 ...
- Egret 按钮点击缩放效果
非代码设计,exml直接操作 设计模式下选中对象,之后[源码],会直接定位到该对象在exml源码中的位置 width.down = "100%" 表示当按钮按下的时候宽度为 100 ...
- Selenium上传文件方法总结
Web上本地上传图片,弹出的框Selenium是无法识别的,也就是说,selenium本身没有直接的方法去实现上传本地文件,这里总结了两种上传文件的方式. 一.利用Robot类处理文件上传. 其大致流 ...
- POJ1068 --(模拟)
这题是在看一个书的时候的一个例题,当时并不明白啥意思,于是便找了下原题,以前没在POJ上刷过,这是开了个头,以后努力刷这个网站 题目大概意思是:http://poj.org/problem?id=10 ...
- db2 reorg runstats rebind具体操作
db2 reorg runstats rebind具体操作 #reorg table db2 -x "select 'reorg table '||rtrim(tabschema)||'.' ...
- JAVA自学笔记17
JAVA自学笔记17 1.Map接口 1)概述 将键映射到值的对象,一个映射不能包含重复的键,每个键最多只能映射到一个值.可以存储键值对的元素 2)与Collection接口的不同: ①Map是双列的 ...
- hihocoder1696 折线中点(几何)
https://hihocoder.com/problemset/problem/1696 求折线中点,一开始想成先求横坐标中点了,肯定是错的. 一定要从线段长度求中点,然后中点公式推了好久.. #i ...
- VBV Rate Control
Part 1 <06/05/07 12:08pm> Manao | he is negating a float by printing it, adding a "-" ...