Hadoop Browse the filesystem 无效处理

当我们安装好并正常运行hdfs后输入http://xxxxxxxxx:50070会进入下图所示的页面。 其中Browse the filesystem 是查看文件系统的入口。

但是在发现这个链接一直无效。通过Chrome的开发工具可以看这个链接访问地址是：nn_browsedfscontent.jsp

下面是nn_browsedfscontent.jsp的代码

 <%@ page
   contentType="text/html; charset=UTF-8"
   import="java.io.*"
   import="java.security.PrivilegedExceptionAction"
   import="java.util.*"
   import="javax.servlet.*"
   import="javax.servlet.http.*"
   import="org.apache.hadoop.conf.Configuration"
   import="org.apache.hadoop.hdfs.*"
   import="org.apache.hadoop.hdfs.server.namenode.*"
   import="org.apache.hadoop.hdfs.server.datanode.*"
   import="org.apache.hadoop.hdfs.protocol.*"
   import="org.apache.hadoop.hdfs.security.token.delegation.*"
   import="org.apache.hadoop.io.Text"
   import="org.apache.hadoop.security.UserGroupInformation"
   import="org.apache.hadoop.security.token.Token"
   import="org.apache.hadoop.util.*"
   import="java.text.DateFormat"
   import="java.net.InetAddress"
   import="java.net.URLEncoder"
 %>
 <%!
   static String getDelegationToken(final NameNode nn,
                                    HttpServletRequest request, Configuration conf)
                                    throws IOException, InterruptedException {
     final UserGroupInformation ugi = JspHelper.getUGI(request, conf);
     Token<DelegationTokenIdentifier> token =
       ugi.doAs(
               new PrivilegedExceptionAction<Token<DelegationTokenIdentifier>>()
           {
             public Token<DelegationTokenIdentifier> run() throws IOException {
               return nn.getDelegationToken(new Text(ugi.getUserName()));
             }
           });
     return token.encodeToUrlString();
   }

   public void redirectToRandomDataNode(
                             NameNode nn,
                             HttpServletRequest request,
                             HttpServletResponse resp,
                             Configuration conf
                            ) throws IOException, InterruptedException {
     String tokenString = null;
     if (UserGroupInformation.isSecurityEnabled()) {
       tokenString = getDelegationToken(nn, request, conf);
     }
     FSNamesystem fsn = nn.getNamesystem();
     String datanode = fsn.randomDataNode();
     String redirectLocation;
     String nodeToRedirect;
     int redirectPort;
     if (datanode != null) {
       redirectPort = Integer.parseInt(datanode.substring(datanode.indexOf(':')
                      + 1));
       nodeToRedirect = datanode.substring(0, datanode.indexOf(':'));
     }
     else {
       nodeToRedirect = nn.getHttpAddress().getHostName();
       redirectPort = nn.getHttpAddress().getPort();
     }
     String fqdn = InetAddress.getByName(nodeToRedirect).getCanonicalHostName();
     redirectLocation = "http://" + fqdn + ":" + redirectPort +
                        "/browseDirectory.jsp?namenodeInfoPort=" +
                        nn.getHttpAddress().getPort() +
                        "&dir=/" +
                        (tokenString == null ? "" :
                         JspHelper.getDelegationTokenUrlParam(tokenString));
     resp.sendRedirect(redirectLocation);
   }
 %>

 <html>

 <title></title>

 <body>
 <%
   NameNode nn = (NameNode)application.getAttribute("name.node");
   Configuration conf = (Configuration) application.getAttribute(JspHelper.CURRENT_CONF);
   redirectToRandomDataNode(nn, request, response, conf);
 %>
 <hr>

 <h2>Local logs</h2>
 <a href="/logs/">Log</a> directory

 <%
 out.println(ServletUtil.htmlFooter());
 %>

从代码中可以看出实际是跳转到一台datanode的browseDirectory.jsp，如：

http://xxxxxxx:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=/

而xxxxxxx是集群中一台datanode的主机名。

那么就可以从以下两个点出发去看为什么无效：

1.50075端口是否正常

2.使用的机器是否能通过主机面访问datanode

检查之后发现自己这两个都没有配置......

最后通过在hdfs-site.xml中添加如下配置

<property>
<name>dfs.datanode.http.address</name>
<value>10.0.0.234:50075</value>
</property>

在hosts中添加datanode的主机命和ip修复了这个问题

记录、共勉。