一,安装java环境

添加java环境变量

vi /etc/profile
 
# add by tank
export JAVA_HOME=/data/soft/jdk/jdk1.7.0_71
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
 
 
二,修改文件句柄数
vi  /etc/security/limits.conf
 
# add by tank
* soft nofile 65536
* hard nofile  65536
 
 
三,设置ssh无密码登录
 
  参考:http://www.cnblogs.com/tankaixiong/p/4172942.html
 任意主机之间可以无密码登录。
 authorized_keys包含了所有主机的密钥,多主机这里可以通过nfs 挂载同步文件authorized_keys,一改全改
 
 
四,设置HSOT
  vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.183.130 tank1
192.168.183.131 tank2
192.168.183.132 tank3
192.168.183.133 tank4
 
五,安装hadoop 环境
 
这里用的是hadoop2.20版本
目录结构:
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAASsAAAD7CAYAAAAy7bIvAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAACHrSURBVHhe7Z3bkx3Ffcf3r9GDnmyXU8U+x66U48KVchl5N+aW4ENCJbEpiI0hBQhMlpUpjlLJg4qykzykpNiJAoIjgexlkYRkIYQkBOiy0uq2sJL2hkBCq+tqf5meOTPb02dmuqdPT0/3Od+Hqd090/f+/j7z656z/RtYWloiXBgDaAAacFUDN27coA0bNtCAqw1Eu2A80AA0wDRgBFY3Fl6lxYtNujr9Ml05t44unxmly6fX0ReTI3TpxC9p4fhamjv6NM2feImuzOyABwcvFhqABkprwACsbgegepluX3ud7tyepatzr9DnZ1+kUwefpFuLb9OdW+8m182rb9LM0RdLNxJPVjxZoQFowACsliJYXd8aXkRL4XXug0fo6N5n6dqlV2np5o7g3hjduvoWTR98DLDCUxUagAZKa8A4rCJgEZ3e92P6aOcv6PzHz4WQunnldbr+xWb69MCjpRuJpyqeqtAANGAGVhdWPKvYuzq55wE6/M4/0mcfPh1CanFhI301+x80tf8nCrBqUWNgkJoT3UyQiTLK1j9BzcEBGhiIrkYrK3/ZNA1q4SmsoJmyc4X0vj0AlGE1Pj5Oo6OjmaL56vyvkmXgzcUttHznKh3f+Zd06O2f0NSBn4eQunJ+A3356b/Q2ff/TkF4JkBjooySgm41VwDbagTAygCNQpqJ5iANNFrhOPG/+yYutLekfvBQKmSDEqzGxsZo9erVtGrVqszCrnw2GsKKger65f+hpVszdGR8DR38/d/SufcfDSF16cw6Wjj5fLAM/CnNndooAZYJ0JgooxuxqdSflSbyvBKvbKJJg1nQg7AVHnrdzB/yuvawkcIqBtXQ0FAurL4890ICqqsL/0XXLu+lj//wffrgzb+iM3sfCSE1d/wpmvnkcbpw+B/o1B8barAKvJDB9pJqsDmR5Gk1VpZasQcSDmxo2NG9wWYzvZTk7nUs0XLvtWGS047CyWRlDjZpoggqmWlEgKlAD4blmmGhPeY1WQgrHlTDw8O5sLp05rnQo2Kg+vLir2lh6l/p8Fvfo/ff+BGdevehBFLTBx+mhYUFmtz9kAKsAujExp63pFriDZn9vuKRhMunZN8rfS+CWrwnJr+XtCOVr2gyBO8oE1h5aVh7+OUjYAXDN2/4Po5pLqxEUBXBamHy6RSoZibX0aGtf077XvsBndxxb/B1hYfps/0P0tR7PwphdfLdv1aAFb/BLhhsCK/Yu2qn6/BSuDwZSynmnYVLraJ7KRhGgknni9uQ3psKPb/2nlOeKPLTwLPy0ZDQ5uqBmgsrtpnO9qj4a2RkJBMysxNPJh4VA9X0kbV04I0/o72bv0cn3l4TQurcH9fQ2d3fD2F1YteD+rBKwYV5JzZhJfeYVDbEi9NwfUqWtngjCBhUDwPXx1i6Z6XSgYtHfhYu/WJQnTv0BO3f8m3a87vv0ESwd8UgdWbX98Lrvt+fpomd9+vDinlV8fKwYDknXwbGAMhaBqbvJftlss1uvm2ppR/nLSmkwdtAGKaK3fVbGiOwmv7o0RSoTr//U9r/2rdoz2+/Tce3R5CKr+9s/ISO77hPH1ZL3PeUBhvUiD0rBgdueVi8wS58hyu1wd65/Gw02P5X0XenmHGlvz+VTh/DSiWNUJZskx5vBfFWsE80YARWU4f+Plz6MY+Kgerknr8JYbV70592wOr0rr+gU+897onAsLndb09v9Nddr9YIrC4ca9KJ3Q/QsZ330bEd9wfXgyGsdm36Lh3Zvib4guj9oTd17J17A1A9RvPntgJWffI0hPG7a/y+zY0RWIVnzSzO0fzUdjq57xk60Lqb9mx5iCY/3EzXrpz3BExZooJn5Zug0d7ehaMxWEEkvSsSzC3m1gUNAFZYjnns+QIiLkDEVhsAK8AKsIIGvNAAYAWheiFUW09v1OOutwpYAVaAFTTghQaswgqBJdx9asGjwNy4rgGLsEJgCdfFgPYBWC5rwCKsEFjCZSGgbQCV6xqoDVYuB5a4fdddmWv4vM+zJrlM2iKR6JZjog+uixft6y/A2oWVzcASygfldU64CUPny2C/510qBqcDLBN9UGkb0vQXMOqcb6uwcjOwhHlY6cDFtHclwjIu33Tb6hQv6u4vUFqFlZuBJczCShcGRd6X7J5otGIb8sAFY+8vY/d9vq3Cym5gCfF89uCcqubKcch8AAo2iUXLNNUlnCkoyIBX9n63y1DfRY729waUrcKq8sASHSeHpoNCJOei5wagiCZVZb+nLDDKGEw3Zcdgylv2ycou006k7Q0I+DKPVmFVfWAJLqBDR+SbggAUwjeYu4WV6ua6CBZeNCptyHsLWeThAVYAjC9wEttpFVbVB5bgAzpkLAOTcPTF51TFBq2z91MEoCKvTbbvpLJBLrY7C0yAFWAFWCn8f1U1gSVYQNN2ANRUQIfuYJXlncj2pPIgp+Ix5XlJqnllbVOBna8iRrv7A8BWPauqAktkB3TQg1WeZ1RmL6jIeynr2agAMM8rK9pY5+/B2PvD2H2fZ6uwcjmwBA8jlf0iXSCVhVWZpaOK96RTv+8iR/t7A8ZWYeVLYImivR4VT0cXZEVLwTKQMVk/DL03DL0X5tEqrNiA+RBYogysym5idwMd1byAFQDTC3Cq9W2gLwNY9BZQXGpVBSuVpWjeeAJWgJUvtlamndY9qzKNs502a4knM3xVWKksH+P9KZkHlfcSAHtWgJRtm7FZH2BV8IVQGTTyNr91vCIZgPL2s1RhqQIym8JDXQBrWQ0AVgrfDys7qEgPQ4QGzGsAsAKsvAgWAOM3b/y+jalVWCFgBATnm4Ggve5o1iKsEDACwndH+JgL/+bCIqwQMAIG4p+BYM7cmbPaYOVywAgI1B2BYi4wF7EG7MLKeMAITCSMGRroFw0Yh9X4+DiNjo5mvmEyHzAiT6jF51X1y+SinwBZL2nAKKzGxsZo9erVtGrVqkxYmQ8YAVj1khjRF8C1SAPGYBWDamhoKBdW5gNGBJMbHrg3QAPh1aBWeJxx/Hfws9FqgzM6RTRKN0CNFoQBOEADPmnACKx4UA0PD+fCqpqAEdzZ6kEgiOjUUHEZGIFKjGjj00ShrQBLv2uga1iJoCqClfGAESmvqu01hZ6UAKvUcccQfb+LHv330wa6hhXbTGd7VPw1MjKSuWdlPGBELoQAKxiknwaJecuft65hVWZwKw8YkfyfH5aBZeYFaQE2HzRgFVbmA0aIG+wrG+qtBr8sZGJMb7xjgx0G6oOBoo0rOrUKK5cDRkAUgBc04LYGrMLKl4AREK3bosX89Of8WIUVE5kPASNgDP1pDJh3t+fdOqwgCLcFgfnB/LiqAcAKJ4XipFBowAsNAFYQqhdCdfVpj3bZ80QTWL2w+TThwhhAA9CASxr4tzfP0djhebp56zYlsLp8fZlwYQygAWjAJQ1MX7pN2w4tUOuD2RVYzV9dJleur+Zeo8WLTbo6/TJdObeOLp8Zpcun19EXkyN06cQvaeH4Wpo7+jTNTLxEs9M7nWm3K+OHdrijZcxFd3OxsLhMny/eoV9tObMCq7mvlsmN604Aqpfp9rXX6c7tWbo69wp9fvZFOnXwSbq1+DbdufVuct28+iad/+RFR9rtyvittCMOnFrmp0wDrCxZmqz7efl0y9NpA/K4p1GVOfkyWPWxpWmyDJy5skyuXCGsrm8NL6Kl8Dr3wSN0dO+zdO3Sq7R0c0dwb4xuXX2Lpg8+5ky7TY5fEWBU62Fl8GnL/p1Xj1iOSnvy8sjKKgParHao5lfpA9LUw4gvrgmwunh5mVy5eFipBJZwpd1VtIMZGys3/hnXkWWEYv2qhsqnU+mD2Jayefj8srJk9/nxUGlHVhrVOnTLR77u2HJJhNX5wCBcuRZLBpaQt3sb3TswSGs/7KaPJsooXz8zJNa/+GdeX2X3+XwqaXVAF+cR2yjWx/8ta4vsflyXarqs8esmr1x75eccZabH7PNg3yq1DPzsi2Vy5SobWELe7m30IwarQ9300UQZ6vUXwSKrvyy9+LkOcFTL5tNl1V10v0zfZGXH9RSlUxkHuYbU5w5lmR2rheDlXwpWn15aJleusoElzh7bJGl7GzQHu+mjiTLU62cGxuYj76c4V3E6lTksk5ZvQ17ZReXFoIjzimllbVEBjViHyhjwaWRtKFse0qvrXGWs5oOXfylYnf18mVy5dAJLFLd9Gw0HntUzv1tP32gHjPjGP08m/f3PB7gAEw9sWxmHA3z69VEZB9rjxN1jASiGf8eNX+694nbwfWAGxP7O+yn2N06XNQ4qBi+b+7zyi+qN28+nEdOr5Je1jR8nlbRlxk6nPOQxy5LZ4OVfClanF5bJlUsnsERx2xkkAiD9yXp6h/Xztw8HEW4epn/v6HMEk6f3s7GI8gz/NhqXd14YDPJk3zu9n0FN/V7SjlS+9PgzI2b1Fv0UIZQ3BnEZuvf5dohlFJWd13a+DBNty2qfCqDz0rhiB2hHZBPsLWwKVpPzy+TKpRNYorjt22gogMk/vR/3Ufg7hFfsXbXTvb+evh7A7e1kXLg87F4Au99wY/ab+wdoKADbZNG9ebEdy5TkE8afGRLrE/8zNi7+87jfcbqscVAxXJW5F+uQ1anatqK6i+rg86mmyxo7lb4jTX18uBC8YErB6sTsMrly6QSWKG77NvphAKun9sV95P7eF4Hn12H/J+mpb7bTsc+/uZ7+kIxLXp6ozF/fN0A//O/g91R5wr1ZsR2svnY+YfxjwIj9Yp+zz+KfKnMmSyu7z9fRTf0q0BTboponrw9Zn6t+pjK2SFM9N6a/FGB1bGaZXLl0AksUt30brQlg9WQAqygd9/emwKsKoLSdfR6CJk7H0gzQmk1Rnu3PR8vAqIz0vSjfw/RKUvZKvqx7X39+MmpHKl96/JlBZfUp/jzvfl4emdGXmfsydbNyi9LLypLdj9tdpo54LFTylhkXpK2GIeztasqzOnJxmVy5dAJLFLd9G90TgOaJvXEf+b8n6YnAuwmXgd98mO4JPKsk3caV5eHXnlufLmPvevoat3RcKTuoI/deVO899zLwRXXeszF73IvgwvrK7qvOlyyt7D5fT5xWJ09We2XlyO7HZaqkE9sej7HqOCJdPYxgbwxTsPrkwjK5cpUNLHFs9+POtL14DNuQ/KN8rJkhZZUVfy67z+eVeVV5ZYn1i+l084ltKxozE3Ww8vly8n53Rf9oR9o+poK34ilYfXR+mVy5Jg+vpxO7H6BjO++jYzvuD64Haf9r36Jdm75LR7avoeM776fjO4J779xLx3Y/RkeObHOm7cVjuI1+EHhWPw9gJRtrZlBZaeLPZff5vHlp4zSy+yxdmfrEdheVL6tbdl/WhxjUReORlUY2P7gv17CpMWJfBUnB6nAgSJeuj6YW6PAnv6eDu56l/W/cTWP/+2Pa9e7/0eEzF51qZ7kxi2D1swBWsnxF3lCcNy8Nu6/iTRXlF+soam9cTl4adl/nXtl+8HVktSluR157ZP2QzRnuy3WtM0ZnRFh9OL1MuNwZA2Y4WfOR97npuYsNt0y5eXmK2izrj+x+3D7T6cr0G2mrs5uPgwfvxMwSvfzG2ZUjYhi9cGEMoAFowCUNTM7doS0ffE5bD3Anhbp07jLagnPAoQFogGmAnRD6xv5Zunbj1opnhSgd9qJ0YKwx1tBAeQ0gFBdCcSEUFzTghQaswurGwqtKgSDmT7xEV2Z2eDGAeEKWf0JizDBmOhqwCKvbpQJBzBx9EbDCEx8agAYSDViE1VIEqxKBIHToizx4akMDvamB2mClEggCoutN0WFeMa86GrALq5KBIKQdmmjSYPCN8OaE7uS3qKGVXzefbjuRT6oFLJd6fslsFVZlA0FUL1Bd6OjmA3Sqn1OMca+OsVVYlQ0EMXdqY8VPC13o6OaDIfWqIaFf1WvbKqx0AkEUi4CHRvv3ZiM5K2qwOcHBboKagyuBIRotNrgZ+ZMlpQCkcMkZ5R9sNtPLR+7ewECDWliSVPyQqd4wAB/3xtgqrHQCQZSDVQCTRisylBaDVgyOCFRpeJWBFQPXAEWAW6KJZnRqaLRXJkAtqLezHvcmHsaIOfFNA1ZhpRMIohys+M12DiKh55Pl8Sh6Viz/YJMmEo9JLJsL5cW8rxiY8LDgYUEDxjRgFVY6gSD8gBWWfr49pdFe/zxLq7DSCQRhBFZLKsvAKE281IuWkfxSr2gZmLXE9E8MMGDMmcsasAornUAQZmAV7y0VbbDH+1ztNI1GehM9hJfKBjuWgS4LHm3zF8hWYVU2EMSp9x43tt6FSP0VKeYOc8c0YBVWF441lQNBnHrvMZo/txWwwgYtNAANhBqwCquwwsU5mp/aTif3PUMHWnfTni0P0eSHm+nalfMQJUQJDUADuRqwDiu49HDpoQFoQEcDgBWeZPBmoAEvNABYQaheCFXnSYw8veXBAVaAFWAFDXihAcAKQvVCqPCSestL0plPq7BCwAgITkekyAPdWP7qAgJGwOhgdNCAvgaselYIGKE/URA5xq7fNVAbrBAwAsbX78aH/pezAbuwMh0wgj/4LhU8AscOwxDKGQLGy/3xMg6r8fFxGh0dzXzDZD5gRB6UACsYn/vGhzkqN0dGYTU2NkarV6+mVatWZcLKfMAIwAqCLyd4jJe/42UMVjGohoaGcmFlJWAEfy56iw/ywAeP8HfCYGyYu37VgBFY8aAaHh7OhVX1ASPSJ3sOxOemdx0MFQbSrwaCfruj/a5hJYKqCFb2AkZ0Lg9bDe7IYnxrHd9ahwa800DXsGKb6WyPir9GRkYyB8JewAgRVsL56hCqd0KFh+OOh1PXXHQNqzINtxcwIorzl8Tvyw3FBQGUmT+khV7q1IBVWNkLGBF5Vo0GC0YaBXlIotbAq4JXBQ14qQGrsELACDyZ63wyo26/9WcVVggY4bdYYOyYvzo1YBVW4TEPCBjhpQtep0hRNyBp+YgYDDiMDhqABvQ1YN2zwmTpTxbGDmPXzxoArPBmCMtSaMALDQBWEKoXQu1njwJ9jzxqwAqwAqygAS804CSsEFgCezPwJqABUQMOwgqBJWCoMFRooFMDDsJqiRBYAsYKY4UGPPCs0rBCYAmIFuCCBpzdYF80HljCwGTjAD8vNmEBNgNad/SFg5PLQPOBJXQmEEEnYPg6ukGeqnTjJKzMB5bQERBgVZXoUK6OHpHHSVhVFlii2UjOt0oO5gtcXnbkcXzu1UCjFSx3osP7Oj9jZ7xnnDraCspNnfce521Qy1GXGsCA8fumASdhVU1giQAgIYgCkTK4DGSBhPemRM+K+5vlj8tqwy463E/IE6TjoeibONBeAM0lDTgJq2oDS2RDJfGighNGm3w4r/B3MQ+DUgy74PdMr6rtXXFQc2ni0RaAyDcNOAmragNLCOBJnc/OlngqsFqiieZg6DXFP8OJx1nveGOIZX9lGnASVtUGlhBg1bHfpAarEEyDjcCr4peTQqAKCLcy4frmFaC93XuyTsKq2sAS4pIu2jAPl4EhfGJYcRvvyab7yr2lpXY+cZkXelfi5nz3EwWxYwz7XQNOwgqBJWCY/W6Y6L8n/xuIwBKAFYwVGvDifwPD/wNCYAns92DPDxrgNODkMhBPVTxVoQFowBvPCmKFWKEBaIDXADwrLDWw1IAGvNAAYAWheiFUeFnwsgArwAqwgga80IBVWCEQBJ6O8JCgAV0NWIQVAkHoThLywcChActxAxEIAkYHo4MGdDVg0bNCIAjdSUI+GDg0YNuzMh4IgjvsLhXQAUcSQ9wAXK9pwKpnZT4QRB6UAKteEyr6A/hahZX5QBCAFYwYRtwvGrAKq8oCQXQcQ9yGWGvlbKnUWeipM6fY4XnR2VRJGqUAENw5WMH5VdEZ7LgwBtBAVRqwCqtqAkFknezZjk6TOhs95wTQJKhDfK66cL56ciZ7IMIkrQA3QAqghgYq14BVWFUbCKIoMk106mfo/YgnebJTPVNRbzgvKS8tzlqvXJhVPZ1Rrr+en1VYVRsIoghWXKy/ItCEIbpEWGWE7AKsACt4UtY1YBVW1QaCEGHF7UGl4JIX1CFvGciVkwgUy0B4KP56KL7OnVVYVRsIotOzajQGk6jKqQ3wjqAOzdQGOwuvlR1hmVsyClGbscEO4/UVAr602yqsEAgCBu2LYaCd7mnVKqwQCMI9AcAoMSe+aMAqrNigIBAEjMMX40A73dKqdVhBAG4JAPOB+fBFA4AVXkFbfwXti3GgnW6BHLACrAAraMALDQBWEKoXQoWX45aXU8d8AFaAFWAFDXihAauwQsAIPB3reCKjzt7QnUVYIWAEjKY3jAbzWM88WoRV+gx2oiVi17kPHqGje5+la5depaWbO+j29TG6dfUtmj74mBeuKYRbj3Ax7v037rXB6vb1rQGsiE7v+zF9tPMXdP7j50JI3bzyOl3/YjN9euBRL2B1+667jLRTtxzdfLGxd5NfN29ePt3yAK7+AJddWBkPGJE1SQXnr6eCSpibYBNGJiuD3ecvE7BhZcjqlYFAJ78urMQxKPo7q92q+WV9xn1ztlNmLK3CynzAiJKwMvDWR1XwokFm5eMnSmb0/P2832P4ZEEtTxSyemVi0smv2hexbtW6VNPlAU3WZ9zvA1iZDxhRD6xUxFrWYGTpswxc5mnJylT1rMoAOs8DzPMEi8ALWNUDBRV915HGqmdlPmBEVtCGomARGaeJNqPTQdmVCiqR44WpAEAVAlmeUJ5XpOqNFKUrCx0dz0M2PioeZ9zOKmClMgZ1GCLqlIPZKqyMB4xgxxDH56cncFENFtFOlzp/PeMIYwFaMmPU3UvKMmJxmSjzpERIytpaZrmo2q+iOkUIyfqcBSsV2Mj6XQSGbvICOHLgdDNGVmFlPGBE+8TPtEfUucGeBIsIT/fMiXKTupc/6KrGIno4eflUlkdFECoyeBk4+HLLGGleWlkZJmClInZZOwCraqGiMkc6aazCynzAiGjQw2OIk0APIqy4YBGGYKUz0LKloczLUIFfGc8qLk/8qdI3HVip1CeDjOx+EfjLPGR0PE6VcUOa7iBpFVbmA0asdJ4BK/KwhIAQHcEiuvesdEWn4umoeFoiuMTlomzJlpdfFQZZ4FXtm2o6nf0yWb+7KVN3zpGvO0Dx42cVVsYDRrRDZ0Ub5PF+U+RZZQeLMLMM1BVgGY8ky9PKeuKXXQaW8eBk/ezGK1P1dEQQ6+QrgljWnJSBtmyMcN9TWPVCwAjeQGWGo+rxqBhMljeUly8PIir1yJarovGVNey6Pau8h4COVwYQmQORylha9ax8DxhRtPySGbFJD0iEkQrIxDR5oC1jtN14VjpLMlUwqqTLGkOVfCpGhTTVQMwqrNgk+hwwQhdWRWDIMxAZ3LLKzPOKdL2ZMvlUDb1MmbIHQB4UZG3Jm0dZPkCoGgipjqt1WKk2zLV0RfBQ8RLy8uvCSgSTjpdTdskn87pUjL1OWMWAz1uex+Oh0g/X9NkP7QGsFP5fMEu8ed4S/7lMQGUApurVlTW0MumzjD3L+9EFkqwtKmOeB6QyYw1o1etB5dkNYKUAKxXPqeySRdWjyvKgdGGg0w9dw80DWzdtl8FM5vmVnSPZwwb37UINsNKEFYRqV6gYb4w3YAVYGTk8EDABTKrWgFVYIWAEBF21oFF+72rMIqwQMAKG1LuGhLmtfm4twgoBIyDo6gWNMe7dMa4NVr0SMALG0bvGgbl1a27twqrugBHYTMdmOjTgrQaMw2p8fJxGR0czB6QXAkbgaevW0xbz0T/zYRRWY2NjtHr1alq1alUmrHohYASMo3+MA3Pt1lwbg1UMqqGhoVxY1R8wYonYEcdxgIiV89v5M7Da52K1j0xOn5Xl1uTBmDAf/aQBI7DiQTU8PJwLq1oCRoQH9GUFghAP4uOj2whHIwdlqES+6SfhoK8ApW0NdA0rEVRFsKonYEQneBLPKi94RMqrantiHVF0IFbbYkV9/a25rmHFNtPZHhV/jYyMZO5Z1RMwgoNV6jx2Fkgi5zz2VLr+FggAgfl3RQNdw6pMR+wFjIghxITGwYotCQebNMFeX4dAKgoeoRb0tEz/kRaGDw3oa8AqrGwGjGhOxIPCLwO5CM6DDWrkeVYJzLI24/UHG0LF2EED+hqwCqteCBgBsemLDWOHsetGA1Zh5XvAiG4GGnlhqNBAdxqwCis2WT4HjIDYuhMbxg/j140GrMOqm8YiL8QODfSvBgAr/GOrt//YCnD1F7gAK8AKsIIGvNAAYAWheiFUeFH95UVlzTdgBVgBVtCAFxpwElYILIGnKDwpaEDUgIOwQmAJGCoMFRro1ICDsEJgCQgVsIIGPIQVAkvAcGG40ED4hfIbN2jDhg00YEMQe/bsocuXL0s38xYrDywhnHGFDVbpnNjQB+oAlIo0YBVWGzdupE2bNtHs7GyhcVQfWAKwAhgABt80YB1WFy9epFdeeYWmpqZygVV9YAnAyjehor2Aq3VYMdExz6rZbNLx48czgWU+sER82F50PtVgUHcjOXgvfY/db7R4YXBnYHXcg4AAEWjAlgZqgRXr3PT0NK1du5bm5+c7gGU8sER4WugKhCaagwG0+FNCOUClThCNQIVgETBIWwaJevK1Vgus5ubmCj2rSgJLxMcZh5vpeeeyRwPFwnWF3hXOYsfGO16+OKMB67CamZmR7lkZDyzBoANYOSM6eA/wVHU0YB1WKm8DzQeWKLsMjOMMYhmoIyrkAYyq0IBVWKl+z8p4YAnmyofBTlU22PnIOPGScSVwRHrzHaKsQpQoE7ry5tQFBJaAWAEsaMCDf2ReIgSWgFABK2jAC1iF/we0OEfzU9vp5L5n6EDrbtqz5SGa/HAzXbtyHhvFeEMFDfShBqzuWeFpiaclNAAN6GoAsOrDJ5SuWJAPoKlTA4AVYIUlFTTghQYAKwjVC6HW+URH3W54lIAVYAVYQQNeaMBJWCFghBtPMngUmAeXNOAgrBAwwiWBoC0AlisacBBWCBjhijjQDoDKJQ04DysEjIDBuGQwaEt9erQKK9V/ZDYfMMLEMcYmyqhvomFkGHvfNWAVVvUFjDABGhNlwGB8Nxi0vz4NW4dVPQEjTIDGRBn1TTSMDGPvuwasw4oNmP2AEW3QtIITQ5MzrSaS75awY4zjs64GGq2V75yExxqrnIElBJng8qUDUBS3w3cxof0AYpUaqAVWrEN1BIwYiI82Dg/ii08D5QXGe09lTxdVCUARlZm0IxWcAkKvUugo23991QIr6wEj+AARYsAI4RTRJOqNwXPbkwAUHe3gglPgW9RefIsa0KsPetZhVUvAiCJYpSLYsDPX2x6SFVhFZ7zjqOT6DADw8WfsrcOqvoAR/Nnq3HKPLQnj5WFqWVZ2GRgvK9P50uG8ontJHEKE+oI3BY9aWQNWYaX6PSvzASPEN3n831zE5cEGNWLPqpsgE6kN9k5INhosyGq0cQ+vyp8nO7yweufKKqxUJ7t3A0bg6w+qGkC6esHg4vg7CaveDRgBWLloBGiTH2B0ElZMPL0ZMAKwAhj8AIOL8+QsrFwcLLQJhgYN1KcBwApvY5TfxsBQ6zNUjH2w2rpxgzZs2ED/D9wreqzgsLCzAAAAAElFTkSuQmCC" alt="" />
设置环境变量:
export HADOOP_HOME=/data/hadoop/hadoop-2.2.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
 
注意:$HADOOP/bin 和$HADOOP/sbin 目录下的文件都有可执行的权限
 
修改配置文件:
[tank@192 hadoop]$ vi core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/hadoop/tmp</value><description>(备注:请先在 /usr/hadoop 目录下建立 tmp 文件夹)A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.149.128:9000</value>
</property>
</configuration>
 备注:如没有配置hadoop.tmp.dir参数,此时系统默认的临时目录为:/tmp/hadoo-hadoop。而这个目录在每次重启后都会被干掉,必须重新执行format才行,否则会出错。
[tank@192 hadoop]$ vi hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/name</value>
<final>true</final>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/data</value>
</property>
</configuration>
 
文件必须已经预先创建好并存在!
 
[tank@192 hadoop]$ vi yarn-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>192.168.149.128:9001</value>
</property>
</configuration>
 
 
注意上面一定要填Ip,不要填localhost,不然eclipse会连接不到!
 
设置主从关系$HADOOP_HOME/etc/hadoop/目录下:
[hadoop@tank1 hadoop]$ vi masters 
192.168.183.130

//主机特有,从机可以不需要
[hadoop@tank1 hadoop]$ vi slaves

192.168.183.131
192.168.183.132
192.168.183.133

 
[hadoop@tank1 hadoop]$ hadoop namenode -format   //第一次需要
 
启动:
sbin/start-all.sh
 
查看状态:主机

[tank@192 hadoop-2.2.0]$ jps
2751 ResourceManager
2628 SecondaryNameNode
2469 NameNode

查看状态:从机
[hadoop@tank2 sbin]$ jps
1745 NodeManager
1658 DataNode

 

 
总共有5个hadoop线程 
 
访问地址查看hdfs 的运行状态:
http://192.168.149.128:50070/dfshealth.jsp
 
 
 从机运行一段时间后NodeManager停止了,查看日志信息:
 
 FATAL org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.ConnectException: Call From tank2/192.168.183.131 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:181)
 
解决方案:
yarn-site.xml 中添加配置
<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>192.168.183.130</value>//主机IP
 </property>
停止,重启
 

hadoop 分布式开发环境搭建的更多相关文章

  1. Hadoop Eclipse开发环境搭建

        This document is from my evernote, when I was still at baidu, I have a complete hadoop developme ...

  2. Hadoop项目开发环境搭建(Eclipse\MyEclipse + Maven)

    写在前面的话 可详细参考,一定得去看 HBase 开发环境搭建(Eclipse\MyEclipse + Maven) Zookeeper项目开发环境搭建(Eclipse\MyEclipse + Mav ...

  3. (转)Hadoop Eclipse开发环境搭建

    来源:http://www.cnblogs.com/justinzhang/p/4261851.html This document is from my evernote, when I was s ...

  4. 【原创干货】大数据Hadoop/Spark开发环境搭建

    已经自学了好几个月的大数据了,第一个月里自己通过看书.看视频.网上查资料也把hadoop(1.x.2.x).spark单机.伪分布式.集群都部署了一遍,但经历短暂的兴奋后,还是觉得不得门而入. 只有深 ...

  5. Hadoop基本开发环境搭建(原创,已实践)

    软件包: hadoop-2.7.2.tar.gz hadoop-eclipse-plugin-2.7.2.jar hadoop-common-2.7.1-bin.zip eclipse  jdk1.8 ...

  6. 【hadoop之翊】——windows 7使用eclipse下hadoop应用开发环境搭建

    由于一些缘故,这节内容到如今才写.事实上弄hadoop有一段时间了,能够编写一些小程序了,今天来还是来说说环境的搭建.... 说明一下:这篇文章的步骤是接上一篇的hadoop文章的:http://bl ...

  7. ubuntu14.04 Hadoop单机开发环境搭建MapReduce项目

    Hadoop官网:http://hadoop.apache.org/ 目前最新的版本是Hadoop 3.0.0-alpha1前提:java 1.6 版本以上 首先从官网下载压缩包(hadoop-3.0 ...

  8. hadoop分布式的环境搭建

    版本: 使用hadoop1.1.2    JDK为java7 1.下载hadoop 2.配置hadoop文件 3测试 1.下载hadoop: 1.1 在https://archive.apache.o ...

  9. hadoop本地开发环境搭建

    1:下载hadoop2.7.3并解压 2:配置hadoop2.7.3环境变量 HADOOP_HOME %HADOOP_HOME%\bin 3:下载hadoop-eclipse-plugin插件 网址: ...

随机推荐

  1. springMVC (优秀篇)

    本文依然是复制的 Spring3 MVC结构简单,应了那句话简单就是美,而且他强大不失灵活,性能也很优秀. 官方的下载网址是:http://www.springsource.org/download  ...

  2. BZOJ4712洪水——动态DP+树链剖分+线段树

    题目描述 小A走到一个山脚下,准备给自己造一个小屋.这时候,小A的朋友(op,又叫管理员)打开了创造模式,然后飞到 山顶放了格水.于是小A面前出现了一个瀑布.作为平民的小A只好老实巴交地爬山堵水.那么 ...

  3. 微信小程序——代码片段汇集

    导航栏 作者:beatzcs       链接:https://www.jianshu.com/p/c681007a6287 这个导航虽然已经很完善了,不过还是要根据自己的来进行修改的 tabs.wx ...

  4. 深度学习框架比较TensorFlow、Theano、Caffe、SciKit-learn、Keras

    TheanoTheano在深度学习框架中是祖师级的存在.Theano基于Python语言开发的,是一个擅长处理多维数组的库,这一点和numpy很像.当与其他深度学习库结合起来,它十分适合数据探索.它为 ...

  5. Number Sequence POJ - 1019 递推 数学

    题意 1 12 123 1234 12345 ....这样的序列 问第n位数字是几   是数字! 1-9! 思路:递推关系 主要是位数的计算   用a[i]=a[i-1]+(int)log10((do ...

  6. centos6.8下安装dc2012

    前言 centos6.8系统中安装synopsys公司的design compiler 2012. 流程 1.请掌握必要的linux知识,否则你将获得成吨的困难. linux系统:centos 6.8 ...

  7. mysql中存储字段类型的查询效率

    检索性能从快到慢的是(此处是听人说的): 第一:tinyint,smallint,mediumint,int,bigint第二:char,varchar第三:NULL 解释(转载): 整数类型1.TI ...

  8. failover swarm 故障转移

    #故障转移 Failover #当其中一个节点关闭宕机时,其节点中的service会转移到另一个节点上.Swarm会检测到node1发生故障并把此故障节点的状态标记为Down; docker node ...

  9. Centos7下源码编译安装python3.6

    测试环境: 操作步骤: 1. 下载Python源码包(python3.6.0) 官网下载地址:https://www.python.org/downloads/ 搜狐下载地址:http://mirro ...

  10. Rocket.Chat 开源IM系统部署

    Rocket.Chat 官方给出的文档也个人觉得太麻烦了,并且对ubuntu的支持程度远高于CentOS,自己就折腾写了个安装的笔记,如果是在公司内部或者是部门内部还是很有用处的,比较看中的功能有和g ...