一,安装java环境

添加java环境变量

vi /etc/profile
 
# add by tank
export JAVA_HOME=/data/soft/jdk/jdk1.7.0_71
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
 
 
二,修改文件句柄数
vi  /etc/security/limits.conf
 
# add by tank
* soft nofile 65536
* hard nofile  65536
 
 
三,设置ssh无密码登录
 
  参考:http://www.cnblogs.com/tankaixiong/p/4172942.html
 任意主机之间可以无密码登录。
 authorized_keys包含了所有主机的密钥,多主机这里可以通过nfs 挂载同步文件authorized_keys,一改全改
 
 
四,设置HSOT
  vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.183.130 tank1
192.168.183.131 tank2
192.168.183.132 tank3
192.168.183.133 tank4
 
五,安装hadoop 环境
 
这里用的是hadoop2.20版本
目录结构:
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAASsAAAD7CAYAAAAy7bIvAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAACHrSURBVHhe7Z3bkx3Ffcf3r9GDnmyXU8U+x66U48KVchl5N+aW4ENCJbEpiI0hBQhMlpUpjlLJg4qykzykpNiJAoIjgexlkYRkIYQkBOiy0uq2sJL2hkBCq+tqf5meOTPb02dmuqdPT0/3Od+Hqd090/f+/j7z656z/RtYWloiXBgDaAAacFUDN27coA0bNtCAqw1Eu2A80AA0wDRgBFY3Fl6lxYtNujr9Ml05t44unxmly6fX0ReTI3TpxC9p4fhamjv6NM2feImuzOyABwcvFhqABkprwACsbgegepluX3ud7tyepatzr9DnZ1+kUwefpFuLb9OdW+8m182rb9LM0RdLNxJPVjxZoQFowACsliJYXd8aXkRL4XXug0fo6N5n6dqlV2np5o7g3hjduvoWTR98DLDCUxUagAZKa8A4rCJgEZ3e92P6aOcv6PzHz4WQunnldbr+xWb69MCjpRuJpyqeqtAANGAGVhdWPKvYuzq55wE6/M4/0mcfPh1CanFhI301+x80tf8nCrBqUWNgkJoT3UyQiTLK1j9BzcEBGhiIrkYrK3/ZNA1q4SmsoJmyc4X0vj0AlGE1Pj5Oo6OjmaL56vyvkmXgzcUttHznKh3f+Zd06O2f0NSBn4eQunJ+A3356b/Q2ff/TkF4JkBjooySgm41VwDbagTAygCNQpqJ5iANNFrhOPG/+yYutLekfvBQKmSDEqzGxsZo9erVtGrVqszCrnw2GsKKger65f+hpVszdGR8DR38/d/SufcfDSF16cw6Wjj5fLAM/CnNndooAZYJ0JgooxuxqdSflSbyvBKvbKJJg1nQg7AVHnrdzB/yuvawkcIqBtXQ0FAurL4890ICqqsL/0XXLu+lj//wffrgzb+iM3sfCSE1d/wpmvnkcbpw+B/o1B8barAKvJDB9pJqsDmR5Gk1VpZasQcSDmxo2NG9wWYzvZTk7nUs0XLvtWGS047CyWRlDjZpoggqmWlEgKlAD4blmmGhPeY1WQgrHlTDw8O5sLp05rnQo2Kg+vLir2lh6l/p8Fvfo/ff+BGdevehBFLTBx+mhYUFmtz9kAKsAujExp63pFriDZn9vuKRhMunZN8rfS+CWrwnJr+XtCOVr2gyBO8oE1h5aVh7+OUjYAXDN2/4Po5pLqxEUBXBamHy6RSoZibX0aGtf077XvsBndxxb/B1hYfps/0P0tR7PwphdfLdv1aAFb/BLhhsCK/Yu2qn6/BSuDwZSynmnYVLraJ7KRhGgknni9uQ3psKPb/2nlOeKPLTwLPy0ZDQ5uqBmgsrtpnO9qj4a2RkJBMysxNPJh4VA9X0kbV04I0/o72bv0cn3l4TQurcH9fQ2d3fD2F1YteD+rBKwYV5JzZhJfeYVDbEi9NwfUqWtngjCBhUDwPXx1i6Z6XSgYtHfhYu/WJQnTv0BO3f8m3a87vv0ESwd8UgdWbX98Lrvt+fpomd9+vDinlV8fKwYDknXwbGAMhaBqbvJftlss1uvm2ppR/nLSmkwdtAGKaK3fVbGiOwmv7o0RSoTr//U9r/2rdoz2+/Tce3R5CKr+9s/ISO77hPH1ZL3PeUBhvUiD0rBgdueVi8wS58hyu1wd65/Gw02P5X0XenmHGlvz+VTh/DSiWNUJZskx5vBfFWsE80YARWU4f+Plz6MY+Kgerknr8JYbV70592wOr0rr+gU+897onAsLndb09v9Nddr9YIrC4ca9KJ3Q/QsZ330bEd9wfXgyGsdm36Lh3Zvib4guj9oTd17J17A1A9RvPntgJWffI0hPG7a/y+zY0RWIVnzSzO0fzUdjq57xk60Lqb9mx5iCY/3EzXrpz3BExZooJn5Zug0d7ehaMxWEEkvSsSzC3m1gUNAFZYjnns+QIiLkDEVhsAK8AKsIIGvNAAYAWheiFUW09v1OOutwpYAVaAFTTghQaswgqBJdx9asGjwNy4rgGLsEJgCdfFgPYBWC5rwCKsEFjCZSGgbQCV6xqoDVYuB5a4fdddmWv4vM+zJrlM2iKR6JZjog+uixft6y/A2oWVzcASygfldU64CUPny2C/510qBqcDLBN9UGkb0vQXMOqcb6uwcjOwhHlY6cDFtHclwjIu33Tb6hQv6u4vUFqFlZuBJczCShcGRd6X7J5otGIb8sAFY+8vY/d9vq3Cym5gCfF89uCcqubKcch8AAo2iUXLNNUlnCkoyIBX9n63y1DfRY729waUrcKq8sASHSeHpoNCJOei5wagiCZVZb+nLDDKGEw3Zcdgylv2ycou006k7Q0I+DKPVmFVfWAJLqBDR+SbggAUwjeYu4WV6ua6CBZeNCptyHsLWeThAVYAjC9wEttpFVbVB5bgAzpkLAOTcPTF51TFBq2z91MEoCKvTbbvpLJBLrY7C0yAFWAFWCn8f1U1gSVYQNN2ANRUQIfuYJXlncj2pPIgp+Ix5XlJqnllbVOBna8iRrv7A8BWPauqAktkB3TQg1WeZ1RmL6jIeynr2agAMM8rK9pY5+/B2PvD2H2fZ6uwcjmwBA8jlf0iXSCVhVWZpaOK96RTv+8iR/t7A8ZWYeVLYImivR4VT0cXZEVLwTKQMVk/DL03DL0X5tEqrNiA+RBYogysym5idwMd1byAFQDTC3Cq9W2gLwNY9BZQXGpVBSuVpWjeeAJWgJUvtlamndY9qzKNs502a4knM3xVWKksH+P9KZkHlfcSAHtWgJRtm7FZH2BV8IVQGTTyNr91vCIZgPL2s1RhqQIym8JDXQBrWQ0AVgrfDys7qEgPQ4QGzGsAsAKsvAgWAOM3b/y+jalVWCFgBATnm4Ggve5o1iKsEDACwndH+JgL/+bCIqwQMAIG4p+BYM7cmbPaYOVywAgI1B2BYi4wF7EG7MLKeMAITCSMGRroFw0Yh9X4+DiNjo5mvmEyHzAiT6jF51X1y+SinwBZL2nAKKzGxsZo9erVtGrVqkxYmQ8YAVj1khjRF8C1SAPGYBWDamhoKBdW5gNGBJMbHrg3QAPh1aBWeJxx/Hfws9FqgzM6RTRKN0CNFoQBOEADPmnACKx4UA0PD+fCqpqAEdzZ6kEgiOjUUHEZGIFKjGjj00ShrQBLv2uga1iJoCqClfGAESmvqu01hZ6UAKvUcccQfb+LHv330wa6hhXbTGd7VPw1MjKSuWdlPGBELoQAKxiknwaJecuft65hVWZwKw8YkfyfH5aBZeYFaQE2HzRgFVbmA0aIG+wrG+qtBr8sZGJMb7xjgx0G6oOBoo0rOrUKK5cDRkAUgBc04LYGrMLKl4AREK3bosX89Of8WIUVE5kPASNgDP1pDJh3t+fdOqwgCLcFgfnB/LiqAcAKJ4XipFBowAsNAFYQqhdCdfVpj3bZ80QTWL2w+TThwhhAA9CASxr4tzfP0djhebp56zYlsLp8fZlwYQygAWjAJQ1MX7pN2w4tUOuD2RVYzV9dJleur+Zeo8WLTbo6/TJdObeOLp8Zpcun19EXkyN06cQvaeH4Wpo7+jTNTLxEs9M7nWm3K+OHdrijZcxFd3OxsLhMny/eoV9tObMCq7mvlsmN604Aqpfp9rXX6c7tWbo69wp9fvZFOnXwSbq1+DbdufVuct28+iad/+RFR9rtyvittCMOnFrmp0wDrCxZmqz7efl0y9NpA/K4p1GVOfkyWPWxpWmyDJy5skyuXCGsrm8NL6Kl8Dr3wSN0dO+zdO3Sq7R0c0dwb4xuXX2Lpg8+5ky7TY5fEWBU62Fl8GnL/p1Xj1iOSnvy8sjKKgParHao5lfpA9LUw4gvrgmwunh5mVy5eFipBJZwpd1VtIMZGys3/hnXkWWEYv2qhsqnU+mD2Jayefj8srJk9/nxUGlHVhrVOnTLR77u2HJJhNX5wCBcuRZLBpaQt3sb3TswSGs/7KaPJsooXz8zJNa/+GdeX2X3+XwqaXVAF+cR2yjWx/8ta4vsflyXarqs8esmr1x75eccZabH7PNg3yq1DPzsi2Vy5SobWELe7m30IwarQ9300UQZ6vUXwSKrvyy9+LkOcFTL5tNl1V10v0zfZGXH9RSlUxkHuYbU5w5lmR2rheDlXwpWn15aJleusoElzh7bJGl7GzQHu+mjiTLU62cGxuYj76c4V3E6lTksk5ZvQ17ZReXFoIjzimllbVEBjViHyhjwaWRtKFse0qvrXGWs5oOXfylYnf18mVy5dAJLFLd9Gw0HntUzv1tP32gHjPjGP08m/f3PB7gAEw9sWxmHA3z69VEZB9rjxN1jASiGf8eNX+694nbwfWAGxP7O+yn2N06XNQ4qBi+b+7zyi+qN28+nEdOr5Je1jR8nlbRlxk6nPOQxy5LZ4OVfClanF5bJlUsnsERx2xkkAiD9yXp6h/Xztw8HEW4epn/v6HMEk6f3s7GI8gz/NhqXd14YDPJk3zu9n0FN/V7SjlS+9PgzI2b1Fv0UIZQ3BnEZuvf5dohlFJWd13a+DBNty2qfCqDz0rhiB2hHZBPsLWwKVpPzy+TKpRNYorjt22gogMk/vR/3Ufg7hFfsXbXTvb+evh7A7e1kXLg87F4Au99wY/ab+wdoKADbZNG9ebEdy5TkE8afGRLrE/8zNi7+87jfcbqscVAxXJW5F+uQ1anatqK6i+rg86mmyxo7lb4jTX18uBC8YErB6sTsMrly6QSWKG77NvphAKun9sV95P7eF4Hn12H/J+mpb7bTsc+/uZ7+kIxLXp6ozF/fN0A//O/g91R5wr1ZsR2svnY+YfxjwIj9Yp+zz+KfKnMmSyu7z9fRTf0q0BTboponrw9Zn6t+pjK2SFM9N6a/FGB1bGaZXLl0AksUt30brQlg9WQAqygd9/emwKsKoLSdfR6CJk7H0gzQmk1Rnu3PR8vAqIz0vSjfw/RKUvZKvqx7X39+MmpHKl96/JlBZfUp/jzvfl4emdGXmfsydbNyi9LLypLdj9tdpo54LFTylhkXpK2GIeztasqzOnJxmVy5dAJLFLd9G90TgOaJvXEf+b8n6YnAuwmXgd98mO4JPKsk3caV5eHXnlufLmPvevoat3RcKTuoI/deVO899zLwRXXeszF73IvgwvrK7qvOlyyt7D5fT5xWJ09We2XlyO7HZaqkE9sej7HqOCJdPYxgbwxTsPrkwjK5cpUNLHFs9+POtL14DNuQ/KN8rJkhZZUVfy67z+eVeVV5ZYn1i+l084ltKxozE3Ww8vly8n53Rf9oR9o+poK34ilYfXR+mVy5Jg+vpxO7H6BjO++jYzvuD64Haf9r36Jdm75LR7avoeM776fjO4J779xLx3Y/RkeObHOm7cVjuI1+EHhWPw9gJRtrZlBZaeLPZff5vHlp4zSy+yxdmfrEdheVL6tbdl/WhxjUReORlUY2P7gv17CpMWJfBUnB6nAgSJeuj6YW6PAnv6eDu56l/W/cTWP/+2Pa9e7/0eEzF51qZ7kxi2D1swBWsnxF3lCcNy8Nu6/iTRXlF+soam9cTl4adl/nXtl+8HVktSluR157ZP2QzRnuy3WtM0ZnRFh9OL1MuNwZA2Y4WfOR97npuYsNt0y5eXmK2izrj+x+3D7T6cr0G2mrs5uPgwfvxMwSvfzG2ZUjYhi9cGEMoAFowCUNTM7doS0ffE5bD3Anhbp07jLagnPAoQFogGmAnRD6xv5Zunbj1opnhSgd9qJ0YKwx1tBAeQ0gFBdCcSEUFzTghQaswurGwqtKgSDmT7xEV2Z2eDGAeEKWf0JizDBmOhqwCKvbpQJBzBx9EbDCEx8agAYSDViE1VIEqxKBIHToizx4akMDvamB2mClEggCoutN0WFeMa86GrALq5KBIKQdmmjSYPCN8OaE7uS3qKGVXzefbjuRT6oFLJd6fslsFVZlA0FUL1Bd6OjmA3Sqn1OMca+OsVVYlQ0EMXdqY8VPC13o6OaDIfWqIaFf1WvbKqx0AkEUi4CHRvv3ZiM5K2qwOcHBboKagyuBIRotNrgZ+ZMlpQCkcMkZ5R9sNtPLR+7ewECDWliSVPyQqd4wAB/3xtgqrHQCQZSDVQCTRisylBaDVgyOCFRpeJWBFQPXAEWAW6KJZnRqaLRXJkAtqLezHvcmHsaIOfFNA1ZhpRMIohys+M12DiKh55Pl8Sh6Viz/YJMmEo9JLJsL5cW8rxiY8LDgYUEDxjRgFVY6gSD8gBWWfr49pdFe/zxLq7DSCQRhBFZLKsvAKE281IuWkfxSr2gZmLXE9E8MMGDMmcsasAornUAQZmAV7y0VbbDH+1ztNI1GehM9hJfKBjuWgS4LHm3zF8hWYVU2EMSp9x43tt6FSP0VKeYOc8c0YBVWF441lQNBnHrvMZo/txWwwgYtNAANhBqwCquwwsU5mp/aTif3PUMHWnfTni0P0eSHm+nalfMQJUQJDUADuRqwDiu49HDpoQFoQEcDgBWeZPBmoAEvNABYQaheCFXnSYw8veXBAVaAFWAFDXihAcAKQvVCqPCSestL0plPq7BCwAgITkekyAPdWP7qAgJGwOhgdNCAvgaselYIGKE/URA5xq7fNVAbrBAwAsbX78aH/pezAbuwMh0wgj/4LhU8AscOwxDKGQLGy/3xMg6r8fFxGh0dzXzDZD5gRB6UACsYn/vGhzkqN0dGYTU2NkarV6+mVatWZcLKfMAIwAqCLyd4jJe/42UMVjGohoaGcmFlJWAEfy56iw/ywAeP8HfCYGyYu37VgBFY8aAaHh7OhVX1ASPSJ3sOxOemdx0MFQbSrwaCfruj/a5hJYKqCFb2AkZ0Lg9bDe7IYnxrHd9ahwa800DXsGKb6WyPir9GRkYyB8JewAgRVsL56hCqd0KFh+OOh1PXXHQNqzINtxcwIorzl8Tvyw3FBQGUmT+khV7q1IBVWNkLGBF5Vo0GC0YaBXlIotbAq4JXBQ14qQGrsELACDyZ63wyo26/9WcVVggY4bdYYOyYvzo1YBVW4TEPCBjhpQtep0hRNyBp+YgYDDiMDhqABvQ1YN2zwmTpTxbGDmPXzxoArPBmCMtSaMALDQBWEKoXQu1njwJ9jzxqwAqwAqygAS804CSsEFgCezPwJqABUQMOwgqBJWCoMFRooFMDDsJqiRBYAsYKY4UGPPCs0rBCYAmIFuCCBpzdYF80HljCwGTjAD8vNmEBNgNad/SFg5PLQPOBJXQmEEEnYPg6ukGeqnTjJKzMB5bQERBgVZXoUK6OHpHHSVhVFlii2UjOt0oO5gtcXnbkcXzu1UCjFSx3osP7Oj9jZ7xnnDraCspNnfce521Qy1GXGsCA8fumASdhVU1giQAgIYgCkTK4DGSBhPemRM+K+5vlj8tqwy463E/IE6TjoeibONBeAM0lDTgJq2oDS2RDJfGighNGm3w4r/B3MQ+DUgy74PdMr6rtXXFQc2ni0RaAyDcNOAmragNLCOBJnc/OlngqsFqiieZg6DXFP8OJx1nveGOIZX9lGnASVtUGlhBg1bHfpAarEEyDjcCr4peTQqAKCLcy4frmFaC93XuyTsKq2sAS4pIu2jAPl4EhfGJYcRvvyab7yr2lpXY+cZkXelfi5nz3EwWxYwz7XQNOwgqBJWCY/W6Y6L8n/xuIwBKAFYwVGvDifwPD/wNCYAns92DPDxrgNODkMhBPVTxVoQFowBvPCmKFWKEBaIDXADwrLDWw1IAGvNAAYAWheiFUeFnwsgArwAqwgga80IBVWCEQBJ6O8JCgAV0NWIQVAkHoThLywcChActxAxEIAkYHo4MGdDVg0bNCIAjdSUI+GDg0YNuzMh4IgjvsLhXQAUcSQ9wAXK9pwKpnZT4QRB6UAKteEyr6A/hahZX5QBCAFYwYRtwvGrAKq8oCQXQcQ9yGWGvlbKnUWeipM6fY4XnR2VRJGqUAENw5WMH5VdEZ7LgwBtBAVRqwCqtqAkFknezZjk6TOhs95wTQJKhDfK66cL56ciZ7IMIkrQA3QAqghgYq14BVWFUbCKIoMk106mfo/YgnebJTPVNRbzgvKS8tzlqvXJhVPZ1Rrr+en1VYVRsIoghWXKy/ItCEIbpEWGWE7AKsACt4UtY1YBVW1QaCEGHF7UGl4JIX1CFvGciVkwgUy0B4KP56KL7OnVVYVRsIotOzajQGk6jKqQ3wjqAOzdQGOwuvlR1hmVsyClGbscEO4/UVAr602yqsEAgCBu2LYaCd7mnVKqwQCMI9AcAoMSe+aMAqrNigIBAEjMMX40A73dKqdVhBAG4JAPOB+fBFA4AVXkFbfwXti3GgnW6BHLACrAAraMALDQBWEKoXQoWX45aXU8d8AFaAFWAFDXihAauwQsAIPB3reCKjzt7QnUVYIWAEjKY3jAbzWM88WoRV+gx2oiVi17kPHqGje5+la5depaWbO+j29TG6dfUtmj74mBeuKYRbj3Ax7v037rXB6vb1rQGsiE7v+zF9tPMXdP7j50JI3bzyOl3/YjN9euBRL2B1+667jLRTtxzdfLGxd5NfN29ePt3yAK7+AJddWBkPGJE1SQXnr6eCSpibYBNGJiuD3ecvE7BhZcjqlYFAJ78urMQxKPo7q92q+WV9xn1ztlNmLK3CynzAiJKwMvDWR1XwokFm5eMnSmb0/P2832P4ZEEtTxSyemVi0smv2hexbtW6VNPlAU3WZ9zvA1iZDxhRD6xUxFrWYGTpswxc5mnJylT1rMoAOs8DzPMEi8ALWNUDBRV915HGqmdlPmBEVtCGomARGaeJNqPTQdmVCiqR44WpAEAVAlmeUJ5XpOqNFKUrCx0dz0M2PioeZ9zOKmClMgZ1GCLqlIPZKqyMB4xgxxDH56cncFENFtFOlzp/PeMIYwFaMmPU3UvKMmJxmSjzpERIytpaZrmo2q+iOkUIyfqcBSsV2Mj6XQSGbvICOHLgdDNGVmFlPGBE+8TPtEfUucGeBIsIT/fMiXKTupc/6KrGIno4eflUlkdFECoyeBk4+HLLGGleWlkZJmClInZZOwCraqGiMkc6aazCynzAiGjQw2OIk0APIqy4YBGGYKUz0LKloczLUIFfGc8qLk/8qdI3HVip1CeDjOx+EfjLPGR0PE6VcUOa7iBpFVbmA0asdJ4BK/KwhIAQHcEiuvesdEWn4umoeFoiuMTlomzJlpdfFQZZ4FXtm2o6nf0yWb+7KVN3zpGvO0Dx42cVVsYDRrRDZ0Ub5PF+U+RZZQeLMLMM1BVgGY8ky9PKeuKXXQaW8eBk/ezGK1P1dEQQ6+QrgljWnJSBtmyMcN9TWPVCwAjeQGWGo+rxqBhMljeUly8PIir1yJarovGVNey6Pau8h4COVwYQmQORylha9ax8DxhRtPySGbFJD0iEkQrIxDR5oC1jtN14VjpLMlUwqqTLGkOVfCpGhTTVQMwqrNgk+hwwQhdWRWDIMxAZ3LLKzPOKdL2ZMvlUDb1MmbIHQB4UZG3Jm0dZPkCoGgipjqt1WKk2zLV0RfBQ8RLy8uvCSgSTjpdTdskn87pUjL1OWMWAz1uex+Oh0g/X9NkP7QGsFP5fMEu8ed4S/7lMQGUApurVlTW0MumzjD3L+9EFkqwtKmOeB6QyYw1o1etB5dkNYKUAKxXPqeySRdWjyvKgdGGg0w9dw80DWzdtl8FM5vmVnSPZwwb37UINsNKEFYRqV6gYb4w3YAVYGTk8EDABTKrWgFVYIWAEBF21oFF+72rMIqwQMAKG1LuGhLmtfm4twgoBIyDo6gWNMe7dMa4NVr0SMALG0bvGgbl1a27twqrugBHYTMdmOjTgrQaMw2p8fJxGR0czB6QXAkbgaevW0xbz0T/zYRRWY2NjtHr1alq1alUmrHohYASMo3+MA3Pt1lwbg1UMqqGhoVxY1R8wYonYEcdxgIiV89v5M7Da52K1j0xOn5Xl1uTBmDAf/aQBI7DiQTU8PJwLq1oCRoQH9GUFghAP4uOj2whHIwdlqES+6SfhoK8ApW0NdA0rEVRFsKonYEQneBLPKi94RMqrantiHVF0IFbbYkV9/a25rmHFNtPZHhV/jYyMZO5Z1RMwgoNV6jx2Fkgi5zz2VLr+FggAgfl3RQNdw6pMR+wFjIghxITGwYotCQebNMFeX4dAKgoeoRb0tEz/kRaGDw3oa8AqrGwGjGhOxIPCLwO5CM6DDWrkeVYJzLI24/UHG0LF2EED+hqwCqteCBgBsemLDWOHsetGA1Zh5XvAiG4GGnlhqNBAdxqwCis2WT4HjIDYuhMbxg/j140GrMOqm8YiL8QODfSvBgAr/GOrt//YCnD1F7gAK8AKsIIGvNAAYAWheiFUeFH95UVlzTdgBVgBVtCAFxpwElYILIGnKDwpaEDUgIOwQmAJGCoMFRro1ICDsEJgCQgVsIIGPIQVAkvAcGG40ED4hfIbN2jDhg00YEMQe/bsocuXL0s38xYrDywhnHGFDVbpnNjQB+oAlIo0YBVWGzdupE2bNtHs7GyhcVQfWAKwAhgABt80YB1WFy9epFdeeYWmpqZygVV9YAnAyjehor2Aq3VYMdExz6rZbNLx48czgWU+sER82F50PtVgUHcjOXgvfY/db7R4YXBnYHXcg4AAEWjAlgZqgRXr3PT0NK1du5bm5+c7gGU8sER4WugKhCaagwG0+FNCOUClThCNQIVgETBIWwaJevK1Vgus5ubmCj2rSgJLxMcZh5vpeeeyRwPFwnWF3hXOYsfGO16+OKMB67CamZmR7lkZDyzBoANYOSM6eA/wVHU0YB1WKm8DzQeWKLsMjOMMYhmoIyrkAYyq0IBVWKl+z8p4YAnmyofBTlU22PnIOPGScSVwRHrzHaKsQpQoE7ry5tQFBJaAWAEsaMCDf2ReIgSWgFABK2jAC1iF/we0OEfzU9vp5L5n6EDrbtqz5SGa/HAzXbtyHhvFeEMFDfShBqzuWeFpiaclNAAN6GoAsOrDJ5SuWJAPoKlTA4AVYIUlFTTghQYAKwjVC6HW+URH3W54lIAVYAVYQQNeaMBJWCFghBtPMngUmAeXNOAgrBAwwiWBoC0AlisacBBWCBjhijjQDoDKJQ04DysEjIDBuGQwaEt9erQKK9V/ZDYfMMLEMcYmyqhvomFkGHvfNWAVVvUFjDABGhNlwGB8Nxi0vz4NW4dVPQEjTIDGRBn1TTSMDGPvuwasw4oNmP2AEW3QtIITQ5MzrSaS75awY4zjs64GGq2V75yExxqrnIElBJng8qUDUBS3w3cxof0AYpUaqAVWrEN1BIwYiI82Dg/ii08D5QXGe09lTxdVCUARlZm0IxWcAkKvUugo23991QIr6wEj+AARYsAI4RTRJOqNwXPbkwAUHe3gglPgW9RefIsa0KsPetZhVUvAiCJYpSLYsDPX2x6SFVhFZ7zjqOT6DADw8WfsrcOqvoAR/Nnq3HKPLQnj5WFqWVZ2GRgvK9P50uG8ontJHEKE+oI3BY9aWQNWYaX6PSvzASPEN3n831zE5cEGNWLPqpsgE6kN9k5INhosyGq0cQ+vyp8nO7yweufKKqxUJ7t3A0bg6w+qGkC6esHg4vg7CaveDRgBWLloBGiTH2B0ElZMPL0ZMAKwAhj8AIOL8+QsrFwcLLQJhgYN1KcBwApvY5TfxsBQ6zNUjH2w2rpxgzZs2ED/D9wreqzgsLCzAAAAAElFTkSuQmCC" alt="" />
设置环境变量:
export HADOOP_HOME=/data/hadoop/hadoop-2.2.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
 
注意:$HADOOP/bin 和$HADOOP/sbin 目录下的文件都有可执行的权限
 
修改配置文件:
[tank@192 hadoop]$ vi core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/hadoop/tmp</value><description>(备注:请先在 /usr/hadoop 目录下建立 tmp 文件夹)A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.149.128:9000</value>
</property>
</configuration>
 备注:如没有配置hadoop.tmp.dir参数,此时系统默认的临时目录为:/tmp/hadoo-hadoop。而这个目录在每次重启后都会被干掉,必须重新执行format才行,否则会出错。
[tank@192 hadoop]$ vi hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/name</value>
<final>true</final>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/data/soft/hadoop/hadoop-2.2.0/hdfs/data</value>
</property>
</configuration>
 
文件必须已经预先创建好并存在!
 
[tank@192 hadoop]$ vi yarn-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>192.168.149.128:9001</value>
</property>
</configuration>
 
 
注意上面一定要填Ip,不要填localhost,不然eclipse会连接不到!
 
设置主从关系$HADOOP_HOME/etc/hadoop/目录下:
[hadoop@tank1 hadoop]$ vi masters 
192.168.183.130

//主机特有,从机可以不需要
[hadoop@tank1 hadoop]$ vi slaves

192.168.183.131
192.168.183.132
192.168.183.133

 
[hadoop@tank1 hadoop]$ hadoop namenode -format   //第一次需要
 
启动:
sbin/start-all.sh
 
查看状态:主机

[tank@192 hadoop-2.2.0]$ jps
2751 ResourceManager
2628 SecondaryNameNode
2469 NameNode

查看状态:从机
[hadoop@tank2 sbin]$ jps
1745 NodeManager
1658 DataNode

 

 
总共有5个hadoop线程 
 
访问地址查看hdfs 的运行状态:
http://192.168.149.128:50070/dfshealth.jsp
 
 
 从机运行一段时间后NodeManager停止了,查看日志信息:
 
 FATAL org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.ConnectException: Call From tank2/192.168.183.131 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:181)
 
解决方案:
yarn-site.xml 中添加配置
<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>192.168.183.130</value>//主机IP
 </property>
停止,重启
 

hadoop 分布式开发环境搭建的更多相关文章

  1. Hadoop Eclipse开发环境搭建

        This document is from my evernote, when I was still at baidu, I have a complete hadoop developme ...

  2. Hadoop项目开发环境搭建(Eclipse\MyEclipse + Maven)

    写在前面的话 可详细参考,一定得去看 HBase 开发环境搭建(Eclipse\MyEclipse + Maven) Zookeeper项目开发环境搭建(Eclipse\MyEclipse + Mav ...

  3. (转)Hadoop Eclipse开发环境搭建

    来源:http://www.cnblogs.com/justinzhang/p/4261851.html This document is from my evernote, when I was s ...

  4. 【原创干货】大数据Hadoop/Spark开发环境搭建

    已经自学了好几个月的大数据了,第一个月里自己通过看书.看视频.网上查资料也把hadoop(1.x.2.x).spark单机.伪分布式.集群都部署了一遍,但经历短暂的兴奋后,还是觉得不得门而入. 只有深 ...

  5. Hadoop基本开发环境搭建(原创,已实践)

    软件包: hadoop-2.7.2.tar.gz hadoop-eclipse-plugin-2.7.2.jar hadoop-common-2.7.1-bin.zip eclipse  jdk1.8 ...

  6. 【hadoop之翊】——windows 7使用eclipse下hadoop应用开发环境搭建

    由于一些缘故,这节内容到如今才写.事实上弄hadoop有一段时间了,能够编写一些小程序了,今天来还是来说说环境的搭建.... 说明一下:这篇文章的步骤是接上一篇的hadoop文章的:http://bl ...

  7. ubuntu14.04 Hadoop单机开发环境搭建MapReduce项目

    Hadoop官网:http://hadoop.apache.org/ 目前最新的版本是Hadoop 3.0.0-alpha1前提:java 1.6 版本以上 首先从官网下载压缩包(hadoop-3.0 ...

  8. hadoop分布式的环境搭建

    版本: 使用hadoop1.1.2    JDK为java7 1.下载hadoop 2.配置hadoop文件 3测试 1.下载hadoop: 1.1 在https://archive.apache.o ...

  9. hadoop本地开发环境搭建

    1:下载hadoop2.7.3并解压 2:配置hadoop2.7.3环境变量 HADOOP_HOME %HADOOP_HOME%\bin 3:下载hadoop-eclipse-plugin插件 网址: ...

随机推荐

  1. CentOS 安装git

    git安装 yum install git 初始化git “git init --bare 文件夹” 添加所有 “git add . ” 提交 “git commit -m "提交信息&qu ...

  2. kebu之rook-ceph

    准备工作 所有节点开启ip_forward cat <<EOF > /etc/sysctl.d/ceph.conf net.ipv4.ip_forward = 1 net.bridg ...

  3. python 模块之-pickle

    Pickle的问题和所有其他编程语言特有的序列化问题一样,就是它只能用于Python,并且可能不同版本的Python彼此都不兼容,因此,只能用Pickle保存那些不重要的数据,不能成功地反序列化也没关 ...

  4. 洛谷P2918 [USACO08NOV]买干草(一道完全背包模板题)

    题目链接 很明显的一道完全背包板子题,做法也很简单,就是要注意 这里你可以买比所需多的干草,只要达到数量就行了 状态转移方程:dp[j]=min(dp[j],dp[j-m[i]]+c[i]) 代码如下 ...

  5. Android 下载App

    转载:http://blog.csdn.net/aicpzl/article/details/52993074 通过DownloadManager来下载APK到本地,下载完成后收到广播再安装APK,可 ...

  6. 一个模拟——抢票部分功能的 简单版(主要实例化一下 Lock 的使用)

    """ 抢票! 多个用户在同时读写同一个数据 """ from multiprocessing import Process,Lock im ...

  7. 洛谷2754 [CTSC1999]家园

    题目链接:[CTSC1999]家园 这个题目我们不是很好在做网络流的时候判断是否有解,因此我们考虑分开来做 对于是否有解的判断,我们唯一需要解决的是飞船的周期停泊问题,对于这个问题,我们可以用并查集解 ...

  8. 【XSY2774】学习 带花树

    题目描述 给你一个图,求最大匹配. 边的描述方式很特殊,就是一次告诉你\(c_i\)个点:\(d_1,d_2,\ldots,d_{c_i}\),表示这些点两两之间都有连边,也就是说,这是一个团.总共有 ...

  9. 启发式合并&线段树合并/分裂&treap合并&splay合并

    启发式合并 有\(n\)个集合,每次让你合并两个集合,或询问一个集合中是否存在某个元素. ​ 我们可以用平衡树/set维护集合. ​ 对于合并两个\(A,B\),如果\(|A|<|B|\),那么 ...

  10. MT【154】拉格朗日配方

    (清华2017.4.29标准学术能力测试24) 设$x,y\in\mathbb{R}$,函数$f(x,y)=x^2+6y^2-2xy-14x-6y+72$的值域为$M$,则______ A.$1\in ...