前段時(shí)間由于配置hadoop做實(shí)驗(yàn),在網(wǎng)上看了許多有關(guān)hadoop的配置,但是這些配置多數(shù)是將namenode和secondaryNameNode配置在同一臺(tái)計(jì)算機(jī)上,這種配置方法如果是做做實(shí)驗(yàn)的還可以,如果應(yīng)用到實(shí)際中,存在較大風(fēng)險(xiǎn),如果存放namenode的主機(jī)出現(xiàn)問(wèn)題,整個(gè)文件系統(tǒng)將被破壞,嚴(yán)重的情況是所有文件都丟失?,F(xiàn)在來(lái)配置hadoop2.2,將namenode和secondaryNameNode配置在不同的機(jī)器上,這樣的實(shí)用價(jià)值更大。
1.假設(shè)我們已經(jīng)配置好了hadoop2.2,這時(shí)namenode和secondaryNameNode在同一臺(tái)機(jī)器上,如cloud001;
2.單獨(dú)配置secondaryNameNode文件,指明secondaryNameNode所在的計(jì)算機(jī)地址,需要masters文件(類似slaves文件指明從節(jié)點(diǎn)),而我下載的hadoop2.2配置文件中不包含masters,所以我們新建一個(gè)masters文件,編輯內(nèi)容為:
[html]
view plaincopysudo vim masters
[html]
view plaincopycloud002
此處將cloud002作為secondaryNameNode的主機(jī)。3.修改hdfs-site.xml的內(nèi)容
[html]
view plaincopy<!--
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>cloud001:9001</value>
</property>
-->
注釋掉之前使namenode與SecondaryNameNode在同一臺(tái)主機(jī)的代碼,并新添如下代碼[html]
view plaincopy<property>
<name>dfs.http.address</name>
<value>cloud001:50070</value>
<description>
The address and the base port where the dfs namenode web ui will listen on.
If the port is 0 then the server will start on a free port.
</description>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>cloud002:50090</value>
</property>
網(wǎng)上也有說(shuō)要修改core-site.xml的代碼,而我之前配置過(guò),所以現(xiàn)在就不需要再改了。修改的內(nèi)容如下:[html]
view plaincopy<property>
<name>fs.checkpoint.period</name>
<value>60</value>
<description>The number of seconds between two periodic checkpoints.
</description>
</property>
<property>
<name>fs.checkpoint.size</name>
<value>67108864</value>
</property>
4.以上修改只是在namenode上修改,在集群中的其他節(jié)點(diǎn)也要作相應(yīng)修改,簡(jiǎn)便方法便是將需要修改的文件覆蓋掉。5.啟動(dòng)hdfs,在hadoop目錄下執(zhí)行:
[html]
view plaincopysbin/start-dfs.sh
[html]
view plaincopyxuhui@cloud001:~/hadoop-2.2.0$ sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [cloud001]
cloud001: starting namenode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-namenode-cloud001.out
cloud004: starting datanode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-datanode-cloud004.out
cloud002: starting datanode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-datanode-cloud002.out
Starting secondary namenodes [cloud002]
cloud002: starting secondarynamenode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-secondarynamenode-cloud002.out
starting yarn daemons
starting resourcemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-resourcemanager-cloud001.out
cloud004: starting nodemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-nodemanager-cloud004.out
cloud002: starting nodemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-nodemanager-cloud002.out
xuhui@cloud001:~/hadoop-2.2.0$ ls
6.檢查配置成功與否:[html]
view plaincopyxuhui@cloud001:~/hadoop-2.2.0$ jps
3490 ResourceManager
3123 NameNode
4784 Jps
xuhui@cloud001:~/hadoop-2.2.0$
[html]
view plaincopyxuhui@cloud002:~$ jps
7162 SecondaryNameNode
7412 NodeManager
6624 DataNode
7698 Jps
xuhui@cloud002:~$
[html]
view plaincopyxuhui@cloud004:~$ jps
6287 NodeManager
6409 Jps
5836 DataNode
xuhui@cloud004:~$
驗(yàn)證結(jié)果很顯然是配置成功的!
7.關(guān)閉hdfs:[html]
view plaincopyxuhui@cloud001:~/hadoop-2.2.0$ sbin/stop-dfs.sh
Stopping namenodes on [cloud001]
cloud001: stopping namenode
cloud004: stopping datanode
cloud002: stopping datanode
Stopping secondary namenodes [cloud002]
cloud002: stopping secondarynamenode
xuhui@cloud001:~/hadoop-2.2.0$