adoop版本:hadoop-0.20.2
Eclipse版本:eclipse-java-helios-SR2-linux-gtk.tar.gz
========================安裝eclipse=======================
1、首先下載eclipse,我下載的是:eclipse-java-helios-SR2-linux-gtk.tar.gz
2、安裝eclipse
(1)把eclipse-java-helios-SR2-linux-gtk.tar.gz解壓到某個(gè)目錄中,我解壓到的是/home/wangxing/Development,得到eclipse目錄
(2)在/usr/bin目錄下創(chuàng)建一個(gè)啟動(dòng)腳本eclipse,執(zhí)行下面的命令來創(chuàng)建:
sudo gedit /usr/bin/eclipse
然后在該文件中添加以下內(nèi)容:
#!/bin/sh
export MOZILLA_FIVE_HOME="/usr/lib/mozilla/"
export ECLIPSE_HOME="/home/wangxing/Development/eclipse"
$ECLIPSE_HOME/eclipse $*
(3)修改該腳本的權(quán)限,讓它變成可執(zhí)行,執(zhí)行下面的命令:
sudo chmod +x /usr/bin/eclipse
3、在Applications(應(yīng)用程序)菜單上添加一個(gè)圖標(biāo)
sudo gedit /usr/share/applications/eclipse.desktop
然后在文件中添加下列內(nèi)容:
[Desktop Entry]
Encoding=UTF-8
Name=Eclipse Platform
Comment=Eclipse IDE
Exec=eclipse
Icon=/home/wangxing/Development/eclipse/icon.xpm
Terminal=false
StartupNotify=true
Type=Application
Categories=Application;Development;
========================安裝hadoop============================
關(guān)于hadoop在Linux下偽分布式的安裝詳見:http://blog.sina.com.cn/s/blog_62186b4601011c6a.html
========================在eclipse配置hadoop====================
1.在eclipse上安裝hadoop插件
復(fù)制 hadoop安裝目錄/contrib/eclipse-plugin/hadoop-0.20.203.0-eclipse-plugin.jar 到eclipse安裝目錄/plugins/ 下
2.重啟eclipse,配置hadoop installation directory。
如果安裝插件成功,打開Window-->Preferens,你會(huì)發(fā)現(xiàn)Hadoop Map/Reduce選項(xiàng),在這個(gè)選項(xiàng)里你需要配置Hadoop installation directory。配置完成后退出。
3.配置Map/Reduce Locations
在Window-->Show View中打開Map/Reduce Locations,在Map/Reduce Locations中新建一個(gè)Hadoop Location。在這個(gè)View中,右鍵-->New Hadoop Location。在彈出的對(duì)話框中你需要配置Location name,如Hadoop,還有Map/Reduce Master和DFS Master。這里面的Host、Port分別為你在mapred-site.xml、core-site.xml中配置的地址及端口。如:
本人hadoop的配置詳見:http://www.oschina.net/code/snippet_176897_7976
4.新建項(xiàng)目。
File-->New-->Other-->Map/Reduce Project,項(xiàng)目名可以隨便取,如WordCount。
復(fù)制 hadoop安裝目錄/src/example/org/apache/hadoop/examples/WordCount.java到剛才新建的項(xiàng)目WordCount下,刪除WordCount.java首行package
5.在本地新建word.txt,內(nèi)容為:
java c++ python cjava c++ javascript helloworld hadoopmapreduce java hadoop hbase
6.通過hadoop的命令在HDFS上創(chuàng)建/tmp/workcount目錄,命令如下:
bin/hadoop fs -mkdir /tmp/wordcount
通過copyFromLocal命令把本地的word.txt復(fù)制到HDFS上,命令如下:
bin/hadoop fs -copyFromLocal /home/wangxing/Development/eclipseWorkspace/word.txt
7.運(yùn)行項(xiàng)目
(1).在新建的項(xiàng)目Hadoop,點(diǎn)擊WordCount.java,右鍵-->Run As-->Run Configurations
(2).在彈出的Run Configurations對(duì)話框中,點(diǎn)Java Application,右鍵-->New,這時(shí)會(huì)新建一個(gè)application名為WordCount
(3).配置運(yùn)行參數(shù),點(diǎn)Arguments,在Program arguments中輸入你要傳給程序的輸入文件夾和你要求程序?qū)⒂?jì)算結(jié)果保存的文件夾,如:
hdfs://localhost:9000/tmp/wordcount/word.txt
(4)點(diǎn)擊Run,運(yùn)行程序
過段時(shí)間將運(yùn)行完成,等運(yùn)行結(jié)束后,查看例子的輸出結(jié)果,使用命令:
bin/hadoop fs -ls /tmp/wordcount/out
發(fā)現(xiàn)有兩個(gè)文件夾和一個(gè)文件,使用命令查看part-r-00000里的運(yùn)行結(jié)果:
bin/hadoop fs -cat /tmp/wordcount/out/part-r-00000
參考:
http://www.ylmf.net/ubuntu/guide/201009051389.html
http://www.cnblogs.com/flyoung2008/archive/2011/12/09/2281400.html
聯(lián)系客服