MacOS下Hadoop3的安装步骤

这篇文章主要介绍“MacOS下Hadoop3的安装步骤”，在日常操作中，相信很多人在MacOS下Hadoop3的安装步骤问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”MacOS下Hadoop3的安装步骤”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！

创新互联建站是创新、创意、研发型一体的综合型网站建设公司，自成立以来公司不断探索创新，始终坚持为客户提供满意周到的服务，在本地打下了良好的口碑，在过去的十多年时间我们累计服务了上千家以及全国政企客户，如成都LED显示屏等企业单位，完善的项目管理流程，严格把控项目进度与质量监控加上过硬的技术实力获得客户的一致赞美。

1 检查JDK

Matching Java Virtual Machines (1):

1.8.0_201, x86_64:"Java SE 8"/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home

/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home

2 安装Brew

nancylulululudeMacBook-Air:~ nancy$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

3 密钥

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

nancylulululudeMacBook-Air:.ssh nancy$ ls

authorized_keysid_rsaid_rsa.pubknown_hosts

nancylulululudeMacBook-Air:.ssh nancy$ chmod 0600 ~/.ssh/authorized_keys

4 brew安装hadoop

brew install hadoop

5 hadoop配置

路径：

/usr/local/Cellar/hadoop/3.1.1/libexec/etc/hadoop

core-site.xml

fs.default.name

hdfs://localhost:8020

hadoop.tmp.dir

file:/usr/local/Cellar/hadoop/tmp

fs.trash.interval

4320

hdfs-site.xml

dfs.namenode.name.dir

file:/usr/local/Cellar/hadoop/tmp/dfs/name

dfs.datanode.data.dir

file:/usr/local/Cellar/hadoop/tmp/dfs/data

dfs.replication

dfs.webhdfs.enabled

true

dfs.permissions.superusergroup

admin

dfs.permissions.enabled

false

yarn-site.xml

yarn.resourcemanager.hostname

localhost

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.nodemanager.aux-services.mapreduce.shuffle.class

org.apache.hadoop.mapred.ShuffleHandler

yarn.resourcemanager.address

localhost:18040

yarn.resourcemanager.scheduler.address

localhost:18030

yarn.resourcemanager.resource-tracker.address

localhost:18025

yarn.resourcemanager.admin.address

localhost:18141

yarn.resourcemanager.webapp.address

localhost:18088

yarn.log-aggregation-enable

true

yarn.log-aggregation.retain-seconds

86400

yarn.log-aggregation.retain-check-interval-seconds

86400

yarn.nodemanager.remote-app-log-dir

/tmp/logs

yarn.nodemanager.remote-app-log-dir-suffix

logs

mapred-site.xml

mapreduce.framework.name

yarn

mapreduce.jobtracker.http.address

localhost:50030

mapreduce.jobhisotry.address

localhost:10020

mapreduce.jobhistory.webapp.address

localhost:19888

mapreduce.jobhistory.done-dir

/jobhistory/done

mapreduce.intermediate-done-dir

/jobhisotry/done_intermediate

mapreduce.job.ubertask.enable

true

slaves

localhost

6 创建文件夹

nancylulululudeMacBook-Air:hadoop nancy$ mkdir /usr/local/Cellar/hadoop/tmp

nancylulululudeMacBook-Air:hadoop nancy$ mkdir -p /usr/local/Cellar/hadoop/tmp/dfs/name

nancylulululudeMacBook-Air:hadoop nancy$ mkdir /usr/local/Cellar/hadoop/tmp/dfs/data

7 hdfs格式化

hdfs namenode -format

格式化成功

2019-03-22 16:48:24,632 INFO common.Storage: Storage directory /usr/local/Cellar/hadoop/tmp/dfs/name has been successfully formatted.

Here comes the last step~

startup！

nancylulululudeMacBook-Air:3.1.1 nancy$ cd /usr/local/Cellar/hadoop/3.1.1/sbin

nancylulululudeMacBook-Air:sbin nancy$ ./start-all.sh

WARNING: Attempting to start all Apache Hadoop daemons as nancy in 10 seconds.

WARNING: This is not a recommended production deployment configuration.

WARNING: Use CTRL-C to abort.

Starting namenodes on [localhost]

Starting datanodes

Starting secondary namenodes [nancylulululudeMacBook-Air.local]

nancylulululudeMacBook-Air.local: Warning: Permanently added 'nancylulululudemacbook-air.local,192.168.1.142' (ECDSA) to the list of known hosts.

2019-03-22 17:24:04,021 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Starting resourcemanager

Starting nodemanagers

nancylulululudeMacBook-Air:sbin nancy$ jps

3266 NodeManager

3171 ResourceManager

2980 SecondaryNameNode

2748 NameNode

2847 DataNode

3327 Jps

登陆地址 http://localhost:9870/

http://localhost:8088/

到此，关于“MacOS下Hadoop3的安装步骤”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注创新互联网站，小编会继续努力为大家带来更多实用的文章！