Apache
Cloudera
Storm
分布式计算 / 实时流计算 / NoSQL存储 (Email: ypf412@163.com, GitHub: https://github.com/ypf412)
hortonworks
考试内容列表
Other
Once the Hadoop cluster is up and running check the web-ui of the components as described below:
| Daemon | Web Interface | Notes |
| NameNode | http://nn_host:port/ | Default HTTP port is 50070. |
| ResourceManager | http://rm_host:port/ | Default HTTP port is 8088. |
| MapReduce JobHistory Server | http://jhs_host:port/ | Default HTTP port is 19888. |
- 1 通过Web界面来查看NameNode运行状况，默认为：
NameNode or
- 2 通过Web界面来查看ResourceManager运行状况，默认为：
- 3 MapReduce JobHistory Server
MapReduce JobHistory Server or
Operating the Hadoop Cluster
Once all the necessary configuration is complete, distribute the files to the HADOOP_CONF_DIR directory on all the machines.
This section also describes the various Unix users who should be starting the various components and uses the same Unix accounts and groups used previously:
Hadoop Startup
To start a Hadoop cluster you will need to start both the HDFS and YARN cluster.
Format a new distributed filesystem as hdfs:
[hdfs]$ $HADOOP_PREFIX/bin/hdfs namenode -format
[hdfs]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR –script hdfs start namenode Run a script to start DataNodes on all slaves as root with a special environment variable HADOOP_SECURE_DN_USER set to hdfs:
[root]$ HADOOP_SECURE_DN_USER=hdfs $HADOOP_PREFIX/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR –script hdfs start datanode Start the YARN with the following command, run on the designated ResourceManager as yarn:
[yarn]$ $HADOOP_YARN_HOME/sbin/yarn-daemon.sh –config $HADOOP_CONF_DIR start resourcemanager Run a script to start NodeManagers on all slaves as yarn:
[yarn]$ $HADOOP_YARN_HOME/sbin/yarn-daemon.sh –config $HADOOP_CONF_DIR start nodemanager Start a standalone WebAppProxy server. Run on the WebAppProxy server as yarn. If multiple servers are used with load balancing it should be run on each of them:
[yarn]$ $HADOOP_YARN_HOME/bin/yarn start proxyserver –config $HADOOP_CONF_DIR Start the MapReduce JobHistory Server with the following command, run on the designated server as mapred:
[mapred]$ $HADOOP_PREFIX/sbin/mr-jobhistory-daemon.sh start historyserver –config $HADOOP_CONF_DIR
Hadoop Shutdown
Stop the NameNode with the following command, run on the designated NameNode as hdfs:
[hdfs]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR –script hdfs stop namenode Run a script to stop DataNodes on all slaves as root:
[root]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR –script hdfs stop datanode Stop the ResourceManager with the following command, run on the designated ResourceManager as yarn:
[yarn]$ $HADOOP_YARN_HOME/sbin/yarn-daemon.sh –config $HADOOP_CONF_DIR stop resourcemanager Run a script to stop NodeManagers on all slaves as yarn:
[yarn]$ $HADOOP_YARN_HOME/sbin/yarn-daemon.sh –config $HADOOP_CONF_DIR stop nodemanager Stop the WebAppProxy server. Run on the WebAppProxy server as yarn. If multiple servers are used with load balancing it should be run on each of them:
[yarn]$ $HADOOP_YARN_HOME/bin/yarn stop proxyserver –config $HADOOP_CONF_DIR Stop the MapReduce JobHistory Server with the following command, run on the designated server as mapred:
[mapred]$ $HADOOP_PREFIX/sbin/mr-jobhistory-daemon.sh stop historyserver –config $HADOOP_CONF_DIR
