hadoop学习笔记之-hbase完全分布模…

2019-04-14 21:30发布

安装环境: OS: Oracle linux 5.6 JDK: jdk1.6.0_18 Hadoop: hadoop-0.20.2 Hbase: hbase-0.90.5 安装准备: 1. Jdk环境已安装:版本为1.6以上 2. hadoop环境已安装:完全分布模式安装如下 http://blog.csdn.net/lichangzai/article/details/8206834 3. hbase版本选择 Hbase 版本必需与 Hadoop版本匹配,否则会安装失败或不能正常使用。关于两者何种版本能正常匹配,可以看官方文档或在网上搜寻安装的成功案例。 4. hbase软件下载 http://mirror.bjtu.edu.cn/apache/hbase/hbase-0.90.5/ 安装概述: l 配置hosts,确保涉及的主机名均可以解析为ip l 编辑hbase-env.xml l 编辑hbase-site.xml l 编辑regionservers文件 l Hbase复制到其它节点 l 启动Hbase l 验证启动 安装步骤: 1. 配置hosts 此步在配置hadoop时已经完成,如下: [root@gc ~]$ cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 192.168.2.101 rac1.localdomain rac1 192.168.2.102 rac2.localdomain rac2 192.168.2.100 gc.localdomain gc 2. 拷贝并解压安装包 [grid@gc ~]$ pwd /home/grid [grid@gc ~]$ tar -xzvf hbase-0.90.5.tar.gz 3. 替换hadoop核心jar包 主要目的是防止因为hbase和hadoop版本不同出现兼容问题,造成hmaster启动异常 $ pwd /home/grid/hbase-0.90.5/lib $ mv hadoop-core-0.20-append-r1056497.jar hadoop-core-0.20-append-r1056497.jar.bak $ cp /home/grid/hadoop-0.20.2/hadoop-0.20.2-core.jar /home/grid/hbase-0.90.5/lib/ $ chmod 775 hadoop-0.20.2-core.jar 4. 编辑hbase-env.xml [grid@gc conf]$ pwd /home/grid/hbase-0.90.5/conf [grid@gc conf]$ vi hbase-env.sh # 添加如下内容 # The java implementation to use. Java 1.6 required. export JAVA_HOME=/usr/java/jdk1.6.0_18 # Extra Java CLASSPATH elements. Optional. export HBASE_CLASSPATH=/home/grid/hadoop-0.20.2/conf # Where log files are stored. $HBASE_HOME/logs by default. export HBASE_LOG_DIR=${HBASE_HOME}/logs # Tell HBase whether it should manage it's own instance of Zookeeper or not. export HBASE_MANAGES_ZK=true 5. 编辑hbase-site.xml [grid@gc conf]$ vi hbase-site.xml # 添加如下内容   hbase.rootdir #设置hbase数据库存放数据的目录 hdfs://gc:9000/hbase     hbase.cluster.distributed #打开hbase分布模式 true     hbase.master #指定hbase集群主控节点 gc:60000     hbase.zookeeper.quorum gc,rac1,rac2 #指定zookeeper集群节点名,因为是由zookeeper表决算法决定的     hbase.zookeeper.property.dataDir #zookeeper集群data目录 /home/grid/hbase-0.90.5/zookeeper   6. 编辑regionservers文件 [grid@gc conf]$ cat regionservers #localhost改为如下 rac1 rac2 7. 将修改的hbase目录同步其它节点 --分别同步到rac1,rac2两节点 [grid@gc ~]$ scp -r hbase-0.90.5 rac1:/home/grid/ [grid@gc ~]$ scp -r hbase-0.90.5 rac2:/home/grid/ 8. 启动/关闭Hbase数据库集群 --启动hbase之前必需检查hadoop是否已经启动 [grid@gc ~]$ hadoop-0.20.2/bin/hadoop dfsadmin -report Configured Capacity: 45702094848 (42.56 GB) Present Capacity: 3562618880 (3.32 GB) DFS Remaining: 3562348544 (3.32 GB) DFS Used: 270336 (264 KB) DFS Used%: 0.01% Under replicated blocks: 4 Blocks with corrupt replicas: 0 Missing blocks: 0   ------------------------------------------------- Datanodes available: 2 (2 total, 0 dead)   Name: 192.168.2.101:50010 Decommission Status : Normal Configured Capacity: 22851047424 (21.28 GB) DFS Used: 135168 (132 KB) Non DFS Used: 20131606528 (18.75 GB) DFS Remaining: 2719305728(2.53 GB) DFS Used%: 0% DFS Remaining%: 11.9% Last contact: Tue Dec 25 09:40:14 CST 2012     Name: 192.168.2.102:50010 Decommission Status : Normal Configured Capacity: 22851047424 (21.28 GB) DFS Used: 135168 (132 KB) Non DFS Used: 22007869440 (20.5 GB) DFS Remaining: 843042816(803.99 MB) DFS Used%: 0% DFS Remaining%: 3.69% Last contact: Tue Dec 25 09:40:13 CST 2012 --启动Hbase集群 ----gc master节点 [grid@gc ~]$ hbase-0.90.5/bin/start-hbase.sh rac2: starting zookeeper, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-zookeeper-rac2.localdomain.out gc: starting zookeeper, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-zookeeper-gc.localdomain.out rac1: starting zookeeper, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-zookeeper-rac1.localdomain.out starting master, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-master-gc.localdomain.out rac1: starting regionserver, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-regionserver-rac1.localdomain.out rac2: starting regionserver, logging to /home/grid/hbase-0.90.5/bin/../logs/hbase-grid-regionserver-rac2.localdomain.out --可以看到多出两个hbase进程 [grid@gc ~]$ jps 2718 HQuorumPeer 6875 JobTracker 6799 SecondaryNameNode 8129 org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar 2864 Jps 6651 NameNode 2772 HMaster --rac1,rac2 slave节点 [grid@rac1 ~]$ jps 23663 HRegionServer 3736 DataNode 23585 HQuorumPeer 23737 Jps 3840 TaskTracker [grid@rac2 ~]$ jps 10579 TaskTracker 29735 HQuorumPeer 29897 Jps 10480 DataNode 29812 HRegionServer --通过浏览器验证: http://192.168.2.100:60010/master.jsp --关闭hbase集群 [grid@gc hbase-0.90.5]$ bin/stop-hbase.sh stopping hbase................... gc: stopping zookeeper. rac2: stopping zookeeper. rac1: stopping zookeeper. 命令行操作: 1. 常用hbase命令 --进入habase [grid@gc ~]$ hbase-0.90.5/bin/hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to leave the HBase Shell Version 0.90.5, r1212209, Fri Dec 9 05:40:36 UTC 2011 hbase(main):001:0> --查看数据库状态 hbase(main):002:0> status 2 servers, 0 dead, 1.0000 average load --查询数据库版本 hbase(main):004:0> version 0.90.5, r1212209, Fri Dec 9 05:40:36 UTC 2011 --帮助命令 hbase(main):003:0> help HBase Shell, version 0.90.5, r1212209, Fri Dec 9 05:40:36 UTC 2011 Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command. Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.   COMMAND GROUPS: Group name: general Commands: status, version   Group name: ddl Commands: alter, create, describe, disable, drop, enable, exists, is_disabled, is_enabled, list   Group name: dml Commands: count, delete, deleteall, get, get_counter, incr, put, scan, truncate   Group name: tools Commands: assign, balance_switch, balancer, close_region, compact, flush, major_compact, move, split, unassign, zk_dump   Group name: replication Commands: add_peer, disable_peer, enable_peer, remove_peer, start_replication, stop_replication   SHELL USAGE: Quote all names in HBase Shell such as table and column names. Commas delimit command parameters. Type after entering a command to run it. Dictionaries of configuration used in the creation and alteration of tables are Ruby Hashes. They look like this:   {'key1' => 'value1', 'key2' => 'value2', ...}   and are opened and closed with curley-braces. Key/values are delimited by the '=>' character combination. Usually keys are predefined constants such as NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type 'Object.constants' to see a (messy) list of all constants in the environment.   If you are using binary keys or values and need to enter them in the shell, use double-quote'd hexadecimal representation. For example:   hbase> get 't1', "keyx03x3fxcd" hbase> get 't1', "key