安装sqoop_wyju_安装sqoop

大大的周 02-07 2113

1.sqoop的下载地址 https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz 2.上传sqoop 3.解压sqoop sudo tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /usr/local/ 4.进入/usr/local目录 cd /usr/local 5.将解压后的sqoop-1.4.7.bin__hadoop-2.6.0目录重命名为sqoop sudo mv sqoop-1.4.7.bin__hadoop-2.6.0/ sqoop 6.授权当前用户hadoop拥有sqoop目录的所有者权限 sudo chown -R hadoop:hadoop sqoop/

7.进入到sqoop目录下的conf里面 cd /usr/local/sqoop/conf/

8.重命名sqoop-env-template.sh为sqoop-env.sh mv sqoop-env-template.sh sqoop-env.sh 9.修改配置sqoop-env.sh,配置hadoop、hbase、zookeeper的安装目录 export HADOOP_COMMON_HOME=/usr/local/hadoop #Set path to where hadoop-*-core.jar is available export HADOOP_MAPRED_HOME=/usr/local/hadoop #set the path to where bin/hbase is available export HBASE_HOME=/usr/local/hbase #Set the path to where bin/hive is available export HIVE_HOME=/usr/local/hive #Set the path for where zookeper config dir is export ZOOCFGDIR=/usr/local/zookeeper

10.进入到sqoop的lib目录下，把mysql-connector-java-5.1.47.jar驱动上传到lib中 cd /usr/local/sqoop/lib/

11.拷贝hive的jar包到lib的目录中 cp /usr/local/hive/lib/hive-common-2.3.7.jar /usr/local/sqoop/lib/ 12.编辑环境变量 a.进入环境变量 vim ~/.bashrc b.添加以下内容 export SQOOP_HOME=/usr/local/sqoop export PATH=$PATH:$SQOOP_HOME/bin c.使环境变量生效 source ~/.bashrc 13.验证是否成功 sqoop version

使用sqoop之前的准备 1.启动zookeeper（三台都要启动）

zkServer.sh start

2.启动hadoop（在master上启动） start-all.sh

3.启动hbase（在master上） start-hbase.sh

4.所有的节点

5.启动hive（在msater） hiveserver2

6.启动mysql mysql -uroot -p123456 7.创建sqoop数据库 create database if not exists sqoop;

8.使用sqoop数据库 use sqoop;

9.创建表 CREATE TABLE IF NOT EXISTS `student`( `id` int PRIMARY KEY COMMENT '编号', `name` varchar(20) COMMENT '名字', `age` int COMMENT '年龄' )COMMENT '学生表'; 10.插入数据 INSERT INTO student VALUES(1, 'zhangsan', 20); INSERT INTO student VALUES(2, 'lisi', 24); INSERT INTO student VALUES(3, 'wangwu', 18); INSERT INTO student VALUES(4, 'zhaoliui', 22); 11.把mysql中student表中的数据导入到hdfs上 sqoop import --connect jdbc:mysql://localhost:3306/sqoop --username root --password 123456 --table student --target-dir /user/student --delete-target-dir -m 1 12.查看是否成功 hdfs dfs -cat /user/student/part-m-00000