解决使用MySql存储Hive元数据遇到的Unable to instantiate问题_Sicilly_琬姗

网络投稿 02-07 3163

问题描述和解决过程

在安装好hive后，直接在控制台输入hive，即可启动hive。但输入show databases;出现了关于下列的异常：

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient；

翻译过来就是hive元数据连接有问题。

然后发现要先安装mysql和mysql的驱动，再配置一个hive-site.xml（在hive安装目录下conf文件夹里有一个模板文件）

然后我安装了mysql5.7

然后下载了mysql的驱动mysql-connector-java-5.1.49.jar,复制mysql的驱动程序到hive/lib下面

然后填写hive-site.xml（注意要填写mysql的地址和用户名和密码，mysql连接地址这里会有坑，后面说）

<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name>  <value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true&useSSL=false</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>false</value> </property> </configuration>

然后执行schematool -dbType mysql -initSchema就开始报错了！

第一次报错：Communications link failure

我推测是jdbc那一行的地址有问题，于是修改了hive-site.xml,把master改为我本机ip，发现还是报错：

amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ vi hive-site.xml amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/amax/Downloads/apache-hive-2.3.9-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://172.31.58.2:3306/hive?createDatabaseIfNotExist=true&useSSL=false Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: root org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. SQL Error code: 0 Use --verbose for detailed stacktrace. *** schemaTool failed ***

然后我查看了3306这个端口，发现有被监听，但写的是127.0.0.1，所以干脆在hive-site.xml中改成127.0.0.1:3306试试，发现这次是另外一个错了not allowed to connect to this MySQL server

amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ netstat -an|grep 3306 tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ vi hive-site.xml amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/amax/Downloads/apache-hive-2.3.9-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true&useSSL=false Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: root org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. Underlying cause: java.sql.SQLException : null, message from server: "Host '172.31.59.5' is not allowed to connect to this MySQL server" SQL Error code: 1130 Use --verbose for detailed stacktrace. *** schemaTool failed ***

估计是权限问题，于是我登录mysql给这个ip授权

amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 5.7.33-0ubuntu0.16.04.1 (Ubuntu) Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> GRANT ALL ON *.* to root@172.31.59.5 IDENTIFIED BY '123456'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> EXIT Bye

再尝试一次，这回终于成功了！

amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ schematool -dbType mysql -initSchema SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/amax/Downloads/apache-hive-2.3.9-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true&useSSL=false Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: root Starting metastore schema initialization to 2.3.0 Initialization script hive-schema-2.3.0.mysql.sql Initialization script completed schemaTool completed amax@master:~/Downloads/apache-hive-2.3.9-bin/conf$ hive SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/amax/Downloads/apache-hive-2.3.9-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/home/amax/Downloads/apache-hive-2.3.9-bin/lib/hive-common-2.3.9.jar!/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive (default)> show databases; OK database_name default Time taken: 5.296 seconds, Fetched: 1 row(s)