flink部署在hadoop上:超详细步骤 整合Apache Hudi
flink部署在hadoop上:超详细步骤 整合Apache Hudi
以下三个包也要放到Flink的lib下,否则同步数据到Hive时会报错
1.3 部署同步到Hive的环境
将hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar包放入到以下路径
[flink@dbos-bigdata-test005 jars]$ pwd
/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/jars
进入到hive lib目录,每一台hive节点都要放置jar包
[flink@dbos-bigdata-test005 lib]$ pwd
/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hive/lib
//建立软链接
[flink@dbos-bigdata-test005 lib]$ ln -ls ../../../jars/hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar
1.4. 安装 YARN MapReduce 框架 JAR
进入平台操作,安装YARN MapReduce框架JAR
![]()