快捷搜索:  汽车  科技

flink部署在hadoop上:超详细步骤 整合Apache Hudi

flink部署在hadoop上:超详细步骤 整合Apache Hudi

以下三个包也要放到Flink的lib下,否则同步数据到Hive时会报错

flink部署在hadoop上:超详细步骤 整合Apache Hudi(1)

1.3 部署同步到Hive的环境

将hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar包放入到以下路径

[flink@dbos-bigdata-test005 jars]$ pwd/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/jars

flink部署在hadoop上:超详细步骤 整合Apache Hudi(2)

flink部署在hadoop上:超详细步骤 整合Apache Hudi(3)

进入到hive lib目录,每一台hive节点都要放置jar包

[flink@dbos-bigdata-test005 lib]$ pwd/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hive/lib//建立软链接[flink@dbos-bigdata-test005 lib]$ ln -ls ../../../jars/hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar hudi-hadoop-mr-bundle-0.10.0-SNAPSHOT.jar

flink部署在hadoop上:超详细步骤 整合Apache Hudi(4)

1.4. 安装 YARN MapReduce 框架 JAR

进入平台操作,安装YARN MapReduce框架JAR

flink部署在hadoop上:超详细步骤 整合Apache Hudi(5)

猜您喜欢: