Flink错误 - org.apache.hadoop.ipc.RemoteException:服务器IPC版本9无法与客户端版本4通信(Flink error - org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4)
我正在尝试使用HDFS中的文件运行flink作业。 我创建了一个数据集如下 -
DataSource<Tuple2<LongWritable, Text>> visits = env.readHadoopFile(new TextInputFormat(), LongWritable.class,Text.class, Config.pathToVisits());
我正在使用flink的最新版本 - 0.9.0-milestone-1-hadoop1(我也试过0.9.0-milestone-1)
而我的Hadoop版本是2.6.0
但是,当我尝试执行作业时,我得到以下异常。 我搜索过类似的问题,它与客户端和hdfs之间的版本不兼容有关。
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4 at org.apache.hadoop.ipc.Client.call(Client.java:1113) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
你能不能让我知道我应该在我的pom中做出哪些改变,以便它指出正确的Hadoop / HDFS版本? 在其他地方改变? 或者我需要降级hadoop安装?
I am trying to run a flink job using a file from HDFS. I have created a dataset as following -
DataSource<Tuple2<LongWritable, Text>> visits = env.readHadoopFile(new TextInputFormat(), LongWritable.class,Text.class, Config.pathToVisits());
I am using flink's latest version - 0.9.0-milestone-1-hadoop1 (I have also tried with 0.9.0-milestone-1)
whereas my Hadoop version is 2.6.0
But, I get the following exception when I try to execute the job. I have searched for similar problem, and it is related to version incompatibility between client and hdfs.
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4 at org.apache.hadoop.ipc.Client.call(Client.java:1113) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Can you please let me know what changes should I make in my pom, so that it points to correct Hadoop/HDFS version? or changes elsewhere? Or I need to downgrade the hadoop installation?
原文:https://stackoverflow.com/questions/30847667