无法从hadoop hdfs检索文件(Can't retrieve files from hadoop hdfs)

 我正在学习如何从/向hdfs读/写文件。 
 
 这是我用来阅读的代码： 
 
import java.io.InputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;

public class FileSystemCat {
public static void main (String [] args) throws Exception {

    String uri = "/user/hadoop/file.txt";
    Configuration conf = new Configuration();
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));

    FileSystem fs = FileSystem.get(URI.create(uri),conf);

    InputStream in = null;
    try{

        in = fs.open(new Path(uri));
        IOUtils.copyBytes(in, System.out, 4096,false);
    }finally{
        IOUtils.closeStream(in);
    }           
}
 
 } 
 
 该文件在那里 
 
 
 但是，当我在eclipse中运行代码时，我得到以下内容 
 
Exception in thread "main" java.io.FileNotFoundException: File /user/hadoop/file.txt does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at hadoop.FileSystemCat.main(FileSystemCat.java:22)
 
 我用作路径文件：///user/hadoop/file.txt和hdfs：///user/hadoop/file.txt 
 
 对于后者，错误略有不同： 
 
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
 
 核心的site.xml 
 
<configuration>
   <property>
     <name>fs.default.name</name>
     <value>hdfs://localhost/</value>
   </property>
</configuration>
 
 HDFS-site.xml中 
 
<configuration>
<property>
   <name>dfs.replication</name>
   <value>2</value>
 </property>

 <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:///usr/local/hadoop_store/hdfs/namenode/</value>
 </property>

 <property>
   <name>dfs.datanode.data.dir</name>
   <value>file:///usr/local/hadoop_store/hdfs/datanode/,file:///mnt/hadoop/hadoop_store/hdfs/datanode/</value>
 </property>

 <property>
   <name>dfs.webhdfs.enabled</name>
   <value>true</value>
 </property>
</configuration>
 
 有顾虑吗？ 
 
 谢谢 

I am learning how to read/write files from/to hdfs. 
 
This is the code I use for reading:
 
import java.io.InputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;

public class FileSystemCat {
public static void main (String [] args) throws Exception {

    String uri = "/user/hadoop/file.txt";
    Configuration conf = new Configuration();
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));

    FileSystem fs = FileSystem.get(URI.create(uri),conf);

    InputStream in = null;
    try{

        in = fs.open(new Path(uri));
        IOUtils.copyBytes(in, System.out, 4096,false);
    }finally{
        IOUtils.closeStream(in);
    }           
}
 
}
 
The file is there
 
 
However, I get the following when I run my code in eclipse
 
Exception in thread "main" java.io.FileNotFoundException: File /user/hadoop/file.txt does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at hadoop.FileSystemCat.main(FileSystemCat.java:22)
 
I used as path both file:///user/hadoop/file.txt and hdfs:///user/hadoop/file.txt
 
For the latter the error is slightly different:
 
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
 
core-site.xml
 
<configuration>
   <property>
     <name>fs.default.name</name>
     <value>hdfs://localhost/</value>
   </property>
</configuration>
 
hdfs-site.xml
 
<configuration>
<property>
   <name>dfs.replication</name>
   <value>2</value>
 </property>

 <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:///usr/local/hadoop_store/hdfs/namenode/</value>
 </property>

 <property>
   <name>dfs.datanode.data.dir</name>
   <value>file:///usr/local/hadoop_store/hdfs/datanode/,file:///mnt/hadoop/hadoop_store/hdfs/datanode/</value>
 </property>

 <property>
   <name>dfs.webhdfs.enabled</name>
   <value>true</value>
 </property>
</configuration>
 
Any concern?
 
Thanks

原文：https://stackoverflow.com/questions/24488269