文件md5散列更改时（对于netty传输）(File md5 hash changes when chunking it (for netty transfer))

 
 
 底部的问题 
 
 
 我使用netty将文件传输到另一台服务器。 由于WebSocket协议，我将文件块限制为1024 * 64字节（64KB）。 以下方法是本地示例文件会发生什么情况： 
 
public static void rechunck(File file1, File file2) {

    FileInputStream is = null;
    FileOutputStream os = null;

    try {

        byte[] buf = new byte[1024*64];

        is = new FileInputStream(file1);
        os = new FileOutputStream(file2);

        while(is.read(buf) > 0) {
            os.write(buf);
        }

    } catch (IOException e) {
        Controller.handleException(Thread.currentThread(), e);
    } finally {

        try {

            if(is != null && os != null) {
                is.close();
                os.close();
            }

        } catch (IOException e) {
            Controller.handleException(Thread.currentThread(), e);
        }

    }

}
 
 该文件由InputStream加载到ByteBuffer中，并直接写入OutputStream 。 该过程中文件的内容不能改变。 
 
 为了获得文件的md5-hashes ，我编写了以下方法： 
 
public static String checksum(File file) {

    InputStream is = null;

    try {

        is = new FileInputStream(file);
        MessageDigest digest = MessageDigest.getInstance("MD5");
        byte[] buffer = new byte[8192];
        int read = 0;

        while((read = is.read(buffer)) > 0) {
            digest.update(buffer, 0, read);
        }

        return new BigInteger(1, digest.digest()).toString(16);

    } catch(IOException | NoSuchAlgorithmException e) {
        Controller.handleException(Thread.currentThread(), e);
    } finally {

        try {
            is.close();
        } catch(IOException e) {
            Controller.handleException(Thread.currentThread(), e);
        }

    }

    return null;

}
 
 所以：理论上它应该返回相同的散列，不是吗？ 问题是它返回两个不同的散列，每次运行都没有区别。文件大小保持不变，内容也是一样。 当我in: file-1 ， out: file-2以及in: file-2和out: file-3运行一次方法时， in: file-2和out: file-3的哈希值是相同的！ 这意味着该方法将每次以相同的方式正确更改文件。 
 
1. 58a4a9fbe349a9e0af172f9cf3e6050a
2. 7b3f343fa1b8c4e1160add4c48322373
3. 7b3f343fa1b8c4e1160add4c48322373
 
 这是一个比较所有缓冲区的小测试，如果它们是相同的。 测试是积极的。 所以没有任何分歧。 
 
File file1 = new File("controller/templates/Example.zip");
File file2 = new File("controller/templates2/Example.zip");

try {

    byte[] buf1 = new byte[1024*64];
    byte[] buf2 = new byte[1024*64];

    FileInputStream is1 = new FileInputStream(file1);
    FileInputStream is2 = new FileInputStream(file2);

    boolean run = true;
    while(run) {

        int read1 = is1.read(buf1), read2 = is2.read(buf2);
        String result1 = Arrays.toString(buf1), result2 = Arrays.toString(buf2);
        boolean test = result1.equals(result2);

        System.out.println("1: " + result1);
        System.out.println("2: " + result2);
        System.out.println("--- TEST RESULT: " + test + " ----------------------------------------------------");

        if(!(read1 > 0 && read2 > 0) || !test) run = false;

    }

} catch (IOException e) {
    e.printStackTrace();
}
 
 问题： 你能帮我在不更改散列的情况下分块文件吗？ 

 
 
Question at the bottom
 
 
I'm using netty to transfer a file to another server. I limit my file-chunks to 1024*64 bytes (64KB) because of the WebSocket protocol. The following method is a local example what will happen to the file:
 
public static void rechunck(File file1, File file2) {

    FileInputStream is = null;
    FileOutputStream os = null;

    try {

        byte[] buf = new byte[1024*64];

        is = new FileInputStream(file1);
        os = new FileOutputStream(file2);

        while(is.read(buf) > 0) {
            os.write(buf);
        }

    } catch (IOException e) {
        Controller.handleException(Thread.currentThread(), e);
    } finally {

        try {

            if(is != null && os != null) {
                is.close();
                os.close();
            }

        } catch (IOException e) {
            Controller.handleException(Thread.currentThread(), e);
        }

    }

}
 
The file is loaded by the InputStream into a ByteBuffer and directly written to the OutputStream. The content of the file cannot change while this process.
 
To get the md5-hashes of the file I've wrote the following method:
 
public static String checksum(File file) {

    InputStream is = null;

    try {

        is = new FileInputStream(file);
        MessageDigest digest = MessageDigest.getInstance("MD5");
        byte[] buffer = new byte[8192];
        int read = 0;

        while((read = is.read(buffer)) > 0) {
            digest.update(buffer, 0, read);
        }

        return new BigInteger(1, digest.digest()).toString(16);

    } catch(IOException | NoSuchAlgorithmException e) {
        Controller.handleException(Thread.currentThread(), e);
    } finally {

        try {
            is.close();
        } catch(IOException e) {
            Controller.handleException(Thread.currentThread(), e);
        }

    }

    return null;

}
 
So: just in theory it should return the same hash, shouldn't it? The problem is that it returns two different hashes that do not differ with every run.. file size stays the same and the content either. When I run the method once for in: file-1, out: file-2 and again with in: file-2 and out: file-3 the hashes of file-2 and file-3 are the same! This means the method will properly change the file every time the same way. 
 
1. 58a4a9fbe349a9e0af172f9cf3e6050a
2. 7b3f343fa1b8c4e1160add4c48322373
3. 7b3f343fa1b8c4e1160add4c48322373
 
Here is a little test that compares all buffers if they are equivalent. Test is positive. So there aren't any differences.
 
File file1 = new File("controller/templates/Example.zip");
File file2 = new File("controller/templates2/Example.zip");

try {

    byte[] buf1 = new byte[1024*64];
    byte[] buf2 = new byte[1024*64];

    FileInputStream is1 = new FileInputStream(file1);
    FileInputStream is2 = new FileInputStream(file2);

    boolean run = true;
    while(run) {

        int read1 = is1.read(buf1), read2 = is2.read(buf2);
        String result1 = Arrays.toString(buf1), result2 = Arrays.toString(buf2);
        boolean test = result1.equals(result2);

        System.out.println("1: " + result1);
        System.out.println("2: " + result2);
        System.out.println("--- TEST RESULT: " + test + " ----------------------------------------------------");

        if(!(read1 > 0 && read2 > 0) || !test) run = false;

    }

} catch (IOException e) {
    e.printStackTrace();
}
 
Question: Can you help me chunking the file without changing the hash?

原文：https://stackoverflow.com/questions/49588868