首页 \ 问答 \ 在Hadoop中实现自定义Writable？(Implementation of custom Writable in Hadoop?)

在Hadoop中实现自定义Writable？(Implementation of custom Writable in Hadoop?)

 我在Hadoop中定义了一个自定义的Writable类，但Hadoop在运行我的程序时给出了以下错误消息。 
 

java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.io.SortedMapWritable.readFields(SortedMapWritable.java:180)
at EquivalenceClsAggValue.readFields(EquivalenceClsAggValue.java:82)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1282)
at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1222)
at org.apache.hadoop.mapred.Task$CombineValuesIterator.next(Task.java:1301)
at Mondrian$Combine.reduce(Mondrian.java:119)
at Mondrian$Combine.reduce(Mondrian.java:1)
at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1442)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1436)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

 

 引起：java.util.concurrent.ConcurrentHashMap.hash的java.lang.NullPointerException（ConcurrentHashMap.java:332）.... 
 

 EquivalenceClsAggValue是我定义的Writable类的名称，这是我的类： 
 

public class EquivalenceClsAggValue implements WritableComparable<EquivalenceClsAggValue>{

public ArrayList<SortedMapWritable> aggValues;  
public EquivalenceClsAggValue(){        

    aggValues = new ArrayList<SortedMapWritable>();
}
@Override
public void readFields(DataInput arg0) throws IOException {

    int size = arg0.readInt();

    for (int i=0;i<size;i++){
        SortedMapWritable tmp = new SortedMapWritable();
        tmp.readFields(arg0);
        aggValues.add(tmp);
    }       
}

@Override
public void write(DataOutput arg0) throws IOException {

    //write the size first
    arg0.write(aggValues.size());

    //write each element
    for (SortedMapWritable s:aggValues){
        s.write(arg0);
    }

}

 

 我想知道问题的根源是什么。 

I have defined a custom Writable class in Hadoop, but Hadoop gives me the following error message when running my program.
 

java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.io.SortedMapWritable.readFields(SortedMapWritable.java:180)
at EquivalenceClsAggValue.readFields(EquivalenceClsAggValue.java:82)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapred.Task$ValuesIterator.readNextValue(Task.java:1282)
at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:1222)
at org.apache.hadoop.mapred.Task$CombineValuesIterator.next(Task.java:1301)
at Mondrian$Combine.reduce(Mondrian.java:119)
at Mondrian$Combine.reduce(Mondrian.java:1)
at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1442)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1436)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

 

Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:332)....
 

EquivalenceClsAggValue is the name of the Writable class I've defined and this is my class:
 

public class EquivalenceClsAggValue implements WritableComparable<EquivalenceClsAggValue>{

public ArrayList<SortedMapWritable> aggValues;  
public EquivalenceClsAggValue(){        

    aggValues = new ArrayList<SortedMapWritable>();
}
@Override
public void readFields(DataInput arg0) throws IOException {

    int size = arg0.readInt();

    for (int i=0;i<size;i++){
        SortedMapWritable tmp = new SortedMapWritable();
        tmp.readFields(arg0);
        aggValues.add(tmp);
    }       
}

@Override
public void write(DataOutput arg0) throws IOException {

    //write the size first
    arg0.write(aggValues.size());

    //write each element
    for (SortedMapWritable s:aggValues){
        s.write(arg0);
    }

}

 

I wonder to know what is the source of the problem.

原文：https://stackoverflow.com/questions/14249128
更新时间：2021-12-02 09:12

最满意答案

 我对.fancybox()并不是很了解，但是你可以试试这个想法。 
 

 将您不希望在页面上显示的图像放在隐藏的容器中。 它们不会在页面上显示，但我想他们会在fancybox中。 
 

 HTML 
 

<body>
    <div id='hiddenContainer' style='display:none;'></div>

    <a rel="example_group" href="./example/7_b.jpg"><img alt="" src="./example/7_s.jpg" /></a>
    <a rel="example_group" href="./example/9_b.jpg"><img alt="" src="./example/9_s.jpg" /></a>
</body>

 

 在jQuery中，将图像附加到该容器。 
 

$(document).ready(function() {
    $("<a rel='example_group' href='./example/8_b.jpg'><img src='./example/8_s.jpg' /></a>")
          .appendTo("#hiddenContainer");
    $("a[rel=example_group]").fancybox();
});


I don't really know much about .fancybox(), but here's an idea you might try.
 

Place the images that you don't want displayed on the page in a container that is hidden. They won't be visible on the page, but I would imagine they would be in the fancybox.
 

HTML
 

<body>
    <div id='hiddenContainer' style='display:none;'></div>

    <a rel="example_group" href="./example/7_b.jpg"><img alt="" src="./example/7_s.jpg" /></a>
    <a rel="example_group" href="./example/9_b.jpg"><img alt="" src="./example/9_s.jpg" /></a>
</body>

 

In jQuery, append the image to that container.
 

$(document).ready(function() {
    $("<a rel='example_group' href='./example/8_b.jpg'><img src='./example/8_s.jpg' /></a>")
          .appendTo("#hiddenContainer");
    $("a[rel=example_group]").fancybox();
});

相关问答

更多

相关文章

更多
  • 自定义Hadoop Writable
  • Hadoop中的Writable分析
  • Hadoop Core 学习笔记(一) SequenceFile文件写入和读取Writable数据
  • 自定义实现Hadoop Key-Value
  • Hadoop0.20+ custom MultipleOutputFormat
  • 自定义Hadoop的可序列化类
  • [How to] Make custom search with Nutch(v 1.0)?(转)
  • Hadoop自定义SdfTextInputFormat用在streaming中
  • Hadoop streaming中指定自定义的inputformat java类
  • Custom SOLR Search Components - 2 Dev Tricks

    • 最新问答

    更多
  • 获取MVC 4使用的DisplayMode后缀(Get the DisplayMode Suffix being used by MVC 4)
  • 如何通过引用返回对象？(How is returning an object by reference possible?)
  • 矩阵如何存储在内存中？(How are matrices stored in memory?)
  • 每个请求的Java新会话？(Java New Session For Each Request?)
  • css：浮动div中重叠的标题h1(css: overlapping headlines h1 in floated divs)
  • 无论图像如何，Caffe预测同一类(Caffe predicts same class regardless of image)
  • xcode语法颜色编码解释？(xcode syntax color coding explained?)
  • 在Access 2010 Runtime中使用Office 2000校对工具(Use Office 2000 proofing tools in Access 2010 Runtime)
  • 从单独的Web主机将图像传输到服务器上(Getting images onto server from separate web host)
  • 从旧版本复制文件并保留它们（旧/新版本）(Copy a file from old revision and keep both of them (old / new revision))
  • 西安哪有PLC可控制编程的培训
  • 在Entity Framework中选择基类(Select base class in Entity Framework)
  • 在Android中出现错误“数据集和渲染器应该不为null，并且应该具有相同数量的系列”(Error “Dataset and renderer should be not null and should have the same number of series” in Android)
  • 电脑二级VF有什么用
  • Datamapper Ruby如何添加Hook方法(Datamapper Ruby How to add Hook Method)
  • 金华英语角.
  • 手机软件如何制作
  • 用于Android webview中图像保存的上下文菜单(Context Menu for Image Saving in an Android webview)
  • 注意：未定义的偏移量：PHP(Notice: Undefined offset: PHP)
  • 如何读R中的大数据集[复制](How to read large dataset in R [duplicate])
  • Unity 5 Heighmap与地形宽度/地形长度的分辨率关系？(Unity 5 Heighmap Resolution relationship to terrain width / terrain length?)
  • 如何通知PipedOutputStream线程写入最后一个字节的PipedInputStream线程？(How to notify PipedInputStream thread that PipedOutputStream thread has written last byte?)
  • python的访问器方法有哪些
  • DeviceNetworkInformation：哪个是哪个？(DeviceNetworkInformation: Which is which?)
  • 在Ruby中对组合进行排序(Sorting a combination in Ruby)
  • 网站开发的流程？
  • 使用Zend Framework 2中的JOIN sql检索数据(Retrieve data using JOIN sql in Zend Framework 2)
  • 条带格式类型格式模式编号无法正常工作(Stripes format type format pattern number not working properly)
  • 透明度错误IE11(Transparency bug IE11)
  • linux的基本操作命令。。。

    • Copyright ©2023 peixunduo.com All Rights Reserved.粤ICP备14003112号

    本站部分内容来源于互联网，仅供学习和参考使用，请莫用于商业用途。如有侵犯你的版权，请联系我们(neng862121861#163.com)，本站将尽快处理。谢谢合作！