`

MapReduce牛逼(4)WritableComparable接口

 
阅读更多
@Public
@Stable

A Writable which is also Comparable.

WritableComparables can be compared to each other, typically via Comparators. Any type which is to be used as a key in the Hadoop Map-Reduce framework should implement this interface.

Note that hashCode() is frequently used in Hadoop to partition keys. It's important that your implementation of hashCode() returns the same result across different instances of the JVM. Note also that the default hashCode() implementation in Object does not satisfy this property.

Example:


     public class MyWritableComparable implements WritableComparable {
       // Some data
       private int counter;
       private long timestamp;
      
       public void write(DataOutput out) throws IOException {
         out.writeInt(counter);
         out.writeLong(timestamp);
       }
      
       public void readFields(DataInput in) throws IOException {
         counter = in.readInt();
         timestamp = in.readLong();
       }
      
       public int compareTo(MyWritableComparable o) {
         int thisValue = this.value;
         int thatValue = o.value;
         return (thisValue < thatValue ? -1 : (thisValue==thatValue ? 0 : 1));
       }

       public int hashCode() {
         final int prime = 31;
         int result = 1;
         result = prime * result + counter;
         result = prime * result + (int) (timestamp ^ (timestamp >>> 32));
         return result
       }
     }




@InterfaceAudience.Public
@InterfaceStability.Stable
public interface WritableComparable<T> extends Writable, Comparable<T> {
}

--------------
org.apache.hadoop.io.Writable


@Public
@Stable

A serializable object which implements a simple, efficient, serialization protocol, based on DataInput and DataOutput.

Any key or value type in the Hadoop Map-Reduce framework implements this interface.

Implementations typically implement a static read(DataInput) method which constructs a new instance, calls readFields(DataInput) and returns the instance.

Example:


     public class MyWritable implements Writable {
       // Some data    
       private int counter;
       private long timestamp;
      
       public void write(DataOutput out) throws IOException {
         out.writeInt(counter);
         out.writeLong(timestamp);
       }
      
       public void readFields(DataInput in) throws IOException {
         counter = in.readInt();
         timestamp = in.readLong();
       }
      
       public static MyWritable read(DataInput in) throws IOException {
         MyWritable w = new MyWritable();
         w.readFields(in);
         return w;
       }
     }
@InterfaceAudience.Public
@InterfaceStability.Stable
public interface Writable {
  /** 
   * Serialize the fields of this object to <code>out</code>.
   * 
   * @param out <code>DataOuput</code> to serialize this object into.
   * @throws IOException
   */
  void write(DataOutput out) throws IOException;

  /** 
   * Deserialize the fields of this object from <code>in</code>.  
   * 
   * <p>For efficiency, implementations should attempt to re-use storage in the 
   * existing object where possible.</p>
   * 
   * @param in <code>DataInput</code> to deseriablize this object from.
   * @throws IOException
   */
  void readFields(DataInput in) throws IOException;
}



分享到:
评论

相关推荐

    mapreduce mapreduce mapreduce

    mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce mapreduce ...

    实验项目 MapReduce 编程

    4 分别在自编 MapReduce 程序 WordCount 运行过程中和运行结束后查看 MapReduce Web 界面。 5. 分别在自编 MapReduce 程序 WordCount 运行过程中和运行结束后练习 MapReduce Shell 常用命令。 。。

    大数据分析技术基础PPT课件(共9单元)4-MapReduce 编程.pdf

    大数据分析技术基础PPT课件(共9单元)4-MapReduce 编程.pdf大数据分析技术基础PPT课件(共9单元)4-MapReduce 编程.pdf大数据分析技术基础PPT课件(共9单元)4-MapReduce 编程.pdf大数据分析技术基础PPT课件(共9单元)4-...

    MapReduce综合案例(4个)

    1.社交网络综合评分案例 2.微博精准营销案例 3.物品推荐案例 4.QQ好友推荐案例

    Hadoop原理与技术MapReduce实验

    (4)完成上课老师演示的内容 二、实验环境 Windows 10 VMware Workstation Pro虚拟机 Hadoop环境 Jdk1.8 二、实验内容 1.单词计数实验(wordcount) (1)输入start-all.sh启动hadoop相应进程和相关的端口号 (2)...

    MapReduce 编程模型

    MapReduce goole MapReduce编程模型

    MapReduce发明人关于MapReduce的介绍

    MapReduce发明人关于MapReduce的介绍

    【MapReduce篇07】MapReduce之数据清洗ETL1

    【MapReduce篇07】MapReduce之数据清洗ETL1

    MapReduce实现join连接

    简单的在MapReduce中实现两个表的join连接简单的在MapReduce中实现两个表的join连接简单的在MapReduce中实现两个表的join连接

    Mapreduce-实践

    (实践三)MapReduce 布隆过滤器 过滤器训练、过滤器应用、结果验证及分析 (实践四)MapReduce Top 10模式示例 在ctrip数据集上进行Top 10排序。 (实践五)去重的用户—针对ctrip数据集去重 对ctrip数据集中的...

    Hadoop mapreduce实现wordcount

    Hadoop 用mapreduce实现Wordcount实例,绝对能用

    图解MapReduce.doc

    图解MapReduce,系统介绍Hadoop MapReduce工作过程原理

    mapreduce源码

    hadoop网站通过SVN下载下来的mapreduce代码。欢迎现在学习!

    MapReduce求行平均值--MapReduce案例

    MapReduce求取行平均值 MapReduce小实例 数据有经过处理已经添加行号的 也有未添加的 行平均值的四种求法

    Mapreduce实验报告.doc

    Mapreduce实验报告 前言和简介 MapReduce是Google提出的一种编程模型,在这个模型的支持下可以实现大规模并行化计 算。在Mapreduce框架下一个计算机群通过统一的任务调度将一个巨型任务分成许多部分 ,分别解决然后...

    mapreduce练习资源

    该资源中中包含MapReduce的练习题,可以是你同过这些练习题更好的掌握MapReduce

    MapReduce 设计模式

    MapReduce 设计模式,深入理解MapReduce编程模式,更好的利用MapReduce模型

    mapreduce项目 数据清洗

    mapreduce基本数据读取,通俗易懂。 此项目情景为,遗传关系族谱。 (爷爷 、父母 、孩子) 经典案例

    MapReduce的实现细节

    MapReduce的实现细节,对mapreduce的具体实现讲解

    MapReduce简单程序示例

    MapReduce简单程序示例

Global site tag (gtag.js) - Google Analytics