二级排序 - 将其留给Hadoop框架或在reducer中自行完成(Secondary sort - leave it to Hadoop framework or do it yourself in reducer)

 我读到了关于二级排序的问题，其中需要对每个键进行排序，不仅是键，还要是值的一部分。 
 
 有两种方法可以做到这一点： 
 
 
 
 在每个键的reducer中缓存值并自己对值进行排序 
 
 
 将作业留给Hadoop框架，通过指定自定义Comparator，Partitioner ...您需要的所有内容不仅要按键排序，还要按值排序 
 
 
 我的问题是，你什么时候会推荐第一种方法？ 
 
 正如我目前所看到的那样 - 如果框架已经执行了排序，为什么不同时按键和值对其进行排序...如果有一些副作用，请纠正我。 例如，哪个应该更快？ 
 
 我知道“in-Reducer排序”的最大问题是记录的数量，但我想全面了解。 

I read about secondary sorting, where there is a need to sort not only be key, but also by part of value, for each of the keys.
 
There are two ways to do this:
 
 
 
Cache values in reducer for each key and sort the values yourself
 
 
Leave the job to Hadoop framework, by specifying custom Comparator, Partitioner...all that you need to enable not only to sort by key, but also by value
 
 
My question is, when would you recommend first and when the second approach?
 
As I currently see it - if the framework already performs sorting, why not sort it by key and value at the same time...please correct me if there is some side-effect. For example, which should be faster?
 
I understand that the biggest problem of "in-Reducer sorting" is the number of records, but I would like to get the whole picture.

原文：https://stackoverflow.com/questions/30050848