Murat Nur

• Dec 28th, 2006

 

Hi,

if ranking means that below

prop_id  rank

1         1

1         2

1        3

2       1

2        1

you can do this first,use sort stage and value of creates the column KeyChange must be set true,it makes data like below

prop_id  rank   KeyChange()

1         1        1

1         2        0

1        3         0

2       1          1

2        1          0

if value change,keychange column set 1 else set 0,after sort stage, use transformer stage variable .

Itishree

• Dec 2nd, 2013

 

You can use either of the below options:
Seq File->Sort->Remove Duplicate ->O/P
in Remove duplicate stage, choose the key column and duplicate to retain last properties
Seq File->Remove Duplicate ->O/P
in Remove duplicate stage,choose the key column and duplicate to retain last properties, and partition the input data by Hash and there sort the data.