How do you remove duplicates in dataset

In how many ways we can delete dataset? If a record is duplicated 3 times then how to get middle duplicated record? Is it advisable to use basic Tfr in Parallel jobs?

pradeep.dwh
Profile Answers by pradeep.dwh Questions by pradeep.dwh
Jan 26th, 2008
4
8411

Questions by pradeep.dwh answers by pradeep.dwh

DataStage

Answer

Showing Answers 1 - 4 of 4 Answers

manoharkolukula
Profile Answers by manoharkolukula Questions by manoharkolukula

Feb 12th, 2008

i think we have to use remove duplicate stage here to remove duplicates.

First we have to take dataset, after that rdstage and target,

or

source as dataset and trasformer(in this give constraint) and target.

like this we can eliminate duplicates.

manoharkolukula
Profile Answers by manoharkolukula Questions by manoharkolukula

Feb 12th, 2008

w/o basic t/r also we can do this know, then y to use basic t/r here.

srkreddy111

Aug 12th, 2011

first you have to open the data set and click on the partitioning and after click hash partition and next click perform sort after click on unique and after ok and after compile and run the job.open the target output,the duplicate records are removed..

chhavis928
Profile Answers by chhavis928 Questions by chhavis928

Oct 8th, 2011

We can use filter stage here to get middle record from 3 duplicate records.