How do you load unique records into one target table and duplicate records into a different target table?
Source Table:
COL1 | COL2 | COL3 |
a | b | c |
x | y | z |
a | b | c |
r | f | u |
a | b | c |
v | f | r |
v | f | r |
Target Table 1: Table containing all the unique rows
COL1 | COL2 | COL3 |
a | b | c |
x | y | z |
r | f | u |
v | f | r |
Target Table 2: Table containing all the duplicate rows
COL1 | COL2 | COL3 |
a | b | c |
a | b | c |
v | f | r |
- Drag the source to mapping and connect it to an aggregator transformation.
- In aggregator transformation, group by the key column and add a new port. Call it count_rec to count the key column.
- Connect a router to the aggregator from the previous step. In router make two groups: one named “original” and another as “duplicate”.
In original write count_rec=1 and in duplicate write count_rec>1.
The picture below depicts the group name and the filter conditions.
Connect two groups to corresponding target tables
0 Comments