Merge Rows (diff)

Let's say we're doing a delta load of new data at specific times ..

Based on keys for comparison, we can use this step to merge reference rows (previous data) with compare rows (new data) to create merged output rows.

A flag in the row indicates how the values were compared and merged. Flag values include:

• identical

The key was found in both rows, and the compared values are identical.

• changed

The key was found in both rows, but one or more compared values are different.

• new

The key was not found in the reference rows.

• deleted

The key was not found in the compare rows.

If the rows are flagged as deleted, the merged output rows are created based upon the original reference rows stream.

For identical, new, or changed rows, the merged output rows are created based upon the original compare rows stream.