Scenario:
We need to load data from Flat file source but we want to redirect all duplicate records to Audit table or File with count and load only records which are unique to our destination table.
Solution :
We will be using Aggregate Transformation with Conditional Split Transformation to perform this requirement.
Step 1:
Drag Data Flow Task to Control Flow Pane and then Drag Flat File Source inside Data Flow Pane. Create Connection to source file as shown below
Click on Columns and see if data is displaying correctly and then click Ok.
Step 2:
Bring Aggregate Transformation to Data Flow Pane and connect Flat File Source to Aggregate Transformation and configure as shown. We need to group them and get count.
Step 3:
Drag Conditional Split Transformation to Data Flow Pane and connect Aggregate Transformation to it. By using Conditional Split we will be redirecting duplicate records to output where we can write them to table/file. I have converted [Count All] to Int by using Cast Function (DT_I4).
Conditional Split Transformation is going to create two outputs, DuplicateRecords for duplicate records and CorrectRecords for unique records.
Step 4:
Bring two Multicast Transformations and connect each of output from Conditional Split Transformation to them. Put Data Viewer between them so we can display the records. In real time you will be writing these records to your destination tables/files.
Final Output:
By using the Conditional Split, the duplicate records are redirected to Multicast 1 transformation and unique records are directed to Multicast transformation.
Video Tutorial for the post
I think this is actually a very informative blog post about SSIS and SQL.There is always a need to find out more and more aspects.
ReplyDeleteSSIS Postgresql Write
Đặt vé máy bay tại Aivivu, tham khảo
ReplyDeletekinh nghiệm mua vé máy bay đi Mỹ giá rẻ
vé máy bay từ mỹ về việt nam bao nhiêu
cách đặt vé máy bay từ nhật về việt nam
bao giờ có chuyến bay từ đức về việt nam
thông tin chuyến bay từ canada về việt nam
mua ve may bay tu han quoc ve viet nam
business analyst training
ReplyDeleteoracle adf training
oracle rac training
msbi training
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download Now
ReplyDelete>>>>> Download Full
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download LINK
>>>>> Download Now
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download Full
>>>>> Download LINK PR
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download Now
ReplyDelete>>>>> Download Full
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download LINK
>>>>> Download Now
Welcome To Techbrothersit: Ssis- How To Redirect Duplicate Records In Ssis Or Remove Duplicate Records In Data Flow Task >>>>> Download Full
>>>>> Download LINK Xn