Monday, March 19, 2012

How to get started on custom Merge Join component?

I need to extend the functionality of the Merge Join transformation to allow more complex joining scenarios. I have found some quickstarts on custom components (i.e. Hash Transform, etc.), however I haven't found any good references on more advanced tasks like processesing multiple inputs. Any ideas?

TIA,

Matthew

First as a disclaimer I would advise against this, as writing a multi imput component is significantly more complex that writing a single input component due to the threading that needs to be done in just about all scenarios.

With that said you can look at any of the samples that have async outputs as a good starting point. The additional complexity comes from the fact that you will, most likely, need to implement your own threading in order to do the merging. This means you will need to make a copy of the buffers to pass to your thread that does the merging. You need this threading because depending on the package both of your inputs could be called on the same thread and if you just waited on one input for the other input to get data then you would hang. All the stock multi input components do their own threading except for UnionAll because that one doesn't have any inter input dependencies to write the data to it output.

HTH,
Matt

|||Matt,

Thanks for the reply. I haven't seen any samples showing async outputs. Could you post a link?

Thanks!

Matthew|||BOL has a RemoveDuplicates sample which has an async output. You can look for sample applications[Integration Services] in the index.

Matt

No comments:

Post a Comment