SSIS - Project Versioning (Project Deployment in SSIS)

Project Versioning in Denali:

Was exploring Denali CTP1, now we can deploy Project to SSISDB Catalog.
Each time we will deploy the project to SSISDB, the project will be saved as new version.
We can see which version we are using currently by checking the properties of project and we can restore any previous version that we want.

How to check version number or project deployed to SSISDB

How to check and restore to previous versions of SSIS Project in SSISDB

**If we don't have Team Foundation Server(TFS) or VSS then this feature is good for version controlling.
**Could not find how to label each of the version or insert some details about project or comments during deployment.


link :

http://social.technet.microsoft.com/wiki/contents/articles/projects-in-sql-server-denali-ctp1-ssis.aspx

SSIS - How to Write Expressions on Data Flow Task for Derived Column Transformation Expressions for derived column

Scenario:

We are developing an SSIS Package and writing business logic by using Derived Column Transformation in Data Flow Transformation.
This business logic is subjected to change. We don't want to open the package every time the business logic changes, Instead of that we want to use configuration to handle the situation. If the logic changes, we will update the value in configuration table or file and package should read the value and apply in Derived Column Transformation.


Solution:


Let’s say we are loading data from flat file to SQL Server table, and flat file has two columns ID and Name.  We need to derive initials from Name column and business logic say that always take the first character from Name column as Initials.


Step1:
You went ahead and used the Flat file source to read the data, then used the Derived Column Transformation and wrote expressions to get the first Characters as shown below.
How to write expression on Data Flow Task to Change business Logic in Derived Column Transformation in SSIS Package


Step2:
Now consider that business decieded to change the logic for Intials. They want you to take two characters from Name column as Initials instead of first single character. That can be easy, you will open the SSIS Package and then make change in Derived Column expressions for DER_Name (Initials Column), save the package, test in QA,UAT and finally deploy to production.

This is long process. You want to take different approach , where you don't have to make any changes to SSIS Package, rather handle the logic change by using variable in SSIS Package and change the value of the variable in Configuration table or file.

To implement this, let's create a variable called VarBusinessLogic as shown below
Create Variable in SSIS Package and use in Data Flow Task to Change Expressions in Derived Column Transformation


Step 3:
As our variable is ready now and it has the value that we would like to use in Derived Column Expressions, Go to Properties of Data Flow Task, click on expressions and use VarBusinessLogic variable as shown below.

How to change expressions in Derived Column Transformation by using Variable value in SSIS Package


By using friendly expression on Data flow task we were able to write expression(Business logic) in derived column.

Step 4:
The final step is to add the VarBusinessLogic variable in Package Configuration. Now if the logic changes, you don't have to make any change to SSIS Package, instead of that you will update the value for VarBusinessLogic variable in Configuration table or file and Package will read and implement in Derived Column Expressions.

This can be helpful when business logic often change and you don't want to make change to package every time for small change.


SSIS - How to Remove Leading,Trailing OR Both Zeros from String in SSIS Package

Scenario:

Sometimes we get the data from flat file and we want to remove Leading Zero, Trailing Zeros OR Leading and trailing both before insert into destination.


Solution:

I assumed that data we are getting is string type, coming from the source (Excel, Flat file etc.)
Use the Derived column Transformation to remove Leading/Trailing OR Both zero from the string. After removing Zeros you can
Cast to Any data type you want such as Numeric, Integer, float etc.


Here are expressions to remove Leading Zeros, Trailing Zeros or Both from string column.


Leading Zeros:
(DT_WSTR,50)(DT_I8)[YourInputColumn]


Trailing Zeros:
REVERSE((DT_WSTR,50)(DT_I8)REVERSE([YourInputColumn]))


Leading and Trailing Zeros:
REVERSE((DT_WSTR,50)(DT_I8)REVERSE((DT_WSTR,50)(DT_I8)[YourInputColumn]))

SSIS - MonthName and Day Name in SSIS

Scenario:

Some time we have requirement to send email with MonthName or dayName in the subject in SSIS Package or need to create folder with Month Name and then Archive files e.g. Archive_Dec_2011. How we can create folder or have subject in email with Month Name or Day Name by using SSIS Package?


Solution:

We don't have Month-Name or Day-Name in SSIS Package.We can create two variables in our package and write expressions on them to use them wherever we need Month Name and Day Name.


Step 1:
Let's create VarDayName and VarMonthName variables in our SSIS Package so we can write expressions on them and use in our Package where we need Month Name an Day Name.

How to get Month Name and Day Name in SSIS Package - SSIS Tutorial for Beginners




Step 2: 
Go to the properties of VarDayName variable and then configure as shown below.


Write expression to get Day Name in SSIS Package - SSIS Tutorial

Expressions for Month Name:
DATEPART("dw",GETDATE())==1? "Sunday":
DATEPART("dw",GETDATE())==2?"Monday":
DATEPART("dw",GETDATE())==3?"Tuesday":
DATEPART("dw",GETDATE())==4?"Wednesday":
DATEPART("dw",GETDATE())==5?"Thursday":
DATEPART("dw",GETDATE())==6?"Friday":
DATEPART("dw",GETDATE())==7?"Saturday":""


Step 3:

Go to the properties of DayMonthName variable and write expressions as shown below.



Expressions for Month Name:
(MONTH(getdate()) == 1 ? "Jan" :
MONTH(getdate()) == 2 ? "Feb" :
MONTH(getdate()) == 3 ? "Mar" :
MONTH(getdate()) == 4 ? "Apr" :
MONTH(getdate()) == 5 ? "May" :
MONTH(getdate()) == 6 ? "Jun" :
MONTH(getdate()) == 7 ? "Jul" :
MONTH(getdate()) == 8 ? "Aug" :
MONTH(getdate()) == 9 ? "Sep" :
MONTH(getdate()) == 10 ? "Oct" :
MONTH(getdate()) == 11 ? "Nov" :
MONTH(getdate()) == 12? "Dec":"")



Now you have Month Name and DayName available through these two variables ,
Those can be used in expression with different transformations as required.


Related Posts
How To Use Expression Task To Get Day Name and Month Name In SSIS Package
Video Demo : How to use Expression Task to get Month Name in SSIS Package

How to Create flat file dynamically from SQL Server Table or View in SSIS Package - SSIS Tutorial

Scenario: Download Script

We have a requirement to create an SSIS Package that should create an output file dynamically from a Table or View.
The columns of View or Table can change anytime but we don't want to open package and remap column/columns. In short our SSIS Package should not care about the number of columns returned by Table or view.

Solution:

As the definition of Table or View can change anytime, It will be good idea to use Scirpt Task and handle the situation. If we use builtin source and destination Transformation, they will fail in case name or number of columns will change in source table or view.


Let's processed with step by step approach

Step 1:
Create the following variables as shown below, the values for the variables can be configured by using SSIS configuration,
This will give us freedom to change the name of file, directory path, delimiter ("," or "" etc.) and file extension (txt or csv etc.) without making modification to SSIS Package.

VarDelimiter :  Provide the delimiter that you want for your flat file
VarDirectoryPath :  Provide the directory path where you would like to create flat file dynamically
VarFileExtension : Provide the extension of file you like to create
VarFileName : Provide the name of file that you would like to create dynamically from SQL Server Table
VarLogFolder: Provide the path for Log folder in case error occur in Script Task
VarObject :  Create Object type variable to save records from SQL Table or View.



Create variables in SSIS Package to create flat file dynamically from SQL Server Table or View


Step 2:
Create OLE DB Connection manager by going to Connection Manager pane. You have to provide SQL Server Instance Name and then choose the database. After creating, I have renamed to DB_Conn_TechBrothersIT as shown below.

Create OLE DB Connection manager in SSIS Package to generate flat file dynamically


Step 3:

After creating variable, Drag Execute SQL Task in control flow pane and configure as show below. Please notice that we are using Select * here. that means get all the columns data from table or view.



Get data from SQL Server Table or View and save to Object type variable in SSIS Package to create flat file dynamically


Map the full result set to Object type variable in Execute SQL Task in SSIS Package as shown below.


Save the query results in Object type variable by using Execute SQL Task in SSIS Package


Step 4:

Bring Script Task to Control Flow Pane and then open by double clicking. We will be using C# as scripting language in our demo, you can use visual basic if you like.
Go to ReadOnlyVariable and then add the user variables those we have created above in Step 1.



Add variables to Script Task in SSIS Package to create flat file dynamically in SSIS Package from SQL Server Table or View


Step 5:

Click on Edit Script button, paste following code and save the the changes and Finally OK button.


Under #region Namespaces, I have added below code

using System.IO;
using System.Data.OleDb;


Under public void Main() { 
I have added below code.
            string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
            try {

                //Declare Variables
           string FileDelimiter = Dts.Variables["User::VarDelimiter"].Value.ToString();
           string DirectoryPath = Dts.Variables["User::VarDirectoryPath"].Value.ToString();
           string FileName = Dts.Variables["User::VarFileName"].Value.ToString();
           string FileExtension=Dts.Variables["User::VarFileExtension"].Value.ToString();
                

                OleDbDataAdapter A = new OleDbDataAdapter();
                System.Data.DataTable dt = new System.Data.DataTable();
                //Get Data From Object Type variable
                A.Fill(dt, Dts.Variables["User::VarObject"].Value);
                
                //Construct File Full Path by using Varaible values
              string FileFullPath = DirectoryPath + FileName +"_"+datetime+ FileExtension;

          int i = 0;
          
           StreamWriter sw = null;
           sw = new StreamWriter(FileFullPath, false);
           
                // Write the Header Row to File
           int ColumnCount = dt.Columns.Count;
           for (int ic = 0; ic < ColumnCount; ic++)
           {
               sw.Write(dt.Columns[ic]);
               if (ic < ColumnCount - 1)
               {
                   sw.Write(FileDelimiter);
               }
           }
           sw.Write(sw.NewLine);

           // Write All Rows to the File
           foreach (DataRow dr in dt.Rows)
           {
               for (int ir = 0; ir < ColumnCount; ir++)
               {
                   if (!Convert.IsDBNull(dr[ir]))
                   {
                       sw.Write(dr[ir].ToString());
                   }
                   if (ir < ColumnCount - 1)
                   {
                       sw.Write(FileDelimiter);
                   }
               }
               sw.Write(sw.NewLine);
           }
           sw.Close();
                Dts.TaskResult = (int)ScriptResults.Success;
        }
             catch (Exception exception)
            {
                // Create Log File for Errors
                using (StreamWriter sw = 
File.CreateText(Dts.Variables["User::VarLogFolder"].Value.ToString()
                    + "\\" + "ErrorLog_" + datetime + ".log"))
                {
                    sw.WriteLine(exception.ToString());
                    Dts.TaskResult = (int)ScriptResults.Failure;
                }

            }



Step 6:
Save the script and then Run your SSIS Package, it should create flat file in destination folder with date-time. You can change the type of extension , delimiter and can test by adding or dropping/adding columns to source table or view if all works fine.

I executed with by provide extension csv, delimiter as , and it created below flat file from my table.
How to create flat file dynamically from SQL Server Table or View in SSIS Package


Check out our other posts/ Videos on related topics