What Are The Pipelines and Activities in Azure Data Factory - ADF Tutorial 2021

 Topic: What are the Piepleines and Activities in Azure Data Factory.

In this article, we are going to learn about the pipelines and activities in the Azure data factory, and also we will learn how to put them together and how to group them and how to create folders.

Definition of pipeline:

A pipeline is a logical grouping of activities that performs a unit of work. Together, the activities in a pipeline perform a task. For example, a pipeline can contain a group of activities that ingests data from an Azure blob and then runs a Hive query on an HDInsight cluster to partition the data.

Activities in Pipeline.

Inside the activities, there are 11 tabs that contain tons of activities to be used for different purposes.



How to create a pipeline in Azure Data Factory.

Open Azure Data Factory Studio, Then go to the Author tab and click on pipelines, then click on the + button then click on the New pipeline to create a new pipeline. Then Name the pipeline and go to the activities tab and then search for copy data activity and then drag it into the working window, now you can create a source dataset to bring data from SQLDB or blob storage, and then you can create a sink dataset to write the data on blob storage or any SQL Server Database.


Then we have some more options, when you click on the ellipses button in front of the pipeline name it will show you some options, you can monitor your pipeline, or make a clone of your pipeline, or rename it, also you can move the pipeline, or you can download the support files or you can delete the pipeline.


How to create a Folder inside the Pipeline.

Click on the ellipses button on the pipeline tab, then click on New Folder.


Then name the pipeline and click on create it will create a new folder under the pipeline tab, once the folder is created you can also create subfolders inside. also you can move pipelines into the folders and you can delete or rename the pipelines.


How to Schedule a Pipeline.

Click on the trigger in and then click on New/Edit.



Name the trigger, then select the type as schedule, set Start date and time, then select a time zone, then set recurrence, and then click Ok.





Video Demo: What Are The Pipelines and Activities in Azure Data Factory



























1 comment:


  1. Its a very nice and informative article, Thank you for sharing this topic.
    visit:- Ace Divino

    ReplyDelete