What Are The Pipelines and Activities in Azure Data Factory - ADF Tutorial 2021

 Topic: What are the Piepleines and Activities in Azure Data Factory.

In this article, we are going to learn about the pipelines and activities in the Azure data factory, and also we will learn how to put them together and how to group them and how to create folders.

Definition of pipeline:

A pipeline is a logical grouping of activities that performs a unit of work. Together, the activities in a pipeline perform a task. For example, a pipeline can contain a group of activities that ingests data from an Azure blob and then runs a Hive query on an HDInsight cluster to partition the data.

Activities in Pipeline.

Inside the activities, there are 11 tabs that contain tons of activities to be used for different purposes.



How to create a pipeline in Azure Data Factory.

Open Azure Data Factory Studio, Then go to the Author tab and click on pipelines, then click on the + button then click on the New pipeline to create a new pipeline. Then Name the pipeline and go to the activities tab and then search for copy data activity and then drag it into the working window, now you can create a source dataset to bring data from SQLDB or blob storage, and then you can create a sink dataset to write the data on blob storage or any SQL Server Database.


Then we have some more options, when you click on the ellipses button in front of the pipeline name it will show you some options, you can monitor your pipeline, or make a clone of your pipeline, or rename it, also you can move the pipeline, or you can download the support files or you can delete the pipeline.


How to create a Folder inside the Pipeline.

Click on the ellipses button on the pipeline tab, then click on New Folder.


Then name the pipeline and click on create it will create a new folder under the pipeline tab, once the folder is created you can also create subfolders inside. also you can move pipelines into the folders and you can delete or rename the pipelines.


How to Schedule a Pipeline.

Click on the trigger in and then click on New/Edit.



Name the trigger, then select the type as schedule, set Start date and time, then select a time zone, then set recurrence, and then click Ok.





Video Demo: What Are The Pipelines and Activities in Azure Data Factory



























5 comments:

  1. Hey! “Will you deliver top-quality if I hire you to write my paper for me?” Quality is our foremost priority, and we have a competent team of editors and quality assurance agents to assure it. They will check your work thoroughly to ensure it has no mistakes, explores the topic conclusively, and meets all your requirements.

    ReplyDelete
  2. Thanks for sharing,its really nice to read. react development services

    ReplyDelete
  3. We assure that we prioritize quality and productivity so that no student has any complaints regarding our website write my essay. This is abest writing website for essaysare available to you now.

    ReplyDelete
  4. Great insights on Azure Data Engineering! I completely agree with your points on the importance of mastering data pipelines for efficient data flow. In my experience, Azure's robust tools like Data Factory and Synapse Analytics have been game-changers in streamlining data processes and ensuring scalability. For anyone looking to deepen their understanding of these tools, I recommend checking out the resources at Code From Basics Azure Data Engineering Training in Chennai. They've got some excellent hands-on training that really makes the concepts click. Looking forward to more of your posts on this topic!

    ReplyDelete
  5. I highly recommend this service to anyone feeling overwhelmed—it's worth the investment for your academic success! I recently used a buy term paper essayshark.com , and it was a game-changer for my academic journey. The expert I worked with not only understood my topic thoroughly but also provided valuable insights that enhanced my research. The process was straightforward, and I appreciated the constant communication and updates throughout. It relieved so much stress, allowing me to focus on other coursework while ensuring my dissertation met high standards.

    ReplyDelete