How to Create Azure Data Lake Storage Gen2 & Copy Files From Blob Storage to Azure Data Lake Storage

Issue: How to Create Azure Data Lake Storage Gen2 & Copy Files From Blob Storage to Azure Data Lake Storage.


In this article, we are going to learn how to create Azure data lake storage Gen2 and copy files from blob storage to Azure Data Lake Storage, first of all, we will create blob storage, and then we will create an Azure Data Lake Storage Gen2, so you can understand the difference between both. 

How to create a blob storage account:

Open your Azure portal, find and go to the Storage accounts, and click on the + Create button to create a new storage account.


In the basics tab, select your subscription, select your resource group, name your storage account, select your region, leave the rest as it is and click on Review + create and then click on create.


How to create an ADLS Gen 2 storage account:

Open your Azure portal, find and go to the Storage accounts, and click on the + Create button to create a new storage account.



In the basics tab, select your subscription, select your resource group, name your ADLS Gen 2 storage account, select your region, then select performance as Premium, select the account type as Block blobs, and then go to the advanced tab.


In the advanced tab enable the Hierarchical namespace, then click on Review + create and then click on create. 


Once our ADLS Gen2 storage account is created, go to the ADLS Gen2 storage, click on containers/folders under the data storage tab, then click on + Create to create a container/folder.


Next, open the azure data factory studio, go to the author tab, find and drag the copy data activity, go to the source tab, and click on the + New button to create a source dataset.


Select the Azure blob storage, and click on continue.


Select binary as format and click on continue.


Name your dataset, select the linked service, then provide the folder path and then click on ok.


Once we are done with our source dataset, let's go to the sink tab and click on the + New button to create a new sink dataset.


Select Azure Data Lake Storage Gen 2, then click on continue.


Select binary as format and click on continue.


Name your sink dataset, select the linked service, provide the folder/container path, and then click on ok



Once we are done with our pipeline, click on the Debug button, and it will copy all the files and folders from our blob storage to our Azure Data Lake Storage Gen2.



Video Demo: How to Create Azure Data Lake Storage Gen2 & Copy Files From Blob Storage to Azure Data Lake Storage
 







No comments:

Post a Comment