How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - Azure Data Factory Tutorial

 Issue: How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - ADF Tutorial.

In this article, we are going to learn how to write web activity output or Azure function JSON output to Azure Blob storage, in this scenario we are going to get data from a website, that provides the free Dummy JSON data, so I have copied the web link and then get data from that link then write it to the blob storage. let's start our demonstration.

Open your Azure data factory studio, go to the Author tab, click on + sign to create a new pipeline, find and bring the Web activity, click on the settings tab, paste the copied web link, in the method select Get, as we are getting the data from this web link.


Click on the white canvas, go to the Variables tab, click on the + New button to create a variable, name your variable and leave the rest as it is.


Find and drag the set variable activity, connect with the web activity, go to the Variables tab, select the variable that we created, then in the value click on Add dynamic content and select the web activity then add the curly brackets in the expressions.


Find and bring the copy data activity, connect with the set variable activity, go to the source tab then click on the + New button to create a new source dataset.


Select Azure blob storage then click on continue.


Select the file format as CSV, then click on continue.


Name your dataset, select the linked service pointing to your blob storage, and then create a template file in your blob storage and then select that file, as we will write the data in that template file, for import schema select from connection/store, then click on ok.


In the Source tab click on the + New button and create an additional column, then provide the column name and in the values, click on ad dynamic content and select my variable. 


Once we are done with our source tab, go to the sink tab, click on the + New button to create a new sink dataset.


Select Azure blob storage then click on continue.


Select CSV as file format and click on continue.


Name your dataset, select the linked service, select the output folder path, import schemas will be from connection/store, then click on ok.


In the sink tab click on the Open button and provide the output file name, then in the escape character select No escape character, and in the Quote, character select No quote character. 




Next, go to the mapping tab, click on import schemas, it will bring the column in the template file, select the column and click on delete, leave the second column which we have created in the additional column, then we are good to go, click on Debug to execute the piepline.




Video Demo: How to Write Web Activity Output OR Azure Function Json Output to Azure Blob Storage - ADF Tutorial















 

14 comments:

  1. Wow, cool post. I'd like to write like this too - taking time and real hard work to make a great article... but I put things off too much and never seem to get started. Thanks though. Digital Agency Melbourne

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Very precious focus of course. Cloud computing services or applications and platforms are rapidly growing across all industries, serving as the IT infrastructure which drives new digital businesses. These type of platforms and applications have revolutionized the ways in which businesses function, and have made processes easier. A recent study proves that more than 78 percent of businesses today have at least some portion of their computing infrastructure in the cloud. Now, the technical details on how to write web activity output or Azure function JSON output to Azure Blob storage you elaborated was totally mind blowing and praiseworthy I think. All practical focuses were very noteworthy. The video tutorial was also very useful. I like the way you brought the activity output issue. Though my browsing intention was to learn about web design service in Edmonton area (I have already visited a professional service named up digital ca. their Six Step Web Design Services are really notable compared to the other services in the region. You can also check out this edmonton web design business), I was completely spell-bound with the knowledgeable details you explored here for us. Thank you very much for this great contribution and sharing online.

    ReplyDelete
  4. Determination of some unacceptable sort hosting could decide the development or disappointment of your business. https://onohosting.com/

    ReplyDelete
  5. Could actually yield by using indistinguishable challenges! Permitted to enable here you may master it will looks. affordable web design for small business

    ReplyDelete
  6. La Touraine watches offers the best homage dive watches and unique style mens watches for under $500. Our curated selection blends value with class that doesn't require you to spend your entire paycheck just to look your best.

    ReplyDelete
  7. In the event that your website is responsive and prepared to support portable clients, you can exploit many devices and supportive applications like the snap to-call button, this empowers a web client to settle on a voice decision to your organization right away.
    http://hippocketdesigns.com

    ReplyDelete
  8. Since WordPress was at first sent off as a contributing to a blog instrument, even the most recent variants expect that your blog is the core of your website. WP CTA PRO

    ReplyDelete
  9. La Touraine watches offers affordable quality watches for less. microbrand gmt watch

    ReplyDelete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Entdecken Sie die zahlreichen Vorteile des IT-Nearshorings in der Ukraine! Dieses Land hat sich zu einem der führenden Ziele für Nearshoring entwickelt, dank seines gut ausgebildeten IT-Personals und seiner wettbewerbsfähigen Preise. Die Ukraine bietet auch eine reiche kulturelle Vielfalt und eine hohe Englischkenntnis, was die Kommunikation erleichtert. Mit Mobilunity als Partner können Sie Ihr Entwicklungsteam in der Ukraine zusammenstellen und von den erstklassigen Fähigkeiten und der Effizienz profitieren. Erfahren Sie, wie Sie Ihre Projekte erfolgreich durch Nearshoring in der Ukraine realisieren können. Seite:

    ReplyDelete
  12. Many have come to the conclusion that Bitcoin and other cryptocurrencies cannot be traced or recovered but it’s incorrect, it can be traced and recovered with the right tools and resources. I was one of those who didn’t believe in it but I was able to recover my Bitcoin after I sent a huge amount to the wrong address with the help of a recovery team called Hack Judas. I thought all hope was lost for good but with the intervention of Judas, I was able to trace and recover my Bitcoins. Truly remarkable work by Hack Judas and I highly recommend their service.

    Judas can be reached via E-mail: hackerjudas9@gmail.com
    INSTAGRAM: hackerjudas

    ReplyDelete
  13. Utilize Google's Search Quality Evaluator Guidelines for SEO insights. www.trevortynes.ca/website-design-sarnia/

    ReplyDelete