With rapidly growing capabilities of data source integration around DevOps and Cloud infrastructure space, it is easy to find the right set of tooling strategies to build the data integration solutions across IaaS and PaaS services offered by various cloud service providers. However, when encountering the circumstances dealing with a hybrid connectivity scenario for replicating data between cloud and on-premises file system, Azure integration services offers the legacy file system connectivity using On-Premises data gateway and Azure Logic Apps file-system connector.
On-Premises Data Gateway
The On-premises data gateway acts as a bridge, providing quick and secure data transfer between on-premises data and Power BI, Microsoft Flow, Logic Apps, and PowerApps
- The gateway facilitates quick and secure behind-the-scenes communication. This communication flows from a user in the cloud to the on-premises data source and then back to the cloud.
- The gateway relies on Azure Service Bus for cloud connectivity. If a firewall blocks outbound connections, configure the firewall to allow outbound connections from the gateway to its associated Azure region.
- The gateway communicates on the following outbound ports: TCP 443, 5671, 5672, and from 9350 through 9354. The gateway doesn’t require inbound ports.
- By default, the installed service will be run-as a newly created account NT SERVICE\PBIEgwService.
Changing this account might create issues when attempting to register the instance in Azure
With Azure Logic app and the On-Premises file system connector, This article explains how to create a logic app workflow to
- Read a file from on-premises file system using On-Premise Data Gateway.
- Using Logic Apps workflow, Replicate the file to Azure blob storage when a file is added or modified in the source on-premises file system.
- Azure Logic App (Standard Plan)
- On-Premises Data Gateway
Connect to On-premises data gateway
To install the On-premises data gateway, refer to the page:
After you create the gateway resource in Azure portal, you can now create a connection between the logic app and the on-premises data source by using the gateway
- In the Azure portal, create or open the logic app in the Logic App Designer.
- Add a connector that supports on-premises connections such as File System connecter.
Upon creating a File System trigger/action, we need to create an API connection. Here is an example:
Create Azure Logic App workflow
To create a new logic app workflow,
- Create the standard logic app resource and add a blank stateful workflow.
Once the workflow created, open the workflow in designer (alternatively, we can use JSON editor) and build the workflow components.
In order to build the Logic Apps Workflow, create the required parameters and connectors. Finally after creating all the components, the workflow looks similar as below screenshot: