In Azure Data Factory, there are more than 90 pre-built connectors that you can use to connect to your data stores. In this blog post, I’ll show you what Azure Data Factory Linked Services are, how to create them, and how to add parameters.
Azure Data Factory Linked Services allow you to establish a connection with your data stores. They are connectors you can use while working with assets in data stores. This includes the configuration to access data stores, as well as connection strings and authentication type.
This blog post applies to Azure Data Factory and Azure Synapse Analytics.
Image copied from Microsoft documentation
Table of Contents
Create Azure Data Factory Linked Services
First, go to the Manage Hub. Click on Linked Services and create a new one.
Second, you can see the different categories and connectors that you can use. For this example, I’m using Azure SQL Databases. Click continue.
Next, you’ll be taken to the main configuration page for the linked service. Each connector requires different configuration parameters (like connection strings or a file path) while some of them are generic for all the parameters (like name, description, annotations, parameters)
- Define a name for your linked service
- Add a description
- Select the Azure Data Factory Integration Runtime to use. You can leave the default one for Azure Linked Services
- Use Azure Data Factory and Key Vault (recommended) or define the connection string and credentials (not recommended)
- Use Azure Data Factory Managed Identities (recommended) or define your SQL Credentials (not recommended)
- Create some annotations that allow you to monitor the workloads that use this linked service
- Create some parameters if needed (covered in the following section).
- Add custom code to the JSON definition
Then, test the connection and finally, create the linked service. The new Azure Data Factory Linked Service becomes available on the list. You can see that it has 0 related items.
Once you start creating Azure Data Factory datasets and pipelines, you can see related objects.
Azure Data Factory Parameters for Linked Services
Now, let’s say that you are working with dozens of databases or servers that are using the same connector. If you are not using parameters, you’ll end up with a really large number of linked services to maintain which becomes a daunting task.
Parameters in Azure Data Factory Linked Services are available to minimize the effort you put in when working on and maintaining your solutions.
Use parameters in conjunction with Azure Key Vault Secrets.
Let’s create a parameter our previous connection.
Click the connection and add a new parameter.
Next, add a parameter. You can add a default value.
Then, go to the secret configuration and add dynamic content.
Click the previously created Azure Data Factory Linked Service Parameter and then finish.
Now, the secret has been updated with the parameter.
If you click Test Connection, you will need to pass a parameter.
Finally, you can save the Azure Data Factory linked service.
When using datasets, you can define a different parameter value for your Azure Data Factory Linked Services which gives you a lot of flexibility.
Summary
To sum up, today you’ve learned how to create, use and add parameters to an Azure Data Factory Linked Service.
What’s Next?
In upcoming blog posts, we’ll continue to explore Azure Data Service features.
Please follow Tech Talk Corner on Twitter for blog updates, virtual presentations, and more!
