Quick answer: Azure Data Factory can securely retrieve credentials and secrets from Azure Key Vault by using the Data Factory managed identity. Create an Azure Key Vault linked service in ADF, grant the factory managed identity permission to read secrets, and then reference the required secret from your data-store or compute linked service.

Prerequisites
1. An Azure subscription with access to Azure Data Factory and Azure Key Vault.
2. A Data Factory with a system-assigned or supported user-assigned managed identity configured.
3. Permission to grant the Data Factory managed identity access to the Key Vault.

Not sure which managed identity type to use? See System-Assigned vs User-Assigned Managed Identity in Azure for the lifecycle and reuse differences.

Why use Azure Key Vault with Data Factory?

Connection strings, passwords, API keys, and other credentials should not be hard-coded in Azure Data Factory pipelines or stored as plain-text deployment parameters. Azure Data Factory can retrieve credentials from Azure Key Vault at runtime through its managed identity.

Microsoft recommends storing secrets in Azure Key Vault and using managed identities instead of managing credentials directly in Data Factory.

Create an Azure Key Vault linked service

In Azure Data Factory Studio, open Manage > Linked services, select New, and choose Azure Key Vault.

Create an Azure Key Vault linked service in Azure Data Factory

Select the Key Vault that contains your secrets. The Key Vault URL uses the format:

https://<key-vault-name>.vault.azure.net
Configure Azure Key Vault linked service in Data Factory

Grant Data Factory access to Key Vault

Azure Data Factory uses its managed identity to authenticate to Azure Key Vault. You can grant access using Azure RBAC or, where your Key Vault uses the legacy access-policy model, an access policy.

Recommended: Azure RBAC

  1. Open the Key Vault in the Azure portal and select Access control (IAM).
  2. Select Add > Add role assignment.
  3. Assign the Key Vault Secrets User role to the Data Factory managed identity.
  4. Under Members, choose Managed identity, select the Data Factory resource, and complete the role assignment.

For the normal scenario of reading secrets from Key Vault, the Data Factory managed identity needs permission to retrieve secret values. Microsoft documents Key Vault Secrets User as the RBAC role for this scenario.

Legacy: Key Vault access policies

If the Key Vault is configured to use access policies instead of Azure RBAC, add an access policy for the Data Factory managed identity and grant the required secret permissions. Microsoft documents Get and List for the Data Factory managed identity when using this pattern.

Grant Azure Data Factory managed identity access to Azure Key Vault

Available on Microsoft Store

SSRS Reports Migration Wizard

A simple Windows tool for migrating SSRS reports, data sources, and related configurations between report servers.

Reference a Key Vault secret from a linked service

When configuring a supported data-store or compute linked service, choose Azure Key Vault for the secret field. Select the Azure Key Vault linked service and provide the secret name.

You can optionally specify a secret version. If no version is specified, Data Factory uses the latest version of the secret.

For connectors that use a connection string, Data Factory can support storing the secret field or, where supported by the connector, the entire connection string in Key Vault.

{ “password”: { “type”: “AzureKeyVaultSecret”, “secretName”: “<secret-name>”, “store”: { “referenceName”: “<key-vault-linked-service>”, “type”: “LinkedServiceReference” } } }
Successful Azure Key Vault connection in Azure Data Factory

Use Key Vault secrets directly in pipeline activities

Data Factory also provides a documented pattern for retrieving a Key Vault secret during pipeline execution with a Web activity. This requires the Data Factory managed identity to have secret access and the Web activity’s Secure Output setting to be enabled. Any downstream activity that consumes the secret should also use Secure Input so the value is not exposed in activity logs.

For connection credentials, prefer a Key Vault-backed linked service rather than retrieving a secret into a pipeline and passing the value between activities. This keeps the credential handling closer to the linked-service authentication boundary.

Security and networking considerations

  • Use the least-privilege Key Vault role required by the Data Factory workload.
  • Do not put secret values in source control, ARM parameters, pipeline expressions, or scripts.
  • When rotating a secret, using the latest secret version can avoid updating the linked service for every rotation. Pin a version only when you deliberately need version-specific behavior.
  • If Key Vault network restrictions are enabled, verify that your Data Factory integration runtime and the selected Key Vault access pattern are compatible with those restrictions.

Pro tips:
1. Prefer Azure RBAC and the Key Vault Secrets User role for normal secret retrieval instead of granting broader Key Vault permissions.
2. Do not parameterize passwords or secret values in linked services. Parameterize the secret name when you need environment-specific secrets and keep the values in Key Vault.
3. If you rotate secrets frequently, consider leaving the secret version unspecified so the latest version is used.
4. For Azure DevOps deployments, keep Key Vault secret values outside the ARM template and deployment source code.
5. See Update Azure Key Vault Secret Using PowerShell to learn how to update a secret.
6. See Access Key Vault from Azure DevOps Pipeline for CI/CD scenarios.

See more

Visual Studio Marketplace

SSIS Catalog Migration Wizard

Extend Visual Studio with an easy way to migrate SSIS Catalog projects.

Get it from Visual Studio Marketplace
Pavan Bangad

9+ years of experience in building data warehouse and big data application.
Helping customers in their digital transformation journey in cloud.
Passionate about data engineering.