In this article, we will see how to compare Azure SQL database schema using sqlpackage.exe and generate a delta script that shows the differences in a controlled and repeatable way. The approach and flow are similar to the DACPAC-based comparison strategy discussed earlier on AzureOps.

Why use sqlpackage for database comparison?

sqlpackage.exe is a command-line utility provided by Microsoft that works with DACPAC and BACPAC files. Using sqlpackage, you can:

• Compare two databases using DACPACs
• Generate a deployment (delta) script
• Control destructive changes explicitly
• Automate comparisons in CI/CD pipelines

For Azure SQL Database scenarios, this method is especially useful when direct schema comparison tools are not available or when automation is required.

Creating a DACPAC

To compare database schemas using sqlpackage, you first need a DACPAC file, which represents the database schema.

You can create a DACPAC using sqlpackage.exe, SSMS, or Visual Studio. For a quick, step-by-step guide covering all methods, refer to this article.

Installing sqlpackage Before You Begin

Before you can compare two Azure SQL database schemas using sqlpackage, you need to install the sqlpackage command-line utility on your machine. sqlpackage.exe is the same tool that Powers the DACPAC deployments discussed in this site’s previous article on deploying DACPACs to Azure SQL Database.

Here are the recommended ways to install sqlpackage, depending on your environment:

Option 1 — Install as a .NET Global Tool (Cross-Platform)

The easiest and most modern way to install sqlpackage on Windows, macOS, or Linux is using the .NET SDK tool installer. This gives you the sqlpackage command directly on your PATH. Run below command in command prompt.

dotnet tool install --global Microsoft.SqlPackage

If you already have it installed and want to update to the latest version, run:

dotnet tool update --global Microsoft.SqlPackage

This method works well for local development and automation scripts, including PowerShell and CI/CD pipelines.

Option 2 — Download Standalone Zip (Windows / macOS / Linux)

If you don’t use the .NET SDK or prefer a self-contained download, Microsoft also publishes standalone builds of sqlpackage:

1. Visit the official SqlPackage download page on Microsoft Docs.
2. Download the appropriate zip for your OS.
3. Extract and place the folder somewhere on your system (e.g., C:\tools\sqlpackage).
4. Add the folder containing sqlpackage.exe to your system PATH so you can call it from any terminal.

This method is useful if you want to avoid installing .NET tools or need to bundle sqlpackage with deployment scripts.

High-level approach

The comparison process consists of three simple steps:

1. Export both Azure SQL databases to DACPAC files
2. Use sqlpackage.exe with /Action:Script
3. Review the generated delta script before applying it

In this article, we focus on step 2, assuming you already have DACPAC files for both databases.

Sample scenario

Let us assume the following setup:

• Source (baseline) database: sales-test-db
• Target (to be aligned) database: sales-prod-db

Our goal is to compare Azure SQL Database schema and generate a SQL script that shows what changes are required to make Production match Test.

Sample sqlpackage command

Below is a sample sqlpackage.exe command to compare two DACPAC files and generate a delta script.

sqlpackage.exe ^
  /Action:Script ^
  /SourceFile:"C:\Dacpacs\sales-test-db.dacpac" ^
  /TargetFile:"C:\Dacpacs\sales-prod-db.dacpac" ^
  /TargetDatabaseName:"sales-prod-db" ^
  /OutputPath:"C:\Dacpacs\sales_test_vs_prod_delta.sql" ^
  /p:DropObjectsNotInSource=True

In case you want to use PowerShell ISE, you can use below script.

# Path to sqlpackage.exe
$sqlPackagePath = "C:\Program Files\Microsoft SQL Server\160\DAC\bin\sqlpackage.exe"

# DACPAC paths
$SourceDacpacPath = "C:\Dacpacs\sales-test-db.dacpac"
$TargetDacpacPath = "C:\Dacpacs\sales-prod-db.dacpac"

# Target database name
$TargetDatabaseName = "sales-prod-db"

# Output script path
$OutputScriptPath = "C:\Dacpacs\sales_test_vs_prod_delta.sql"

# Publish options
$DropObjectsNotInSource = "True"

# Execute sqlpackage
& $sqlPackagePath `
    /Action:Script `
    /SourceFile:$SourceDacpacPath `
    /TargetFile:$TargetDacpacPath `
    /TargetDatabaseName:$TargetDatabaseName `
    /OutputPath:$OutputScriptPath `
    /p:DropObjectsNotInSource=$DropObjectsNotInSource

This command does not apply any changes. It only generates a SQL script that you can review and execute manually if needed.

Understanding each parameter

Let us break down all the parameters used in the command.

/Action:Script

Specifies that sqlpackage should generate a deployment script instead of publishing changes directly to a database. This is the safest option when comparing environments like Test and Production.

/SourceFile

Path to the source DACPAC file. This represents the desired state of the database.

/SourceFile:"C:\Dacpacs\sales-test-db.dacpac"

In our example, the Test database schema is treated as the source of truth.

/TargetFile

Path to the target DACPAC file. This represents the database you want to compare against.

/TargetFile:"C:\Dacpacs\sales-prod-db.dacpac"

The generated script will contain changes required to make the target match the source.

/TargetDatabaseName

Logical name of the target database.

/TargetDatabaseName:"sales-prod-db"

This parameter is mandatory for the Script action, even when you are comparing DACPAC-to-DACPAC.

/OutputPath

Specifies where the generated delta script should be written.

/OutputPath:"C:\Dacpacs\sales_test_vs_prod_delta.sql"

Always review this script carefully before executing it in Production.

/p:DropObjectsNotInSource

Controls whether objects that exist in the target but not in the source should be dropped.

/p:DropObjectsNotInSource=True

• True – Generates DROP statements for extra objects in the target
• False – Preserves objects that exist only in the target

⚠️ Use this option with caution, especially for Production databases.

Common additional publish parameters

Depending on your requirements, you may also consider these commonly used parameters:

• /p:BlockOnPossibleDataLoss=False
• /p:IgnorePermissions=True
• /p:IgnoreRoleMembership=True
• /p:ExcludeObjectTypes=Users;Logins
• /p:ExcludeSchemaValidation=True

These options help fine-tune the comparison and avoid unwanted differences.

Best practices

• Always generate a script first (/Action:Script)
• Review the delta script line by line
• Never auto-apply changes to Production without validation
• Store DACPACs and scripts in source control
• Use the same approach in CI/CD pipelines for consistency

Conclusion

Comparing two Azure SQL databases using sqlpackage.exe is a reliable and automation-friendly approach. By working with DACPAC files and generating delta scripts, you gain full visibility and control over schema changes before they reach Production.

This method fits well into modern DevOps workflows and provides a safe alternative to manual or GUI-based comparisons.

If you regularly manage schema changes across multiple environments, sqlpackage should be a core part of your database deployment tool.

Pro tips
1. If you want to generate BACPAC file of an Azure SQL database, you can refer to this post

See more

The post Compare Azure SQL Database Schema Using sqlpackage appeared first on AzureOps.

Use Case – Imagine this scenario:

• You have a main transactional database named app-db hosted on Azure SQL Server app-db.database.windows.net.
• You also have a reference or lookup database named source-db, hosted on a separate Azure SQL Server: source-server.database.windows.net.

You want your app-db to query the table dbo.external_table from the source-db — without importing or copying the data. The goal is to enable live, read-only access to shared reference data across databases in a secure and manageable way.

Prerequisites
Before diving into the setup, ensure the following:
1. Azure SQL Databases: Both the source and target databases should be Azure SQL Databases.
2. Same Region: Ensure both databases are in the same Azure region to minimize latency.
3. User Credentials: A SQL user say (ext_table_user) must exist in the source database with the necessary permissions to access the target table.

Why Create Separate user in the Source Database?

The ext_table_user is a SQL user in the source database that the target database uses to authenticate and access data. This user must have the necessary permissions (e.g., SELECT) on the target table.

Creating a dedicated user for external access enhances security by allowing you to:

• Limit Permissions: Grant only the necessary permissions to this user.
• Monitor Access: Track and audit activities performed by this user.
• Revoke Access Easily: If needed, you can revoke this user’s access without affecting other users.

Fow these steps to how to grant access to Azure SQL database.

How to Setup Cross Database Access in Azure SQL

Follow the below quick steps to setup cross database access in Azure SQL database.

Step 1: Create a Master Key in the Target Database

The master key is essential for encrypting credentials used in external data sources.

-- Execute in the target database

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'YourStrongPasswordHere';

Skip this if SELECT * FROM sys.symmetric_keys WHERE name = '##MS_DatabaseMasterKey##' already returns a row.

Step 2: Create a Database Scoped Credential

This credential will be used to authenticate against the source database.

-- Execute in the target database

CREATE DATABASE SCOPED CREDENTIAL ext_table_user
WITH IDENTITY = 'ext_table_user',
     SECRET = 'ActualPasswordOfExtTableUser';

Step 3: Define the External Data Source

This step establishes a connection to the source database.

-- Execute in the target database

CREATE EXTERNAL DATA SOURCE LinkedAzureSQLDB
WITH (
    TYPE = RDBMS,
    LOCATION = 'source-server.database.windows.net',
    DATABASE_NAME = 'source-db',
    CREDENTIAL = ext_table_user
);

Ensure: Replace 'source-server.database.windows.net' and 'source-db' with your actual server and database names.

Step 4: Create the External Table

Now, define the external table in the target database to reference the table in the source database.

-- Execute in the target database

CREATE EXTERNAL TABLE dbo.external_table (
    [ID] NUMERIC(18, 0) NOT NULL,
    [list_id] INT NOT NULL,
    [type_id] INT NOT NULL,
    [description] VARCHAR(50) NULL
)
WITH (
    DATA_SOURCE = LinkedAzureSQLDB,
    SCHEMA_NAME = 'dbo',
    OBJECT_NAME = 'external_table'
);

Note: The schema and object names should match those in the source database.

Step 5: Query the External Table

With the setup complete, you can now query the external table as if it were local:

-- Execute in the target database

SELECT TOP 10 * FROM dbo.external_table;

Performance Tip: Monitor query performance, as cross-database queries may have higher latency.

Pro tips:
1. Use Managed Identities: For enhanced security, consider using Azure Managed Identities instead of SQL authentication.
2. Monitor Performance: Regularly monitor the performance of cross-database queries to ensure they meet your application’s requirements.
3. Secure Credentials: Store sensitive information, like passwords, securely using Azure Key Vault.
4. Elastic query feature in Azure SQL is still in preview.

By following this guide, you can efficiently set up cross databse query in Azure SQL, enabling seamless cross-database querying while maintaining security and performance.

See more

The post Cross Database Query in Azure SQL Database appeared first on AzureOps.

You write the query, run it, catch a missing condition, tweak the join, test again—repeat until it’s right. SSMS 21 brings Copilot into that loop.

TL;DR:

• SSMS 21 now includes built-in Copilot—no extensions. Just require connection to your Azure OpenAI deployment (like GPT-4o).
• Works only with SQL Server connections (not SSIS, SSAS, or SSRS).
• Runs in read-only mode—won’t execute any DML or modify data.
• Can generate queries, fix broken SQL, and suggest optimizations.
• Supports prompts like: “fix this query,” “optimize for performance,” or “generate insert script.”
• Ideal for SQL developers who want help writing, debugging, or learning better query patterns—without giving up control.

Installation: Copilot Comes Built-in

The installer for SSMS 21 clearly shows that Copilot is not an afterthought—it’s front and center. Once you launch the installer, you’ll notice the “Copilot Integration” checkbox enabled by default. This means no manual setup is required post-install. It’s part of the core experience now.

First Run: Setting Up Your Azure OpenAI Keys

Once SSMS launches, you’ll be prompted to connect Copilot to your Azure OpenAI setup.

In the first prompt, you’ll be asked to provide:

• Azure OpenAI service endpoint
• Azure OpenAI deployment
• Azure OpenAI API key

If you haven’t already created this in Azure, it’s a quick process: deploy a model, generate a key, and you’re good to go. we see how this information is pasted into SSMS.

It’s a one-time step, and once entered, Copilot starts responding instantly within SSMS.

To get an Azure OpenAI deployment name for SSMS Copilit, we need Gen AI model. That’s the engine powering your Copilot. GPT-4o is optimized for speed and real-time interaction, making it well-suited for something as dynamic as SQL writing and analysis.

If you already use GPT-based models in Azure, you can reuse the same deployment. If not, create a new one and pick GPT-4o or other depending on availability.

Which Connections Are Supported?

As of now, Copilot in SSMS works only with SQL Server connections—whether it’s a local database or Azure SQL.

It does not support:

• SQL Server Integration Services (SSIS)
• SQL Server Analysis Services (SSAS)
• SQL Server Reporting Services (SSRS)

So if you’re connected to an SSIS or SSAS instance, you won’t see Copilot suggestions. It’s focused purely on Transact-SQL (T-SQL) scenarios for now.

Read-Only Mode – No Live Data Changes

Copilot works in a read-only mode inside SSMS for GOOD.

It will not execute updates, deletes, or any data-modifying queries on its own. It doesn’t have permissions to alter databases or run DML commands. You remain in full control.

That said, it can generate update, insert, and delete queries for you, which you can review, tweak, and run manually. This helps reduce the risk of accidental changes while still speeding up development.

Quick Demo: Fixing a Broken Query

I pasted a malformed SELECT query into SSMS. Copilot immediately offered a fix and even explained what was wrong.

Few Prompt Examples: Result Accuracy

In below example, we asked Copilot to generate a complex query involving filters, aggregates, and joins. The result was accurate and ready to run with minimal edits.

the same query was further optimized by Copilot after being prompted with: “Make it efficient.” It introduced indexing suggestions and added a WITH (NOLOCK) hint—something even experienced developers can forget.

In another example, we asked Copilot another question on a different query context (another database connection), and it just worked fine.

These examples show that Copilot isn’t just copying boilerplate SQL—it understands context and best practices.

Should You Use This?

If your daily work involves writing or reading SQL, then yes—this is probably the most useful update to SSMS in years.

Here’s what it helps with:

• Writing long queries faster
• Understanding joins and filters
• Fixing syntax issues
• Learning better SQL patterns
• Generating stored procedure templates
• Drafting DML scripts (but not running them automatically)

What It Won’t Do (Yet)

Let’s be clear:

• It won’t connect to SSIS/SSAS/SSRS
• It won’t execute anything that modifies data
• It won’t automatically run DDL/ DML code it suggests

This is intentional, for safety and audit reasons. Think of it as a smart assistant—not a trigger-happy automation engine.

A Note on Privacy and Control

Copilot integrates directly with your Azure environment, so you control what it accesses and shares. It processes your prompts and data securely within your subscription. According to Microsoft, it doesn’t store any query logs outside your environment, which helps meet compliance requirements for enterprise users.

SSMS 21 doesn’t just add Copilot—it integrates it naturally into the flow of writing, editing, and reviewing SQL. It’s fast, context-aware, and doesn’t get in the way.

Pro tips:
1. While it’s good to have Copilot like features in SSMS 21, it seems to have lost some very important features like Import Export Wizard for databases like Azure SQL.
2. Learn how to build data agent in Microsoft Fabric.

See more

The post SSMS 21 Copilot: Here’s How It Works appeared first on AzureOps.

Prerequisites:
1. Visual Studio with data tools installed on your system.
2. An Azure SQL Database set up and accessible.
3. A DACPAC file ready for deployment.

Create a DACPAC File

In this post, we cover the process of creating a DACPAC file, a critical artifact that encapsulates the schema and objects of a SQL Server database. Learn how to build and validate your DACPAC files to ensure seamless deployments.

Open a DACPAC File in Visual Studio

Discover how to open and inspect a DACPAC file in Visual Studio. This guide provides details of how to open DACPAC file in Visual Studio.

Deploy DACPAC to Azure SQL Database from Visual Studio

Follow the below steps to deploy the DACPAC project from visual Studio. You can follow the same below steps to deploy database project to any SQL Server database or Azure Managed Instance. Make sure to select correct target platform and provide applicable connection details.

1. Open Visual Studio

1. Launch Visual Studio and open your solution or project.
2. Ensure you have the SQL Server Data Tools (SSDT) installed.

2. Change target platform of database project

1. Right-click on database project and click properties.
2. Change the Target platform to Microsoft Azure SQL Database as shown in the below image.

3. Publish the Database Project

Right-click on the project and select Publish from the context menu.

In Solution Explorer, locate your database project.

4. Configure the Deployment Target

1. In the Publish Database dialog, click Edit to specify the target database.

2. In the Edit Target Database Connection window:

• Select Azure SQL Database as the target.
• Enter the server name, authentication details, and select the target database.
• Test the connection to ensure it is valid.
• Click OK to confirm.

5. Review and Publish

1. Back in the Publish Database dialog, review the settings to ensure the correct DACPAC file and target database are selected.

2. Optionally, click Advanced to configure deployment options such as excluding certain schema changes as shown in the below image.

3. Click Publish to start the deployment process.

6. Monitor and Verify Deployment

1. Monitor the progress of dacpac publish in Visual Studio as shown in the below image.
2. Once the deployment is complete, open SQL Server Management Studio (SSMS) or use Visual Studio’s built-in tools to verify the schema changes in your Azure SQL Database.
3. Run test queries to confirm the database is functioning as expected.

Pro tips:
1. Refer to this post if you wish to Automate Azure SQL Database deployment using Azure DevOps CICD pipeline.
2. If you skip target platform change step during the deployment, you may get an error “A project which specifies SQL Server 2022 or Azure SQL Database Managed Instance as the target platform cannot be published to Microsoft Azure SQL Database v12.”. Refer to this post to learn more about it.
3. Test your DACPAC deployment on a local environment before pushing it to Azure.
4. Always take a backup of your target Azure SQL Database before deployment.
5. Use tools like SQL Compare to identify schema differences between the DACPAC and the target database.

FAQ

See more

The post Deploy DACPAC to Azure SQL Database appeared first on AzureOps.

Why JSON in SQL Server?

JSON allows applications to exchange structured data with simplicity and efficiency. SQL Server’s JSON support makes it easy to:

1. Store JSON data in table columns.
2. Query JSON data directly from the database.
3. Transform JSON data into relational formats.

The key functions provided by SQL Server for JSON manipulation include:

• OPENJSON(): Parses JSON text and returns objects in tabular format.
• JSON_VALUE(): Extracts a scalar value from a JSON string.
• JSON_QUERY(): Retrieves a JSON object or array from a JSON string.

How to Parse JSON in SQL Query

Let us see various examples of parsing JSON strings in SQL Server.

Parsing JSON with OPENJSON()

The OPENJSON() function is the most versatile tool for working with JSON in SQL Server. It can parse JSON text and return a result set that can be queried like a regular table.

Example 1: Basic JSON Parsing

Suppose you have the following JSON stored in a variable:

DECLARE @json NVARCHAR(MAX) =
    N'{
        "employees": [
            {"id": 1, "name": "John Doe", "role": "Manager"},
            {"id": 2, "name": "Jane Smith", "role": "Developer"}
        ]
    }';

To parse this JSON and extract the employee details, you can use OPENJSON() with a specified path:

SELECT id, name, role
FROM OPENJSON(@json, '$.employees')
WITH (
    id INT '$.id',
    name NVARCHAR(50) '$.name',
    role NVARCHAR(50) '$.role'
);

Output:

Explanation:

• '$' denotes the root of the JSON object.
• $.employees navigates to the employees array within the JSON.
• The WITH clause specifies the schema for the JSON fields to be extracted.

Extracting Values with JSON_VALUE()

The function is ideal if you need to extract individual values from a JSON string.

Example 2: Extract a Specific Value

Using the same JSON example above, you can retrieve the name of the first employee:

SELECT JSON_VALUE(@json, '$.employees[0].name') AS FirstEmployeeName;

Output:

Extracting Nested Objects with JSON_QUERY()

The JSON_QUERY() function is used to extract complex objects or arrays from a JSON string.

Example 3: Extract an Array

To extract the entire employees array:

SELECT JSON_QUERY(@json, '$.employees') AS EmployeesArray;

Combining JSON Functions in Queries

You can combine these functions to manipulate and query JSON data effectively.

Example 4: Parsing JSON Stored in a Table

Assume you have a table named EmployeeData with a column EmployeeJSON that stores JSON strings.

CREATE TABLE EmployeeData (
    Id INT IDENTITY PRIMARY KEY,
    EmployeeJSON NVARCHAR(MAX)
);

INSERT INTO EmployeeData (EmployeeJSON)
VALUES
(N'{"id": 1, "name": "John Doe", "role": "Manager"}'),
(N'{"id": 2, "name": "Jane Smith", "role": "Developer"}');

To extract and display employee details:

SELECT
    JSON_VALUE(EmployeeJSON, '$.id') AS EmployeeId,
    JSON_VALUE(EmployeeJSON, '$.name') AS EmployeeName,
    JSON_VALUE(EmployeeJSON, '$.role') AS EmployeeRole
FROM EmployeeData;

Output:

Parsing Complex JSON Objects

SQL Server can handle deeply nested JSON structures. Let’s consider a more complex example:

Example 5: Parsing Nested JSON with Arrays and Objects

Suppose you have a JSON structure with nested objects:

DECLARE @complexJson NVARCHAR(MAX) =
    N'{
        "company": {
            "name": "TechCorp",
            "employees": [
                {
                    "id": 1,
                    "name": "John Doe",
                    "details": {
                        "role": "Manager",
                        "salary": 75000
                    }
                },
                {
                    "id": 2,
                    "name": "Jane Smith",
                    "details": {
                        "role": "Developer",
                        "salary": 60000
                    }
                }
            ]
        }
    }';

To extract employee details including nested properties:

SELECT id, name, role, salary
FROM OPENJSON(@complexJson, '$.company.employees')
WITH (
    id INT '$.id',
    name NVARCHAR(50) '$.name',
    role NVARCHAR(50) '$.details.role',
    salary INT '$.details.salary'
);

Explanation:

• The path $.company.employees navigates to the employees array inside the company object.
• Nested properties like role and salary are accessed using the full JSON path ($.details.role and $.details.salary).

Output:

Pro tips:
1. SQL Server’s JSON functions make it incredibly simple to parse and manipulate JSON data without requiring additional tools or complex workarounds. Whether you’re working with JSON stored in variables or tables, these functions provide powerful options for extracting and querying data.
2. Understanding these features simplifies JSON handling and enhances your ability to work effectively with modern data formats. Start leveraging SQL Server’s JSON capabilities in your next project to streamline your data workflows.
3. SQL Server 2025 private preview is announced. Read more about it here.
4. If you want to improve performance of SQL Server database, you can find long running queries by following this post.

See more

The post Parse JSON in SQL Server appeared first on AzureOps.

Error

##[error]*** An error occurred during deployment plan generation. Deployment cannot continue.
##[error]A project which specifies SQL Server 2022 or Azure SQL Database Managed Instance as the target platform cannot be published to Microsoft Azure SQL Database v12.
##[error]The Azure SQL DACPAC task failed. SqlPackage.exe exited with code 1.Check out how to troubleshoot failures at https://aka.ms/sqlazuredeployreadme#troubleshooting-

Solution

1. Open the database dacpac project in Visual Studio.

2. Right Click on the database project and Click on ‘Properties’.

3. Open ‘Target platform’ drop down and select ‘Microsoft Azure SQL Database’.

4. Save the project and build it.

5. If the build is successful, raise a PR to merge this change with main branch and initiate the Azure DevOps CICD pipeline.

See more

Pro tips:
1. Follow the steps mentioned in the article deploy azure sql database using devops to automate Azure SQL database deployments.

The post A project which specifies SQL Server 2022 or Azure SQL Database Managed Instance as the target platform cannot be published to Microsoft Azure SQL Database v12. appeared first on AzureOps.

SQL query to find long running queries in SQL database

-- Query to check currently running SQL queries in Azure SQL Database or SQL Server database.

SELECT 
    r.session_id,
    r.status,
    r.command,
    r.start_time,
    r.percent_complete,
    SUBSTRING(qt.text, (r.statement_start_offset/2) + 1, 
    ((CASE r.statement_end_offset 
        WHEN -1 THEN DATALENGTH(qt.text)
        ELSE r.statement_end_offset 
        END - r.statement_start_offset)/2) + 1) AS query_text,
    s.host_name,
    s.login_name,
    s.program_name,
    s.client_interface_name,
    r.database_id,
    DB_NAME(r.database_id) AS database_name,
    r.total_elapsed_time,
    r.cpu_time,
    r.reads,
    r.writes,
    r.logical_reads
FROM 
    sys.dm_exec_requests r
JOIN 
    sys.dm_exec_sessions s ON r.session_id = s.session_id
CROSS APPLY 
    sys.dm_exec_sql_text(r.sql_handle) AS qt
WHERE 
    r.session_id <> @@SPID
ORDER BY 
    r.start_time DESC;

Here,
sys.dm_exec_requests (r): Lists currently executing requests.
sys.dm_exec_sessions (s): Contains info on active user connections and internal tasks.
sys.dm_exec_sql_text (qt): Retrieves the text of the SQL batch for the given SQL handle.

Pro tips:
1. DMV queries should be run with caution in production environments.
2. Above query can also be run the check currently running queries in SQL Server database.
3. In case you want to automate database maintenance in Azure SQL, refer to this post.

See more

The post Find Long Running Queries in Azure SQL Database appeared first on AzureOps.

Prerequisites:
1.Service connections in Azure DevOps project for each resource groups containing Azure SQL databases.

Create a database project and link it to Azure Repos

When setting up a CI/CD pipeline for Azure SQL Database, it’s essential to understand how to create a DACPAC file and utilize Visual Studio to a database project from it. This project then needs to be integrated with Azure Repos for setting up an automated build and deployment pipeline. In case you already have this setup, you can skip this section.

Create a DACPAC file

A DACPAC (Data-tier Application Component Package) file is a portable representation of a SQL Server database schema, making it crucial for efficiently managing database changes. Follow the steps mentioned in this post to learn how to create it. If you already have a database project, you can skip this step.

Create a database project from the DACPAC file

Once the DACPAC file is ready, integrating it into a Visual Studio project becomes imperative for further development and deployment. Detailed instructions on opening a DACPAC file in Visual Studio can be found in the guide

Link database project with Azure Repos

Create a repo in Azure DevOps under the DevOps organization project and link the database project with it. Follow the steps mentioned in this post to learn about it.

Setup Azure SQL Server CICD pipeline

Once we have Azure repo ready for Azure SQL database projects, let us now implement SQL DACPAC deployment Azure DevOps pipeline.

Enable access to Azure Key Vault from Azure DevOps pipelines

To connect to Azure SQL Database from Azure DevOps deployment pipeline, we need to fetch database credentials from Azure Key Vault. Follow the steps mentioned in this article to enable access to Azure Key Vault secrets from Azure DevOps pipeline.

Create a new pipeline in Azure DevOps project

It’s time to set up a deployment pipeline for the Azure SQL database.

1. Login to Azure DevOps portal and navigate to the DevOps organization. Select the DevOps project containing the database Azure repo we created in the above step.

2. Select Pipelines and Click ‘New pipeline’ button to create a new pipeline.

3. Select ‘Azure Repos Git’ as a source of your code.

4. Select the Azure repo containing the database project we created in previous steps.

5. On the Configure screen, select the ‘Starter pipeline’ option as shown in the image below.

6. Add below YAML code to the editor on Review screen and make necessary changes as per your environment. You can use YAML pipeline tasks to generate the code snippets based on your parameter values.

trigger:
# Name of the main (master) branch of the Azure DevOps repos. 
- main

variables:
  vmImageName: 'windows-latest'
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

stages:
- stage: build
  displayName: Build Stage
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: NuGetToolInstaller@1
    - task: NuGetCommand@2
      inputs:
        restoreSolution: '$(solution)'
    - task: VSBuild@1
      inputs:
        solution: '$(solution)'
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'
    - task: CopyFiles@2
      inputs:
        SourceFolder: '$(agent.builddirectory)'
        Contents: '**/bin/$(buildConfiguration)/**'
        TargetFolder: '$(build.artifactstagingdirectory)'
        CleanTargetFolder: true
    - task: PublishBuildArtifacts@1
      inputs:
        PathtoPublish: '$(build.artifactstagingdirectory)'
        ArtifactName: 'drop'
        publishLocation: 'Container'

- stage: dev
  displayName: Dev Deploy Stage
  dependsOn: build
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
  jobs: 
  - deployment: Deploy
    displayName: Deploy Dev
    pool:
      vmImage: $(vmImageName)
    environment: dev
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureKeyVault@1
            inputs:
             # Enter name of the service connection created to deploy resources to dev Azure SQL database. 
              AzureSubscription: 'dev-service-connection-name'
             # Enter name of the key vault that stores Azure SQL database credentials. 
              KeyVaultName: 'dev-keyva'
             # Enter comma separated names of the key vault secrets that store the Azure SQL database credentials.
              SecretsFilter: 'azuresqldb-dbconnstring'
              RunAsPreJob: true

          - task: SqlAzureDacpacDeployment@1
            inputs:
             # Enter name of the service connection created to deploy resources to dev Azure SQL database. 
              AzureSubscription: 'dev-service-connection-name'
              AuthenticationType: 'connectionString'
             # Provide the database connection string variable value to ConnectionString parameter. (key vault secret name)
              ConnectionString: '$(azuresqldb-dbconnstring)'
              DeployType: 'DacpacTask'
              DeploymentAction: 'Publish'
              # Provide path of the dacpac file. Replace 'AzureOps.Sql' with name of your AzureRepo and AzureOps.Sql.dacpac with name of your database dacpac file.
              DacpacFile: '$(Pipeline.Workspace)/drop/s/AzureOps.Sql/bin/Release/AzureOps.Sql.dacpac'
              AdditionalArguments: '/p:DropObjectsNotInSource=true /p:BlockOnPossibleDataLoss=true /p:IgnorePermissions=true /p:ExcludeObjectTypes="Users;Permissions"'
              IpDetectionMethod: 'AutoDetect'

# similarly, customise below block of code for test environment deployment.
- stage: test
  displayName: Test Deploy Stage
  dependsOn: dev
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
  jobs: 
  - deployment: Deploy
    displayName: Deploy Test
    pool:
      vmImage: $(vmImageName)
    environment: test
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureKeyVault@1
            inputs:
              AzureSubscription: 'test-service-connection-name'
              KeyVaultName: 'test-keyva'
              SecretsFilter: 'azuresqldb-dbconnstring'
              RunAsPreJob: true

          - task: SqlAzureDacpacDeployment@1
            inputs:
              azureSubscription: 'test-service-connection-name'
              AuthenticationType: 'connectionString'
              ConnectionString: '$(azuresqldb-dbconnstring)'
              DeployType: 'DacpacTask'
              DeploymentAction: 'Publish'
              DacpacFile: '$(Pipeline.Workspace)/drop/s/AzureOps.Sql/bin/Release/AzureOps.Sql.dacpac'
              AdditionalArguments: '/p:DropObjectsNotInSource=true /p:BlockOnPossibleDataLoss=true /p:IgnorePermissions=true /p:ExcludeObjectTypes="Users;Permissions"'     
              IpDetectionMethod: 'AutoDetect'

Note:
You can add AdditionalArguments to SqlAzureDacpacDeployment task like DropObjectsNotInSource=true which will drop objects from the database which do not exists in DACPAC file. Read more about all the properties supported in SqlAzureDacpacDeployment. Exercise this with caution.

Review and test the pipeline

Click on ‘Run pipeline’ button to manually trigger the pipeline and test if the pipeline works without any issues.

How to Manually Deploy DACPAC project to Azure SQL Database

Refer to this post if you want to deploy DACPAC project to Azure SQL database using Visual Studio.

Pro tips:
1. When publishing a DACPAC, the pipeline might fail with potential data loss errors. Customize the publish parameters according to your specific requirements in such cases.
2. Before implementing a block for production deployment, it’s advisable to thoroughly test the pipeline in lower environments. This ensures that any issues or bugs are identified and resolved before reaching critical stages of deployment.
3. To prevent unintended code deployments to higher environments, incorporate mandatory approvals at each stage of deployment. This ensures that changes are reviewed and approved before progressing to the next environment, maintaining the integrity of the deployment process.
4. If you come access error ‘A project which specifies SQL Server 2022 or Azure SQL Database Managed Instance as the target platform cannot be published to Microsoft Azure SQL Database v12., follow this article to resolve the issue.

See more

The post Implement Azure SQL Database Deployment CI CD pipeline appeared first on AzureOps.

Problem

Performing extensive DML operations, particularly in data warehouse scenarios, may lead to transaction log space constraints, risking transaction failures and database downtime during rollbacks.

Solution

Transaction batching in SQL Server: Transaction batching, dividing larger transactions into smaller, equal-sized batches, helps optimize resources and minimize the impact of potential rollbacks.

Example

Consider a scenario where we batch updates and deletes in a temporary table named #temp. We use the SET ROWCOUNT to limit batch sizes and the @@ROWCOUNT function to track the number of records affected in each batch.

Batching Updates

CREATE TABLE #temp (
  id INT IDENTITY(1,1),
  name VARCHAR(10),
  flag INT
)

INSERT INTO #temp (name) VALUES 
  ('abc'), ('abc'), ('abc'),
  ('xyz'), ('xyz'), ('xyz')

DECLARE @updatecount INT = 1

WHILE @updatecount <> 0
BEGIN
  BEGIN TRAN
    SET ROWCOUNT 2
    UPDATE #temp
    SET flag = 1
    WHERE name = 'abc' AND flag IS NULL
    SET @updatecount = @@ROWCOUNT
    SET ROWCOUNT 0
  COMMIT TRAN
  CHECKPOINT -- Frees the transaction log space.
END

Batching Deletes

DECLARE @deletecount INT = 1

WHILE @deletecount <> 0
BEGIN
  BEGIN TRAN
    SET ROWCOUNT 2
    DELETE FROM #temp WHERE flag = 1
    SET @deletecount = @@ROWCOUNT
    SET ROWCOUNT 0
  COMMIT TRAN
  CHECKPOINT -- Frees the transaction log space.
END

Conclusion

By implementing transaction batching, we can efficiently manage transaction log space and mitigate the risks associated with large-scale DML operations. This approach optimizes resource utilization and reduces the likelihood of extensive rollbacks, ensuring smoother database operations.

Pro tips:
1. Learn how to find and manage deadlocks in SQL Server.

See more

The post Transaction Batching in SQL Server appeared first on AzureOps.

How to create a DACPAC file.

To create a DACPAC file from an existing SQL Server or Azure SQL database, you must ensure certain things. Refer to this post to learn how to create a DACPAC file.

Follow the below steps to open DACPAC file in Visual Studio.

Pre-requisites:
1. Visual Studio with SSDT extension installed.

1. Create a new blank project in Visual Studio

1. Open Visual Studio, click on the File menu item, and Select ‘New’ – > ‘Project’ as shown in the below image.

2. Select the project type as ‘SQL Server Database Project’. Provide the name to the project and Click OK.

This should create a blank database project in Visual Studio.

2. Import DACPAC file into the database project

1. Right-click on the project we just created in the previous step. Click ‘Import’ – > ‘Data tier Application (*.dacpac)’ option.

2. Browse the .dacpac file you want to unpack in the Visual Studio project. Click Start to begin the import process.

3. if there are no errors, you will see the progress shown in the image below.

4. You should see all database objects from the DACPAC under the project.

Pro tips:
1. It is advisable to always review the content of a DACPAC file before publishing it to a SQL Server or Azure SQL database.
2. If Visual Studio is not installed on your system and you wish to access the contents of a DACPAC file, you can do so by renaming the file with a .zip extension. This will enable you to extract the contents of the file and directly view the database objects in File Explorer.

See more

The post Open DACPAC File in Visual Studio appeared first on AzureOps.