Executing Azure Synapse Analytics Pipelines With Azure Logic Apps

Let’s create the solution for Executing Azure Synapse Analytics Pipelines With Azure Logic Apps. 

First, navigate to your Azure Logic Apps. Then, select blank canvas, or in my case, I want to trigger the Azure Logic Apps when an HTTP request is received.

I am my case; I am going to include the pipeline name as part of the HTTP request.

{

    “properties”: {

        “PipelineName”: {

            “type”: “string”

        },

        “SynapseName”: {

            “type”: “string”

        }

    },

    “type”: “object”

}

 

Secondly, select the HTTP request activity and then the HTTP action.

 

 

Configure the HTTP action as follows:

 

 

 

At this stage, the solution will execute the Azure Synapse Analytics pipeline asynchronously, so you will not know the final status of the pipeline.

 

 

The following section includes the steps for validating if the pipeline has been executed correctly.

Check Azure Synapse Analytics Pipeline Status

The following steps will help define some variables:

1. Parse response from the previous step to get the the runId (required for identifying the status of the pipeline)
   1. {
          “properties”: {
              “runId”: {
                  “type”: “string”
              }
          },
          “type”: “object”
      }
2. Initialize Variable Action: Create a variable to store the status of the pipeline
3. Initialize Variable Action: Create a variable to stop checking the status if the pipeline is taking too long
4. Condition Control Action: To validate if the pipeline has finished
   1. the @or(not(equals(variables(‘PipelineStatus’), ”)),equals(variables(‘ExitIfTakingTooLong’), 0))
5. Delay Action: modify the seconds between attempts to check status
6. HTTP Action: Check the status of the pipeline
   1. GETthe 
   2. https://@the {triggerBody()?[‘SynapseName’]}.dev.azuresynapse.net/pipelineruns/@{body(‘Parse_JSON’)?[‘runId’]}?api-version=2020-12-01
   3. Content-Type: application/json
   4. Auth Type: Managed Identity
   5. Audience: httIdentity.azuresynaIdentity
7. Parse JSON Action: Parse Status Response
   {
       “properties”: {
           “annotations”: {
               “type”: “array”
           },
           “debugRunId”: {},
           “durationInMs”: {},
           “id”: {
               “type”: “string”
           },
           “invokedBy”: {
               “properties”: {
                   “id”: {
                       “type”: “string”
                   },
                   “invokedByType”: {
                       “type”: “string”
                   },
                   “name”: {
                       “type”: “string”
                   }
               },
               “type”: “object”
           },
           “isLatest”: {
               “type”: “boolean”
           },
           “lastUpdated”: {
               “type”: “string”
           },
           “message”: {
               “type”: “string”
           },
           “parameters”: {
               “properties”: {},
               “type”: “object”
           },
           “pipelineName”: {
               “type”: “string”
           },
           “returnValue”: {},
           “runDimension”: {
               “properties”: {},
               “type”: “object”
           },
           “runEnd”: {},
           “runGroupId”: {
               “type”: “string”
           },
           “runId”: {
               “type”: “string”
           },
           “runStart”: {
               “type”: “string”
           },
           “status”: {
               “type”: “string”
           }
       },
       “type”: “object”
   }
    
8. Set Variable Action: Update status variable
9. Decrement Variable Action: reduce attempts variable
10. Check final status

Testing the Solution

To test the solution, execute by passing the correct body parameters.

 

{

    “SynapseName”: “syn-ada-dev-aue”,

   “PipelineName”: “Test”            

}

 

In Azure Synapse Analytics, you will see the pipeline being executed.

 

In Azure Logic Apps, execute the output.