End-to-End Correlation using Logic Apps
By Anshul
- 4 minutes read - 789 wordsHey Everyone! In this post, I am going to quickly show you a couple of ways to implement correlation between different Logic Apps you might be using in your Integration.
Why do we need Correlation?
Whenever you are working on an integration scenario, you would always try to keep the services involved as decoupled. So when you have multiple services running by passing data to/from each other, you need to make sure those runs are interconnected in some way. How that helps is when at a later point of time, if something fails and it’s your responsibility to figure out which data exactly got failed and need to be reprocessed, you would be able to get this information via correlation which is nothing but an extra field (as metadata) to define the connection between the runs of different services/apps for each data.
How to implement it?
Ok, so here comes the interesting part. Let’s say you are using a Logic App with HTTP trigger to process the incoming data from an external system ( let’s say D365 F&O). And in your workflow, you are calling another Logic App to isolate the data transformation or any other business logic.
This setup will surely cause some issues and head-itchings when you sit to trace the error from the historical runs using Azure Monitor, in case of a failure. Although you can dig up the huge list of runs and figure out the issue, I have got a time-saving solution for you. When querying in Log Analytics, all these runs have these fields to filter or sort by – Logic App Name, Status. Duration, RG Name, Tracking ID, Run Id, Start time, End time, and Tracked properties.
Tracking ID and Tracking properties both sound promising in terms of providing a better querying solution. But latter will need some extra configurations to be done in each logic app you are working with. So here we will talk about Tracking Id.
Generally, Logic App with HTTP trigger assigns a unique ID to this parameter every time it runs. So if you have nested Logic Apps, you can filter on the same Tracking ID (although weird and long and no straightforward way to know beforehand). This is how the Tracking Id looks for the main Logic App calling another logic app in the workflow-
Pretty interesting huh! So how about a way we can customize this long set of characters into something meaningful?
Let’s take a scenario and build our solution on top of it. So, you need to set up an integration between D365 F&O to process each new PO and for that, you can implement a Main Logic App to fetch the PO details and then call another Logic App to transform the data according to the target system with which you are connecting your child Logic App. So in this case, instead of having this long guid, it would make more sense to track something more meaningful like PO number.
Interestingly, Microsoft allows you to send your own custom tracking parameter right in the header of the incoming request to the main Logic app. The name of the header is – ‘x-ms-client-tracking-id’. Now, you would ask, how will this property propagate to the child logic apps. Well, Azure automatically adds this header to the subsequent requests to those child apps. Hence, after adding this property, if you go and check the runs in Azure monitor you would see something like this-
In case you couldn’t figure out how to add this property, see the screenshot below of the POST request in Postman
Isn’t this easy? Well, pretty much. But things get complicated when we have to use other services to pass on the message to other Logic App instead of a direct nested call. As part of a best practice, It is always recommended to use a message broker to make sure all messages are processed at least once.
So, in this integration, we might have to incorporate Service Bus to pick up the PO data from the Main logic app and send it to a subscribed Logic App. In this case. you would have to make some changes to the approach. You main logic app may look like this –
And your child logic app is subscribed to a topic/queue and process the message you passed as Content in ServiceBus action. So in order to make sure, the child app also has the same correlation Id, you need to tweak the trigger properties like this
"correlation": {
"clientTrackingId": "@{coalesce(triggerBody().Properties?.correlationId, guid())}"
}
Once done, you will be able to correlate the two Logic Apps with a unique and meaningful identifier.
So that’s it for now. I hope this has been helpful. Cheers!