Using Azure as an Integration Platform

This blog is extracted from the Microsoft Azure webinar on “How to use Azure as an Integration platform” presented by Rodrigo De Carvalho (Sr. Product Marketing Manager, Microsoft), Pete Roden (Program Management Architect, Microsoft) and Matthew Farmer (Senior Program Manager, Microsoft).

Introduction

Today, fundamentally all the businesses depend on Applications. These Applications are changing constantly and the kind of applications that you build is becoming more myriad diverse. Business work out the right data and the right applications to connect these two together to get the real value. Digital transformation is the buzzword today and it is all about connecting that right data to the right application. This transformation might be within your business itself or be with partner organizations. It might be also with people you don’t even know who might be able to build applications with you. Organizations are looking into how to achieve this efficiently and get a competitive advantage by doing so.

Digital transformation helps you to innovate and bring new products and services to the market quickly. Application integration is the backbone because you don’t have to completely build a new application each time. Leveraging the existing assets and be able to move quickly, innovate and build integrations makes complex applications a reality. A quote from Gartner says, about 65% of the large enterprises in a few years’ time will have created an integration platform. Enterprises will get competitive advantage rather than thinking traditionally about integration and build applications as separate assets each time.

Integration Scenarios

There’s a bunch of different integration scenarios that people are using to transform data. The classic one that we use to move the data between applications is Application to Application and Business to Business integration. The B2B integration takes information out of your business applications and quickly route it into the applications of your business partners (customers, suppliers, banks, logistics providers).

SaaS is another scenario where licensing is based on a subscription basis and the data is centrally hosted. The number of Software as a Service (SaaS) applications deployed by businesses has grown exponentially, creating various new integration use cases.

And, finally, IoT is a big change in the way we think about technology. In the world of digital transformation, how we see applications and the data has changed, IoT is absolutely the vanguard of this change. These days you get more and more data from applications, but, when you get that data what do you do with it? You want the data to be able to give it to other people you want to aggregate it. Therefore, you want to be able to integrate it with other systems to make the data meaningful.

Integration Challenges

When you want to build applications that integrate data and talk to different systems, these are the basic things you need to consider.

• What interfaces and API’s are you going to use?
• The standards that you are going to approach?
• What data sources are you going to have and the formats they are going to communicate?
• How are you going to create your applications? There are a bunch of different patterns you could use to build these applications.
• And then where are you going to host and run this?

There are 4 key components that are core parts to build an integration platform. So, let us look at the individual parts of the Azure Integration services and what they actually do.

1. API Management
2. Azure Logic Apps
3. Azure Service Bus
4. Azure Event Grid

Azure API Management

Azure API Management allows you to publish API’s safely and connect to backend systems hosted anywhere. Businesses everywhere are looking to extend their operations as a digital platform, creating new channels, finding new customers and driving deeper engagement with existing ones. You can use Azure API Management to take any backend and launch a full-fledged API program based on it.

• Work with any host, API, and scale
• Attract more developers
• Secure and optimize your API’s
• Gain Insights into your API’s with a whole bunch of metrics that you can use to determine how well your API’s are performing

Azure Logic Apps

It allows you to easily automate workflows and orchestrate business processes. Logic Apps simplifies how you design and build scalable solutions for app integration, data integration, system integration, enterprise application integration (EAI), and business-to-business (B2B) communication, whether in the cloud, on-premises, or both. With Azure Logic apps you get:

• Out-of-the-box connectors to reduce integration challenges
• Connect and integrate data from the cloud to on-premises
• B2B and enterprise messaging in the cloud
• Powerful web-based workflow designer

Azure Service Bus

Azure Service Bus is a reliable and scalable enterprise messaging solution. Data is transferred between different applications and services using messages. A message is in binary format, which can contain JSON, XML, or just text.

• Azure Service Bus simplifies enterprise cloud messaging
• Build reliable, scalable cloud solutions
• Implement complex messaging routing when you build enterprise messaging platform

Azure Event Grid

Azure Event Grid simply gives you a really simple event-based pub-sub model. Building applications with an event-driven pattern allow you to only react and do stuff when things happen and make it easy for others to program interfaces on the back of those events.

• A simple HTTP-based event delivery
• Build better and more reliable applications through reactive programming — It also allows you to kick off some of this orchestration that you are building inside a Logic Apps
• Focus on product innovation — Because this is entirely a managed solution you’re getting to focus on product innovation and not having to spend a lot of time on other non-core activities.

So in Microsoft Azure, we have four technologies that embody those particular services and they are Azure API Management, Logic Apps, Service Bus and Event Grid. And together we bring those under the banner of Azure Integration Services. It makes it easy to connect across boundaries to partners, to other businesses create relationships, create new business opportunities and all this in quick time to market.

Sample Architecture

Here is an example of how you can use the combination of all the services to achieve a given integration goal. Let us consider, you can have a mobile or web application that are connecting to on-prem API through the API management. This will allow you to securely connect to those systems and carry the processes throughout the orchestrations. In this architecture, components such as API Management, Logic Apps, the on-prem data gateway allows you to connect to your systems that are running within your data center and also SaaS provided by the third-party organizations serving a business need.

The idea is to present how potentially you can leverage those services and to satisfy any enterprise-scale integration. Patagonia, a Microsoft customer has been leveraging these services to build innovative solutions and improve their customer experience. By using Azure as an integration platform they were able to gain significant productivity in confirming/delivering the products to their customers.

Demo

Let us consider an order processing scenario, Fabricon uses core components like Azure API Management, Service Bus, Logic Apps and Event Grid to complete the order processing system. The process starts with the API, the call is made through API Management to the Logic Apps. In turn, the order is created in Dynamics and then acknowledge it back to the customer with the “Thank you” message. At the same time, the Logic Apps triggers a message to the Service Bus Topic. At this stage, the Topic actually publishes the messages to the Event Grid. Letting the Event Grid know that the message has arrived. Then the Event Grid reacts to the message appearing and triggering the Logic App to perform the process of orchestrating the call to different systems to complete the order.

Receive Orders is the Logic App that effectively receives the order and does all the steps in order to make that order a reality. The Logic App is designed to orchestrate the steps that need to happen.

As a first step, the Logic Apps parses the message to make sure they come from the correct schema, then the message is sent to a number of different systems. Post this the acknowledgment is sent back to the customer with the “Thank you” message.

The Process Order is another Logic App that acknowledges the customer order and puts the message on the queue for further processing.

This Logic App has an HTTP trigger and a response, it is set as an HTTP API operation. Similar to the other Logic App, you have the schema validation to check the message in the Logic App. The next logic is to create the order record in the Dynamics.

The logic continues to compose further messaging where you can add a Property into the JSON Message. The message is then flown to the Topic with the Order ID.

Finally, all these operations are exposed in API Management which allows securing this Logic App as an API. Apply settings like rate limiting, security policies, transformations and change the version of the API if needed.

The API Management developer portal comes out of the box with the API Management, this helps the developers to test the application.

Interesting Q&A’s

Any reason why some of the Azure services such as Azure Data Factory, Event Hubs, IoT Hub are not there as part of Azure Integration Services?

Those services are all complementary and can absolutely be used in conjunction with Azure Integration Services. For now, we wanted to simplify the pieces that we see customers needing to build a platform for enterprise integration.

Do you require Service Bus to send messages to Event Grid?

No, Event Grid supports a number of Azure services. You can even create custom events from your own applications. We recommend using it with Service Bus to reduce the need for queue polling.

Is it possible to track all the steps an request took thought the components of this solution from a single place inside Azure?

You can use Azure Log Analytics to trace events across services

What sort of security option is there for Messages?

Service Bus messages are secure with a SAS token. Depending on the scope of the solution you can reach to Microsoft partner or representative to discuss your requirements.

Can we author our own private connectors for Logic Apps consumption?

Yes, you can create custom connectors that you can reuse within your organization.

What factors need to be considered to take the decision to select Flow vs Logic Apps?

Flow is geared towards individual users, whereas Logic Apps is designed for enterprise workflow scenarios. For instance, each Flow is tied to a specific user instead of the organization as a whole.

What kind of centralized overview can I get if I start building many logic apps that might integrate with each other via queues or in other ways

Logic Apps has support for Log Analytics to get the information in detail. We are also looking at how we can provide a unified view across Azure Integration Services.

What process would I have to go through in order to create a connector?

Check this page — https://docs.microsoft.com/en-us/azure/connectors/apis-list this might help. You can also write to [email protected] for support.

When to choose Microsoft Flow and Logic Apps? They seem like very similar technologies?

They are based on the same underlying engine. Flow is a technology for business users to build simple automation. Logic Apps is a developer/integration specialist technology (with support for versions, IDEs, CI/CD etc). If you are building a service/application, you should use Logic Apps.

Similar to the above question, we at Serverless360 have noticed many people asking “When to use Azure Logic Apps and Functions?” If you have the same question nudging in your mind, read this blog by Steef-Jan Wiggers on When to use Azure Logic Apps and Functions.

You can watch this session “How to use Azure as an Integration platform” on demand by registering here.

Serverless360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Serverless360 free for 30 days!

Originally published at www.serverless360.com on September 19, 2018.