CRM and Azure Service Bus Integration Part 3

This is the third part of the series. Please check the last blog first, if you haven’t already.In this blog we go through a step by step tutorial to setup a CRM-Azure integration using service bus queue via SAS(Share Access Signature).

Prerequisite

• A CRM Online instance
• Microsoft Azure account
• Microsoft Azure SDK
• Visual Studio 2013/2015
• CRM2016 SDK V8.1 or later ( We will be using the plugin registration tool and sample code from the SDK 8.1 as the SAS featured is added to CRM2016 Update 1)

Setting up a Service Bus Namespace and Queue

For this post we will using the same Service Bus namespace we have created in the last post http://mscrmshop.blogspot.com.au/2016/06/crm-and-azure-service-bus-integration_20.html. We will add a new queue to the existing Service Bus namespace.

1. Login to Azure Portal.
2. Select the Service Bus namespace. Click the “Queues” tab and select “Create A New Queue”
   
   
3. Enter the values as shown in the following screenshot and click “Create A New Queue”
   
4. A new queue will be created as shown in the screenshot.
   
5. Double click on the queue name and select “Configure”. Change the general settings as required. I am using the default settings. Create a new policy to mange the access to the queue.
   
   
   
   
6. Click on “DASHBOARD” and select “View Connection String” as shown in the screenshot.
   
7. It will display the "access connection information" as shown in the following screenshot. Copy the connection string. It will be used in the next step.
   
   

Setting up the CRM Service Endpoint

Here are the steps

1. Start the plugin registration tool. It is available in the \SDK\Tools location of the SDK. I am using SDK v8.1
2. Click on Register>>Register New Service Endpoint
    
3. It will prompt for the connection string. Paste the connection string for the queue here and click “Next”
   
4. The registration tool will display the following page. It will prefill most of the values. I have updated the message format to JSON for this post. Click “Save” to save the service endpoint.
   
5. Select the service endpoint and select "Register New Step", as shown in the screenshot.
   
   
6. For this blog post I am using create message on account entity. You can chooses the other messages and entities as required. Click “Register New Step” to complete the registration.
   

Test the Integration

Queue integration does not require an active listener. So just create a new account from the front end. It should post a message to the queue. You can check status of the integration in the CRM system jobs. view.
Also, you can login to azure. Navigate to the queue and check the queue length. This column displays the number messages received by the queue.

Reading from the queue

Microsoft has provided the Azure sample. For this post, I am using the Azure sample code that comes with the CRM SDK. Here are the steps

1. Open the “PersistentQueueListener” project from the location  \SDK\SampleCode\CS\Azure\PersistentQueueListener.
2. Resolve the missing reference as done in the last blog post (http://mscrmshop.blogspot.com.au/2016/06/crm-and-azure-service-bus-integration_20.html)
3. Run the code. Provide the values for service namespace, issuer name and issuer secret.The code will read the message from the queue and display its contents.
   Note: The CRM sample code is using the ACS issuer name and issuer secret to generate SAS token. So please provide the ACS values as explained in the last blog.
   

CRM and Azure Service Bus Integration Part 2

This is the second part of the series. We have discussed the contract types, security and a general overview of CRM–Azure Service Bus integration in the previous blog. Please read part 1 for the context.
In this blog we go through a step by step tutorial to setup a CRM-Azure Service Bus one way listener contract using ACS.

Prerequisite

• A CRM Online instance
• Microsoft Azure account
• Microsoft Azure SDK
• Visual Studio 2013/2015
• CRM2016 SDK V8.0.1 or less ( We will be using the plugin registration tool and sample code from the SDK. The plugin registration tool does not have ACS options from SDK v8.1 onwards)

Here are the steps to create the integration.

Setting up a Service Bus Namespace

The first thing you need for the integration is a Service Bus namespace. If you already have an ACS namespace, you can skip this step.
In the past you could login to the Azure portal and create an ACS namespace. But now, Azure portal does not allow the creation of ACS namespaces from the portal. It only creates a SAS namespace by default. We need to use PowerShell to create an ACS namespace.

1. Open “Windows PowerShell ISE”
2. Type Add-AzureAccount. It will open the login dialog. Enter the username and password for your Azure account and press login.If everything goes smoothly, the PowerShell output will look like the following screenshot.
   
   
3. Type the following command below to create a Service Bus namespace. Replace ‘MSCRMShopBus’ with the namespace you want and press enter
   
    New-AzureSBNamespace -Name "MSCRMShopBus" -Location "Australia East" -CreateACSNamespace $true  -NamespaceType Messaging. Replace the parameters values as required. 
4. The output screen will look like the following screenshot. It will create a namespaces for both ACS and SAS.
   
   
5. This step is optional. Login to your Azure account and check the connection information of the Service Bus namespace. It will look like the following screen.
   
   
   
   

Get the Certificate file and Issuer name for CRM

For CRM Online
To get the security certificate and the issuer name for CRM go to Settings>>Customizations>>Developer Resources. Download the certificate file and also notice the issuer name of the certificate.


For CRM On Premise
For CRM on premise, follow the following article for step by step instructions.
https://msdn.microsoft.com/en-us/library/gg328249.aspx

Setting up the CRM Service Endpoint

Here are the steps

1. Start the plugin registration tool. It is available in the \SDK\Tools location of the SDK. I am using SDK v8.0.1.
2. Click on Register>>Register New Service Endpoint
   
3. It will open the the following page. Enter the Values as required and press “ Save & Configure ACS”.
   
   Solution Name is the Service Bus namespace we created in the beginning of this blog.
   
   Path is the path of your listener project. For example, when you run the listener project for a one-way listener for this blog, the service endpoint URL will look like https://mscrmshopbus.servicebus.windows.net/RemoteService

. The red part of the URL represents the path.

Contract is the type of contract we are using for the service endpoint. For the difference types of contracts check the last blog.

The registration tool will display the following screen. Enter the information below and press “Configure ACS”. Management Key is the default key when you create a Service Bus namespace. The certificate file and issuer name comes from the “Get the Certificate file and Issuer name for CRM” section of the blog.

Press “Yes” on the following screen.

It will create the management service, rulegroup and sample rules for the service endpoint. Press “Close”.

Press “Save & Verify Authentication”.It will test the authentication and save the configuration of the service endpoint. Press Close to close the screen.

Press “Save” to close the service endpoint registration screen.

Add a new step to the service endpoint as shown in the screenshot below. It is exactly the same as registering the step for a plugin assembly.

For this blog, I am creating a step for on creation of a new account. Choose the step values as required or copy the values as shown in the screenshot.

Setting up a listener application

1. For this blog, I am using the Azure sample code that comes with the CRM SDK. Open the “OnewayListener” project from the location  \SDK\SampleCode\CS\Azure\OneWayListener.
2. Resolve the missing references. The 2 highlighted DLLs are available in the Azure SDK.
   
   
3. Run the sample code. It will prompt for the following information.
   
   
   Service Namespace is the Service Bus namespace.
   Issuer name is the “Default Issuer”  on the Service Bus connection information as shown in the step 5 of “Setting up a service bus namespace”. It is not the issuer name in CRM Online.
   Issuer Secret is the “Default Key” on the Service Bus connection information.
4. If everything is working properly it will display the service address as shown in the screenshot of step 3.

Testing the integration

1. The listener application should be up and running.
2. Login to CRM and create a new account. If everything is working properly, it will display the data context in the listener application.
   
3. To check the status of the jobs you can go to the “System Jobs” view.
   
4. If the listener application is not available, the status of the job will change to “Waiting for Retry”. It will keep trying to post the message and ultimately change to “failed” if it is not successful after X number of tries. The screenshot below display the details of the job with the status “waiting for retry”.
   
   

CRM and Azure Service Bus Integration Part 1

I am writing a series of blog posts to look at different ways to integrate CRM and Service Bus. Most of the contents/code I will cover in this series is available on the MSDN site and in the CRM SDK. There is a lot of information on the internet regarding CRM and Service Bus integration but it is scattered everywhere. I am trying to create a beginners guide for the integration.

Prerequisite

You have to have knowledge of CRM Plugins and Workflow Assemblies to understand integration discussed in this blog. The blog will refer to IpluginExecutionContext, which will be posted to the Service Bus as a part of the integration. In short, the IpluginExecutionContext defines the contextual information passed to a plugin at runtime. It contains the information about the runtime environment that the plug-in is executing in, information related to the execution pipeline, and entity business information.

Common Integration Scenario

The most common scenario for this integration is to post the CRM data to Service Bus, to be used by the line of business (LOB) applications. The integration uses the data context available in Plugins and custom workflow assemblies, that will be posted to the Service Bus and Microsoft Azure Service Bus “CRM aware” solutions can listen and read the data from Service Bus and integrate with the other LOB applications.

How does it work

Dynamics CRM provides the functionality to create the service end points that connect CRM and Azure Service Bus. These service endpoints are the contracts between CRM and Service Bus that defines the handling and security of the messages. Registering a service endpoint in CRM is exactly like registering a new plugin assembly. Once the service endpoints are registered, you have to register the plugin ‘step’/steps in the event execution pipeline.
Once these steps are initiated (when a record is created or updated etc.) by the user, workflow or custom application, the service endpoint notification service notifies the Asynchronous service to post the data context to Service Bus, based on your registered step. Each post is performed by the system job of the Asynchronous service and the status of each job can be checked using the System Jobs view in CRM. If the listener or endpoint is not available then the message won’t be posted to the bus. The Asynchronous bus will keep trying to post these messages to Service Bus
The following diagram from Microsoft describes the physical elements of the integration

The sequence of events are as follows:

1. A listener application is registered on a Microsoft Azure Service Bus solution endpoint and begins actively listening for the Microsoft Dynamics CRM remote execution context on Service Bus.
   
2. A user performs some operation in Microsoft Dynamics CRM that triggers the execution of the registered OOB plug-in or a custom Azure-aware plug-in. The plug-in initiates a post, through an asynchronous service system job of the current request data context to  Service Bus.
   
3. The claims posted by Microsoft Dynamics CRM are authenticated. Service Bus then relays the remote execution context to the listener. The listener processes the context information and performs some business-related tasks with that information. Service Bus notifies the asynchronous service of a successful post and sets the related system job to a completed status.

Contract Types

The service endpoints support the following types of contracts. Most of the definitions in this section come from Microsoft’s documentation.

• Queue
  A queue contract provides a message queue in the cloud. With a queue contract, a listener doesn’t have to be actively listening for messages on the endpoint. For queues, there is a destructive read and a non-destructive read. A destructive read reads an available message from the queue and the message is removed. A non-destructive read doesn’t remove a message from the queue. It does not require an active listener.
• One way
  Requires an active listener on the endpoint, otherwise the post to Service Bus fails.
• Two way
  Similar to a one way listener except it can return a string to the plugin or workflow assembly.
• REST
  Similar to two way listener but on REST endpoints.
• Topic
  Similar to queue except one or more listener can subscribe to receive a message
• Event Hub
  This contract applies to Azure’s Event Hub Solution

 

Security

Service Bus uses Shared Access Signature (SAS)  or Microsoft Azure Active Directory Control Service (ACS) for authentication and authorization.
The support for SAS authorisation is added in CRM Online 2016 Update 1 and CRM 2016 Service Pack 1 (on-premises), Before that, CRM only supported ACS for authentication and authorisation. For further information please check the following link
https://azure.microsoft.com/en-us/documentation/articles/service-bus-authentication-and-authorization/
It looks like SAS is the preferred method to work with Service Bus.
Use the SDK Plugin registration tool to configure Microsoft Azure Service Bus issuer, scope, and rules, which allow a listener application to read the Microsoft Dynamics CRM messages posted to the Microsoft Azure Service Bus.
For ACS configuration use the registration tool from SDK v8.1, and for SAS configuration SDK v8.1 or later.
For a CRM Service bus integration walkthrough for one way service contract, check my next blog CRM and Azure Service Bus Integration Part 2.

Note: I am no expert on CRM and Service Bus integration. I am learning myself. Please feel free to correct me if I am writing anything wrong.

Auditing Security Roles in CRM

If you are a CRM professional, you would know about the auditing feature of CRM. There are hundreds of blogs that tell you how to enable/disable auditing in CRM. In general, we don’t think about auditing the security roles. How many times have you heard from the customer that they had access to feature x yesterday, but they can’t do it now? You have no clue if someone has updated the security role or removed access to a specific role.
Auditing of security roles can provide the answers to all those questions. You can enable the the auditing of the security roles entity by selecting the audit checkbox as shown in the following screenshot.


You can also enable the auditing for the Field Security Profile and Field Permission entities.
Now auditing is enabled and you can tell when a new security role is created. If a permission is added or updated in the security role, It will tell you when you assign this security role or remove the security role from the the user or the team record. The following screenshot displays the some of the events associated with security role auditing.

In the screenshot above, we can see:

• A ”Create” event when a new security role is created.
• A ”Add Privileges to Role” event when new privileges are added to the security role.
• A ”Replace Privileges to Role” event when the privileges are updated to the security role.
• A ”Associate Entities” event when the security role is assigned to a user/team.
• A ”Disassociate Entities” event when the security role is removed from a user/team.

This is very valuable information for the system administrator.

CRM and PowerApps

In my last blog, I wrote about CRM and Microsoft Flow. While researching for building and registering a custom API for Microsoft Flow, I came across another preview product named Microsoft PowerApps.

What is Microsoft PowerApps?

Microsoft PowerApps is a preview product that enable the users to build custom business mobile/web apps and share those apps with colleagues in no time. The user does not need coding skills to build PowerApps. If you have basic Microsoft Excel skills then you can build an app. Like Microsoft Flow, PowerApps also allows users to connect to cloud apps like SharePoint, CRM Online, Twitter, Google Drive, OneDrive, Office 365 and Custom APIs etc. Microsoft PowerApps and Microsoft Flow work very well together. Both have the same set of prebuilt connections. You can even use Microsoft Flow within PowerApps.
Check the following link to know more about PowerApps.
https://powerapps.microsoft.com

Creating a sample app using CRM Online connection

PowerApps allow you create an app using:

• Existing data sources like CRM Online,  Salesforce and Twitter etc.
• Pre-built templates
• A start from scratch

For this blog, I am creating a PowerApp using a CRMOnline Connection. Here are the steps.

1. Signup for PowerApps and install the PowerApps app.
2. In PowerApps, Select New  >> “Create an app from your data” >> Dynamics CRMOnline, as shown in the screenshot below.
   
3. Create a new connection to your CRM Online organisation. In my case, I already have connections, as shown in the screenshot. Click on HPE (or the dataset for your organisation).
   
4. Select Contacts from the dropdown and click Connect, as shown in the screenshot.
   
5. PowerApps will create an app. The app will have 3 Screens:
   • BrowseScreen1(Home Page with a list of the contacts and a search box)
   • DetailScreen1 (Displays the details of the selected item on BrowseScreen1)
   • EditScreen1(Opens the selected record/new record in edit mode)
6. On BrowseScreen1, select the gallery as shown in the screenshot and change the items property to as shown below. It will allow the user to search for a name in Full name in CRM and sort the list based on the last name.
   
   
7. Now, select the fields in the first item of the gallery and choose the CRM field you would like to display in that control. Please have a look at the screenshot below.
   
8. Select DetailScreen1. Select any field on the Details screen. It will display all the fields on the screen in the options pane as shown in the screenshot. Click on the purple eye in the options pane to add or remove the fields on the form.
   
9. Select EditScreen1. Use the options pane to add and remove the fields on EditScreen1 as explained in step 8 above.
10. For this app, I wanted to add a camera feature. Click on New Screen. Rename the screen to CameraScreen1.
    
    
    
11. Add the camera control to CameraScreen1, as shown in the screenshot. Add the camera control to the screen, as shown in the screenshot.
    
    
12. Rename the camera control to MyCamera.
13. Set the OnSelect property of MyCamera to Back().
    
    
    
14. Select EditScreen1 and select the entityimage field from the options pane.
    
15. Select the menu represented by 3 dots to go to the Advanced options.
    
    
16. Change the ImageX.Image property to If((photos=true), MyCamera.Photo,Parent.Default), as shown in the screenshot.
    
    This will the set the image property of the control to MyCamera.Photo when the value of the photos variable is true, else leave the value to default.
17. Also set the visibility property of the entityimage control to If((photos=true),true,false).
    
    It will display the entity image only when the value of photos variable is true.
18. Now add the camera icon next to the save button on EditScreen1 as shown.
    
19. Set the OnSelect property of the icon to Navigate(CameraScreen1,ScreenTransition.Cover);UpdateContext({photos:true}). When the user clicks on the camera icon it will take the user to the camera screen and set the value of the photo’s variable to true. The picture will be displayed in the entityimage field added in step 14.
20. Select DetailScreen1. Change the OnSelect property of the edit button to Navigate(EditScreen1, ScreenTransition.None,{photos:false}). On select of the edit button, the screen will navigate to EditScreen1 and set the value of the photo’s variable to false. This means EditScreen1 will not display the entityimage control unless the camera button is clicked.
21. Save your app. It will add your app to PowerApps and be available to run on your phone and computer.

Testing the App

• To test this app on iPhone or android, you will need to install the Powerapps app on your phone.
• When you start the app, it will display the list of contacts. The user can search for a contact by typing the name in the search box. Click the right arrow to go to the details screen.
  
  
• Click on the edit icon to edit the record.
  
• The user can edit the details by changing the values on the edit screen. Click on the camera to add a picture.
  
• Click anywhere on the camera to take the picture.
  
  
• The app will take you back to the editscreen1 and display the picture as shown in the screenshot.
  
  
• The app will save the record in CRM. Open the record in CRM and check if the image is updated.

A few things that did not work for me

I am very new to Powerapps. There are a few things I have noticed that does not work well.

• For the CRM Online connection, If the record has a lookup field, the record will return the guid and object type, but not the name field of the lookup record.
• Could not figure out how to display the CRM entity image in the app
• The Add Picture control does not display the camera roll on iPhone. It will always display the iCloud option to choose a picture.