Page 43 - MSDN Magazine, June 2018

you can see the record count, but we’ve split it so that this is reported by Event Name, giving us a graph and data for the record count for different events. Here we’re comparing record counts taken when the source data was read to record counts taken later when consolidated data was loaded into a DataFrame. This is an important feature since Record Count might be a pretty common measurement across our parent operation, but we’d like to see it at each operation or event.
If you see something in one of the operational graphs that calls for some research, you can search through all the telemetry. Figure 8 depicts the results of a search and a graph showing an occurrence count over time in the left pane. In the right pane you can look at all of the information recorded in the event. Recall the track_event([name], [properties], [measurements]) signature. We’ve pulled up the detail of a Persist to SQL DB event in which you can see the custom properties at the top. In the middle, labeled Custom Data, is where you can find the custom measurements that were sent with the telemetry. At the bottom right are all of the Related Items where you can easily navigate to all of the events that belong to the operation or parent operation. In addition, there’s a line at the bottom to see all available telemetry at the time of the event. If you’ve standardized on Application Insights for your runtime
Figure 9 Setting Up an Alert on Metric msdnmagazine.com
monitoring, this is a great tool for understanding the overall system state and the operational context of an event. Having insight into what’s going on broadly might help explain when record counts are off or duration is askew.
The last thing we want to cover for Application Insights is the ability to set up an alert. In Figure 9 you can see part of the alert configuration. Like the other elements we looked at, the custom information we sent in the events shows up here for us to choose as criteria for alerting.
Having insight into what’s going on broadly might help explain when record counts are off or duration is askew.
As you might expect, the alert can send an e-mail. However, it can also call a WebHook, which makes for a nice and easy way to take any other action you might desire. Azure Functions is a per- fect fit for this setup and will allow you to create whatever custom action you like. More interestingly, Application Insights is directly integrated with Logic Apps. This enables native capability to inte- grate and orchestrate actions across a wide variety of integrations and Microsoft Azure. Thus, an Application Insights alert could notify people while starting to take compensating and/or correc- tive actions via Logic Apps orchestration, including actions and integrations with downstream and upstream systems.
In Closing
We want to make sure we highlight the key bits of information. Application Insights is not a Log Analytics solution. It integrates with Azure Log Analytics, which provides post hoc analysis and long-term log retention. Application Insights is for monitoring and analytics of your runtime operations, giving you information, insights and alerts about what’s happening now. Direct integration with other Azure services and broad availability of platform SDKs makes it a nice fit to help operationalize your Azure Databricks jobs. As a result, monitoring for those jobs isn’t done in a silo, but rather within the context of the full solution architecture. n
Joseph Fultz is a cloud solution architect at Microsoft. He works with Microsoft customers developing architectures for solving business problems leveraging Microsoft Azure. Formerly, Fultz was responsible for the development and architecture of GM’s car-sharing program (mavendrive.com). Contact him on Twitter: @JosephRFultz or via e-mail at jofultz@microsoft.com.
Ryan MuRphy is a solution architect living in Saint Louis, Mo. He’s been building and innovating with data for nearly 20 years, including extensive work in the gam- ing and agriculture industries. Currently, Murphy is helping some of the world’s largest organizations modernize their business with data solutions powered by Microsoft Azure Cloud. Follow him on Twitter: @murphrp.
thanks to the following Microsoft technical expert for reviewing this article: Darren Brust
June 2018 37