Page 23 - MSDN Magazine, July 2017

only a per-partition or per-row function and the system takes care of distributing the data and the tasks in the most efficient manner within given resource constraints.
Training a good model requires a lot of data and machine learning expertise—both of which are rare commodities. To help, U-SQL packages many of the machine learning models that power the Microsoft Cognitive Services. Built by some of the leading minds in the industry, these models are trained against massive data sets, and are highly performant and accurate. This integration of the cognitive models in U-SQL lets you easily add intelligent features—such as emotion detection, face and speech recognition; language understanding and sentiment analysis—to applications that work on massive amounts of data.
In this article, we’ll concentrate on how to use these pre-trained models to build intelligent applications using U-SQL, using either a pre-trained model or a built-in cognitive model.
Cognition with U-SQL
U-SQL provides built-in support for the following cognitive models, allowing you to build applications with powerful algo- rithms using just a few lines of code:
Face detects one or more human faces in an image, along with face attributes that contain machine learning-based predictions based on features such as age, emotion and gender.
Emotion analyzes facial expressions in an image to detect a range of emotions, currently including anger, contempt, disgust, fear, happiness, neutrality, sadness and surprise.
Image tagging returns information about visual content found in an image. It can be used along with descriptions and domain-specific models to identify content in an image.
Optical character recognition (OCR) detects and extracts handwritten text from images of notes, letters, whiteboards and so forth, and returns a machine-readable character stream.
Sentiment analysis detects sentiment using classification tech- niques based on the input text.
Key phrases extraction identifies key phrases, topics and language from the input text.
Landmark detection finds landmarks in an image. This model rec- ognizes 9,000 natural and man-made landmarks from around the world.
Cognition at Scale with U-SQL
Suppose you want to find out if the human population, in general, is happy when there are animals around them. One way to do this is to examine pictures posted by people who have animals with them and analyze the emotion of those people. In Microsoft, the set of Bing-crawled images would represent a valid dataset for this task, but the sheer scale of the data set would make this simple task cumbersome without an intelligent data-processing engine. Let’s see how U-SQL makes this easy. First, however, you’ll have to manually enable cognitive capabilities in your Azure Data Lake account.
Registering Cognitive Capabilities in U-SQL To get started with the Python, R, and Cognitive extensions, open your Data Lake Analytics account in the Azure Portal and click on Sample Scripts.
If you haven’t installed them already, you’ll see a notification at the top of the Sample Scripts blade for U-SQL Advanced Analytics. msdnmagazine.com
Figure 1 Cognition at Scale Example
REFERENCE ASSEMBLY ImageCommon; REFERENCE ASSEMBLY ImageEmotion; REFERENCE ASSEMBLY ImageTagging;
// Load in images @imgs =
EXTRACT FileName string, ImgData byte[]
FROM @"/usqlext/samples/cognition/{FileName:*}.jpg" USING new Cognition.Vision.ImageExtractor();
// Extract the number of objects and tags from each image @objects =
PROCESS @imgs PRODUCE FileName,
NumObjects int,
Tags string
READONLY FileName
USING new Cognition.Vision.ImageTagger();
// Extract all the images with dog and cat @pets =
SELECT FileName, T.Tag
FROM @objects
CROSS APPLY
EXPLODE(SqlArray.Create(Tags.Split(';'))) AS T(Tag)
WHERE T.Tag.ToString().Contains("dog") OR T.Tag.ToString().Contains("cat");
// Detect emotions from human face @emotions =
PROCESS @imgs
PRODUCE FileName string,
NumFaces int,
Emotion string
READONLY FileName
USING new Cognition.Vision.EmotionAnalyzer();
// Correlation to find all the images which has both human and animals @preres =
SELECT @pets.FileName, Emotion
FROM @pets
JOIN @emotions
ON @pets.FileName == @emotions.FileName;
// Distribution of human emotions when animals are in the image @result =
SELECT Emotion, COUNT(FileName) AS frequency FROM @preres
GROUP BY Emotion;
OUTPUT @result
TO @"/my/cognition/output/sample_dog_cat.csv" USING Outputters.Csv();
Click it to begin the installation process.
Once you’ve chosen to install the extensions, the system will copy U-SQL extension-related files into the default Azure Data Lake Store (ADLS) associated with your ADLA account. A notification that files are being copied will appear near the Notification icon in the upper right of the page. When the files are copied, you’ll see an updated notification that the file copying was successful and that a special U-SQL Job was submitted to finish the registration. You can find the special job and its status by using View All Jobs in the upper-left corner of the Azure Portal. The Job normally will take a few minutes to complete.
At that point, you can discover what the job did by browsing the catalog for the master database. The job simply registers advanced analytic cognitive assemblies in the master database, which you can see by using the Data Explorer.
Seeing these assemblies in the master database is evidence that your ADLA account is setup correctly and you can now write a U-SQL script that uses cognitive functions to build intelligent applications.
July 2017 19