Azure Stream Analytics Job States

In this post, we are going to discuss about the various states an Azure Stream Analytics job can be in. There are four states for an Azure Stream Analytics Job: 1. Running: This means that the Stream Analytics job is working as intended i.e. reading incoming event data from the inputs, processing the data andContinue reading “Azure Stream Analytics Job States”

Azure Stream Analytics Inputs

Azure Stream Analytics is a real time, big data, events processing and analytics engine. Stream Analytics jobs can accept multiple inputs and are highly integrated with other Azure services. Azure Stream Analytics natively supports inputs from Azure Event Hubs, Azure IoT hub and Azure Blob Storage. Stream Analytics data inputs can be of two types:Continue reading “Azure Stream Analytics Inputs”

ADE Q05: Azure Cosmos DB, Data Factory and Azure Storage Quiz

1. Which Azure Data Factory Data Flow Partitioning Scheme should be used when no good partition key candidates are available in the data? A. Round Robin B. Dynamic Range C. Hash D. Key 2. What is the primary key for data stored in Azure Table Storage? A. PartitionKey B. RowKey C. PartitionKey and RowKey D.Continue reading “ADE Q05: Azure Cosmos DB, Data Factory and Azure Storage Quiz”

Azure Cosmos DB Synthetic Partition Key

In previous post, we discussed about querying Cosmos DB containers using partition keys. It is especially important to select a good partition key that distributes the data evenly across multiple partitions. However, if there is no suitable column with properly distributed values, we can create a Synthetic Partition Key. There are three ways to createContinue reading “Azure Cosmos DB Synthetic Partition Key”

Azure Cosmos DB Container Queries

In previous post, we have discussed about partitions. The purpose of creating partitions is to make the data retrieval process more effective. Designing partition wisely, by choosing an effective partition key is only one part of getting optimum data retrieval from Cosmos DB. The other part is to write queries that use the partitions effectively.Continue reading “Azure Cosmos DB Container Queries”

Azure Cosmos DB Partition Sets

We discussed replica-sets in previous post. In today’s post, we are going to discuss Partition sets.  Partition sets are geographically distributed structures that are used to manage physical partitions using a set of keys. Unlike, replica sets, which are limited within a cluster, partition sets are scoped to scale the whole region which might spanContinue reading “Azure Cosmos DB Partition Sets”

Passed the DP-201: Designing an Azure Data Solution Exam

I have some good news today, I passed the DP-201: Designing an Azure Data Solution exam. Along with DP-200, this exam is a requirement towards Microsoft’s Azure Data Engineer Associate certification, which means that I am officially a Microsoft certified, Azure Data Engineer now! The exam was of moderate difficulty and builds on the conceptsContinue reading “Passed the DP-201: Designing an Azure Data Solution Exam”

Azure Cosmos DB Replica Sets

In the previous post, we discussed about global data distribution in Cosmos DB. In this post, we are going to do a deep dive into replica sets. Replica sets are a way to implement physical partitions. Replicas are dynamically placed and spread across the machines within a cluster for load balancing. Replicas can also beContinue reading “Azure Cosmos DB Replica Sets”