In the previous post, we discussed about streaming units in Azure Stream Analytics. In today’s post, we will learn about allocating the optimum number of streaming units to Stream Analytics jobs.
The optimum number of streaming units to be allocated depends on two factors:
- Partition configuration for the inputs
- The query used to process events
Streaming units can be allocated on the Scale page in the azure portal. It is recommended, to start with 6 Streaming Units for queries that do not use PARTITION BY clause. The best method to find out the correct number of streaming units is trial and error. This is done by performing a test data load with the equivalent amount of data and monitoring the SU% utilization metric. Allowed number of SUs that can be allocated per job are 1,3,6 or multiples of 6.
There are upper limits to the number of SUs that can be allocated to a job, depending on the number of steps in the query and the number of partitions for each job. There is also an upper limit of 500 SUs at the subscription level that can be allocated to jobs. If there is a need to increase the number of SUs beyond 500, customers can contact Microsoft support.
Reference: https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-streaming-unit-consumption
Azure Data Engineering 