Azure ML Online Endpoint Performance Monitoring Metrics

AI-300 By islever Editorial Last updated

Deploy machine learning models for production environments Monitor and maintain machine learning models in production

Short answer

BCE Requests per minute, Connections active, and Request latency are the three metrics to monitor for identifying performance degradation in an Azure ML online endpoint.

Question

A team manages an Azure Machine Learning workspace and deploys a model to an endpoint.
A deployed online endpoint shows inconsistent response times during periods of high traffic.
You need to identify potential performance degradation.
Which three metrics should you monitor? Each correct answer presents part of the solution. Choose three.
NOTE: Each correct selection is worth one point.
  1. Feature count
  2. Requests per minute Correct Answer
  3. Connections active Correct Answer
  4. Dataset size
  5. Request latency Correct Answer

Key Takeaways

The main trap is confusing model attributes like feature count with runtime performance metrics such as latency and throughput.

Identify the correct metrics for monitoring Azure Machine Learning online endpoints during high traffic. The solution involves Requests per minute, Connections active, and Request latency.

Common mistake: Selecting Feature count (A) or Dataset size (D) is incorrect because they describe the model's data structure, not its real-time performance or health.

Expert Analysis

Endpoint Performance Metrics

To identify performance degradation in an Azure Machine Learning online endpoint under high traffic, you must monitor metrics that reflect the system's operational state and responsiveness. Request latency (E) is critical because it measures the time taken to process a request; spikes here directly indicate slowdowns. Requests per minute (B) tracks throughput, helping to correlate load with response times. Connections active (C) indicates concurrency levels, showing how many simultaneous requests the service is handling.

Irrelevant Runtime Metrics

Options A and D are static or data-centric properties, not runtime performance indicators. Feature count (A) refers to the number of input features in the model schema, which does not change during deployment and has no direct bearing on real-time endpoint health. Dataset size (D) relates to the training data volume, which is irrelevant once the model is deployed and serving predictions.

Operational Consequences

Monitoring B, C, and E allows for proactive scaling decisions. If latency increases while connections remain stable, the issue may be computational. If both rise, the endpoint may need more instances. Failing to monitor these can lead to undetected service degradation, impacting downstream applications relying on the ML model.

Decision Context

This question tests your ability to distinguish between model characteristics (static) and endpoint performance (dynamic). The key trap is selecting metrics related to the model's architecture rather than its hosting environment. In production, observability focuses on the 'how' (speed, load) rather than the 'what' (data shape).

Comments support BCE, noting that feature count and dataset size are not endpoint runtime metrics. This aligns with Azure Monitor capabilities for ML endpoints, which expose latency, throughput, and connection counts.

Exam Strategy

When asked about endpoint performance, focus on dynamic runtime metrics: latency, throughput, and concurrency. Avoid static attributes like feature count, model version, or dataset size unless specifically asked about model configuration. Use Azure Monitor dashboards to visualize these metrics in real-time.

Frequently Asked Questions

Why is feature count not a performance metric?

Feature count is a static attribute of the model's input schema. It does not change during inference and does not reflect the endpoint's processing speed or resource utilization.

How do Connections active and Requests per minute differ?

Connections active measures concurrent sessions at any given moment, indicating immediate load. Requests per minute measures the rate of incoming requests over time, indicating overall throughput trends.

More AI-300 FAQ →

Community Votes

B
34%
C
33%
E
33%

34% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 2 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 2

  1. RICHARDALEX007 👍 2
    Performance degradation on an online endpoint under load is surfaced by traffic (requests per minute), concurrency (active connections) and request latency. Feature count (A) and dataset size (D) are not endpoint runtime performance metrics.
  2. exam_upskill 👍 1
    B. Requests per minute ✅ C. Connections active ✅ E. Request latency ✅ B + C + E

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Evaluating RAG Answer Quality with Measurable Techniques

S-grade

Azure ML Asset Registration for Reusability

S-grade

Assessing and Improving RAG Retrieval Quality Sequence

S-grade

Implementing RAG Flow with Microsoft Foundry and Azure OpenAI

S-grade

← Back to AI-300 Study Guide