Dynamic Thresholds and Configuration
Last updated September 6, 2024
When monitoring your machine learning models and data, setting effective thresholds for alerts is crucial. Static thresholds, set at fixed values, might not always be optimal for capturing nuanced changes in your data or model performance. Evidently AI offers ways to make your monitoring more adaptive and dynamic, allowing you to set thresholds that adjust based on your data's characteristics.
Dynamic Thresholds and Configuration:
- Understand Thresholds: Thresholds define the boundaries for triggering an alert. When a metric or check result exceeds a threshold, an alert is generated. Dynamic thresholds adapt to changes in your data, making your monitoring more sensitive to real drifts.
- Use the API: Leverage the Evidently AI API to customize thresholds and adjust their values dynamically within your monitoring workflows.
- Data-Driven Thresholds:
- Calculate Percentiles: Use statistical techniques to dynamically calculate percentiles based on your data. For example, set the threshold for data drift as the 95th percentile of the drift score distribution, meaning that an alert is triggered only when the drift score exceeds the 95th percentile.
- Configure and Adjust:
- Dynamically Update Settings: Use the API to update your thresholds during runtime, based on your data and monitoring requirements.
- Dynamically Adjust Parameters: Define and adjust parameters used in calculating dynamic thresholds. This could include modifying window sizes for moving averages, selecting different percentiles, or adjusting statistical significance levels.
- Example (Using a rolling average):
Was this article helpful?