Integrating with Monitoring Tools
Last updated September 6, 2024
Integrating Evidently AI with Monitoring Tools
Evidently AI provides powerful data and model monitoring capabilities, but you can enhance its effectiveness by seamlessly integrating it with your existing monitoring tools and dashboards. This integration allows you to centralize monitoring data, streamline workflows, and gain a unified view of your machine learning systems.
Integrating with Monitoring Tools:
- Choose Your Monitoring Tool: Select the monitoring tool or dashboard that best suits your needs and existing infrastructure. Some popular options include:
- Grafana: Grafana is a popular open-source platform for creating dashboards and visualizing time-series data. You can integrate Evidently AI reports into Grafana panels.
- Prometheus: A monitoring system that collects and stores metrics, often used alongside Grafana. You can integrate Evidently AI metrics into Prometheus for robust monitoring.
- Datadog: A cloud-based monitoring platform offering a wide range of features. You can integrate Evidently AI data into Datadog dashboards for centralized monitoring.
- Cloud Monitoring Services: Major cloud providers (AWS CloudWatch, Azure Monitor, Google Cloud Monitoring) offer advanced monitoring services. You can likely integrate Evidently AI reports or data into these services.
- Identify Data Sources: Determine the data sources from your Evidently AI reports that you want to integrate into your monitoring tool. This might include metrics like data drift scores, model performance metrics, or data quality statistics.
- Use the API: Leverage the Evidently AI API to extract the necessary data from your reports. You can programmatically query the report object to retrieve the required values.
- Format Data: Ensure that the data retrieved from Evidently AI is formatted in a way that's compatible with your chosen monitoring tool. This often involves converting data types, structuring the data into appropriate formats, and mapping specific metrics to their corresponding variables.
- Create Dashboards: Set up dashboards in your monitoring tool that visualize the data from Evidently AI. Use graphs, charts, tables, or other visualization methods to provide clear insights into the performance and health of your machine learning systems.
- Set up Alerts: Configure alerts in your monitoring tool to notify you of significant events. For example, you might set alerts for data drift exceeding a specific threshold, a significant drop in model performance metrics, or issues related to data quality.
- Integration Example using Grafana:
1. **Generate Evidently AI Data:** Use the Evidently AI API to generate your report data as needed, saving the data in a format that's easily readable by Grafana (e.g., CSV or JSON).
2. **Configure Grafana:** In Grafana, add a new data source for the format you've chosen to store the Evidently AI data. (Typically you might add a file data source or define a custom data source if necessary).
3. **Create Panels:** Create panels in your Grafana dashboard that query the data source. Visualize your Evidently AI data using line charts, bar charts, or other appropriate graphs.
4. **Set Alerts:** Within Grafana, configure alerts to trigger notifications when certain thresholds are met (e.g., data drift exceeds a set limit, model performance drops below a specified value).