Getatlas Ai6qy6oo3j

Evidently AI

Stay updated on the status of all your company's support tickets.

Customer portal

While Evidently AI provides intuitive visualizations and basic statistical insights, it can be further extended to perform deeper statistical analysis for a more comprehensive understanding of data drift, model performance changes, and trends.  

Advanced Statistical Analysis with Evidently AI:

   Utilize the Evidently AI API to access the raw data and statistical calculations from the reports.  This gives you greater control over the analysis and the ability to implement custom statistical methods.

  Define custom metrics using the `Metric` class to calculate more advanced statistical measures. These can range from specific statistical tests to complex combinations of calculations.  

  Use the API to implement statistical hypothesis testing to determine if observed changes in data or model performance are statistically significant or due to random fluctuations. 

  Use statistical tools to identify trends in your data over time. This might involve calculating rolling averages, performing time series analysis, or fitting statistical models to identify patterns or anomalies.

  Dive deeper into the analysis of data distributions.  Use statistical tests like the Kolmogorov-Smirnov test to compare the distributions of features in the reference and current data and identify significant differences.

  Use statistical methods like Pearson or Spearman correlation to explore relationships between features in your data.  Identify strong correlations that can indicate potential issues or insights.

  Leverage the API to apply machine learning models (e.g., clustering, classification) to understand the underlying patterns and structure in your data.  This can help detect subtle data drift or identify potential issues related to model bias or fairness.

Example (Hypothese testing using the API):

Dive into techniques for deeper statistical analysis of data drift, model performance changes, and trends.

Advanced Statistical Analysis

Evidently AI provides a wide range of predefined metrics and checks to monitor your machine learning models and data, but sometimes you'll need to go beyond the standard options. This article shows you how to define custom metrics and checks to precisely address your unique requirements for data monitoring and analysis. 

  Determine what specific information you want to monitor and the metrics or checks that are not directly provided by Evidently AI's default features. 

  Familiarize yourself with the Evidently AI API.  This will allow you to create custom metrics and checks by interacting programmatically with Evidently AI's capabilities. 

Learn how to define and implement your own custom metrics and data quality checks beyond the defaults.

Custom Metrics and Checks

When monitoring your machine learning models and data, setting effective thresholds for alerts is crucial.  Static thresholds, set at fixed values, might not always be optimal for capturing nuanced changes in your data or model performance.  Evidently AI offers ways to make your monitoring more adaptive and dynamic, allowing you to set thresholds that adjust based on your data's characteristics.

  Thresholds define the boundaries for triggering an alert.  When a metric or check result exceeds a threshold, an alert is generated.  Dynamic thresholds adapt to changes in your data, making your monitoring more sensitive to real drifts.

  Leverage the Evidently AI API to customize thresholds and adjust their values dynamically within your monitoring workflows.

Explore how to set dynamic thresholds for alerts based on data patterns and model performance.

Advanced Features & Concepts

Advanced Statistical Analysis

Custom Metrics and Checks

Dynamic Thresholds and Configuration