Abstract

The data-centric AI framing shifts investment from model architecture search to systematic improvement of training data quality. By late 2025 the tooling ecosystem has matured enough to make this operationally practical. Libraries for label error detection, dataset versioning, slice-based evaluation, and influence function analysis integrate into CI/CD pipelines much as linters do for code. The dataset is now treated as a living artifact with its own iteration lifecycle, not a one-time preprocessing step.

Label Error Detection with Cleanlab

Cleanlab identifies likely label errors using confident learning: it estimates the joint distribution between observed noisy labels and true latent labels by cross-referencing classifier predicted probabilities against the label matrix. The workflow: train any probabilistic classifier via cross-validation to get out-of-sample predicted probabilities, then pass them with labels to cleanlab.filter.find_label_issues. The output is a ranked list of likely mislabeled samples. On datasets with 5-15% error rates (common in crowd-sourced annotation), Cleanlab surfaces 70-90% of true errors in the top 5% of flagged samples.

Dataset Versioning with DVC

Data Version Control (DVC) extends Git to track large dataset artifacts via content-addressed storage on S3, GCS, or Azure Blob. A dvc.yaml pipeline links preprocessing scripts to their inputs and outputs; dvc repro reruns only stages invalidated by upstream changes. Annotation platforms such as Labelbox and Scale AI export versioned snapshots with embedded provenance metadata. Combining DVC pointers with a dataset card documenting class distribution, known biases, and intended use gives teams the traceability needed for training regression diagnosis and regulatory audits.

Slice-Based Evaluation and Influence Analysis

Aggregate accuracy hides systematic failures on underrepresented subpopulations. Slice-based evaluation computes metrics separately for subsets defined by metadata attributes or learned embedding clusters. Hugging Face Evaluate and Snorkel AI’s SliceBuilder both support this pattern. When a cluster shows accuracy 15 points below the overall average, it defines a slice worth labeling more densely.

Influence functions (Koh and Liang 2017) estimate each training example’s contribution to a given prediction. For a misclassified sample, influence analysis surfaces training examples most responsible for the error - often a cluster of mislabeled or distribution-shifted records. Exact computation scales poorly, but the kronfluence library (2024) implements an EK-FAC approximation that makes this feasible for models up to a few hundred million parameters.