On the use of causal models for testing machine learning systems

Verification of machine learning systems is a complex task that involves analyzing the dependencies between parts of the system. It seems promising to use the metamorphic testing method for verifying such systems. This paper proposes to apply causal models to root cause analysis of the defects identified during metamorphic testing of machine learning systems. Influence estimates are calculated based on the model. As a result, the components that have the most significant impact on violations of metamorphic relations can be identified. Prioritizing bug fixes for these components helps reduce the rate of violations of metamorphic relations. The applicability and usefulness of the method are shown using the example of a multi-component generative artificial intelligence system.