Early detection of autism spectrum disorder risk in children from spontaneous home videos using Zero-Shot vision-language models

This paper addresses the problem of identifying the risk of autism spectrum disorders (ASD) in children based on the analysis of behavioral patterns in unstructured video data. The problem is inherently complex, weakly formalized, and traditionally requires substantial expert effort, which limits its scalability and applicability in early screening. At present, universal technical approaches enabling automated analysis of such data under limited annotation conditions remain insufficiently developed. The aim of the study is to test the hypothesis that the ASD risk can be reliably detected in children using spontaneous home video recordings through deep learning methods that do not require additional training on specialized labeled datasets. We propose a zero-shot approach for recognizing behavioral patterns in video data based on multimodal vision-language models. The method relies on an ensemble of textual prompts, specialized design of behavioral descriptions, and hierarchical aggregation of behavioral markers. The prompt ensemble strategy ensures robust alignment between video data and textual representations by using multiple linguistically diverse formulations for each behavioral category. The proposed hierarchical aggregation scheme accounts for semantic similarity between different types of behavior and improves the robustness of zero-shot classification. The results demonstrate the proposed approach to be quite applicable for analyzing rare and underrepresented behavioral patterns characteristic of early ASD manifestations. The method is also shown to be able to produce an interpretable temporal profile of behavioral features, enabling not only classification but also temporal localization of behavioral patterns. This extends the applicability of the approach to more detailed analysis of behavioral structure over time. The scientific novelty of the work lies in the application of the zero-shot paradigm to behavioral pattern analysis in home video data, as well as in the development of a prompt ensemble strategy and hierarchical behavioral marker aggregation. The proposed approach reduces epistemic uncertainty in interpreting behavioral manifestations, decreases reliance on subjective factors, and provides a foundation for scalable and accessible solutions for early ASD risk detection.