Joint analysis of radiological reports and ct images for automatic validation of pathological brain conditions

We consider a problem of validation of radiological medical reports and computed tomography images for an automated analysis of brain structures. Two methods for solving the problem are proposed: a method based on the ruCLIP multimodal model, and a method based on the joint use of two separate classifiers - for a text report and for a brain CT image. We discuss methods evaluation and the obtained results. The proposed approaches make it possible to correctly classify 99.6 % of radiological reports from a test sampling into 15 possible diagnoses.