Smart Real-Time Object Detection for Blind Individuals

Автор: Mr. Rajeev Bilagi, Mahesh S.K., Mr. Ajith Padyana, Snehal Deepak Bhasme, Vaishnavi K.

Журнал: Science, Education and Innovations in the Context of Modern Problems @imcra

Статья в выпуске: 4 vol.7, 2024 года.

Бесплатный доступ

The integration of real-time object detection with advanced text-to-speech (TTS) systems offers transformative potential for enhancing accessibility for visually impaired individuals. This paper presents a system employing cutting-edge artificial intelligence (AI) techniques, specifically YOLOv8 for object detection and Tesseract OCR for text recognition, to process visual data into audible feedback. The system achieves real-time performance with a robust architecture that integrates preprocessing, detection, and audio feedback modules. Testing demonstrates approximately 90% accuracy in object detection and 85% accuracy in text recognition, paving the way for a scalable and user-friendly assistive technology. Furthermore, the system’s adaptability to diverse environments and seamless integration with existing technologies underline its utility in addressing accessibility challenges.

Еще

Короткий адрес: https://sciup.org/16010304

IDR: 16010304   |   DOI: 10.56334/sei/7.4.11

Текст научной статьи Smart Real-Time Object Detection for Blind Individuals

Visually impaired individuals face significant challenges navigating their surroundings and accessing textual information. These challenges often result in a reduced sense of independence and can hinder their ability to engage with everyday tasks that sighted individuals may take for granted. Real-time object detection and image-to-speech systems offer potential solutions by converting visual data into audio feedback. With over 2.2 billion people globally experiencing some form of vision impairment, the need for innovative assistive technologies is more pressing than ever. Such technologies hold the promise of empowering visually impaired individuals by enhancing their interaction with the physical world through accessible, non-visual means.

The primary goal of this study is to leverage advancements in artificial intelligence (AI) to develop a sophisticated system capable of real-time performance, enabling users to identify objects and read text audibly. By employing state-of-the-art algorithms like YOLOv8 for object detection and Tesseract OCR for text recognition, the proposed system addresses significant limitations in existing assistive methods. Current technologies often lack the adaptability required for diverse environments or the precision necessary for accurate recognition. In contrast, the proposed system integrates high- speed processing, contextual accuracy, and robust adaptability, ensuring effective performance across varied scenarios such as crowded spaces, dim lighting conditions, and complex textual backgrounds.

Furthermore, the proposed solution prioritizes ease of use, making it accessible to nontechnical users. The system is designed to offer an intuitive interface and seamless interaction, thereby reducing the learning curve for end-users. Beyond its core functionality, the study also explores potential avenues for future improvements, such as the inclusion of gesture-based controls, integration with wearable devices, and multilingual support. By addressing these aspects, the research aims to deliver a comprehensive assistive technology solution that significantly enhances the quality of life for visually impaired individuals.

II LITERATURE REVIEW

Several studies have explored AI-driven object detection and OCR technologies. Sankalamaddi and Reddy (2022) utilized deep learning for real-time detection but encountered scalability issues with new objects. Their approach demonstrated the potential of optimized architectures in improving detection rates but highlighted the challenges of adapting to dynamic environments.

Kwang et al. (2023) enhanced OCR performance on distorted text but struggled with multilingual challenges, showcasing the need for robust preprocessing and adaptable recognition models. Similarly, Yi and Tian (2021) proposed an assistive text-reading system that relied heavily on structured environments, limiting its utility in real-world scenarios.

These gaps necessitate a system that integrates robust detection and recognition capabilities while addressing variability in lighting, object size, and language.

III Background

The World Health Organization (WHO) estimates that approximately 2.2 billion people worldwide suffer from visual impairment, with millions experiencing severe challenges in their daily lives. For visually impaired individuals, accessing information about their surroundings is essential but often inaccessible without assistance. Traditional tools like walking canes, tactile indicators, and guide dogs provide some level of aid but lack the adaptability and scalability required in dynamic environments. Moreover, these tools do not offer detailed contextual information, such as the identification of objects or the reading of text.

Recent advancements in artificial intelligence (AI) and computer vision have revolutionized the way assistive technologies can be developed. Deep learning models, such as YOLO (You Only Look Once), have demonstrated exceptional capabilities in real-time object detection, offering both speed and accuracy. Similarly, Optical Character Recognition (OCR) technologies, such as Tesseract, enable the extraction of textual information from images, opening new possibilities for accessibility. When combined with natural language processing and text-to-speech engines, these technologies create a pathway for visually impaired individuals to interact with their surroundings more effectively.

Existing research highlights several challenges in developing assistive technologies for the visually impaired. One major issue is ensuring real-time processing in resource-constrained environments. Low-latency solutions are critical to providing timely feedback to users. Another challenge is achieving robust performance across varying environmental conditions, such as low lighting, cluttered backgrounds, and overlapping objects. Many existing systems struggle with these scenarios, reducing their reliability in practical applications.

The proposed system aims to address these challenges by integrating YOLOv5 for object detection and Tesseract OCR for text recognition, with real-time audio feedback as the primary output. By leveraging advancements in hardware and optimized algorithms, the system provides accurate and timely assistance to visually impaired individuals. Additionally, the modular design ensures scalability and adaptability, making it suitable for a wide range of environments and use cases.

The importance of such a system extends beyond personal navigation. Applications in public transportation, retail, education, and healthcare can greatly benefit from accessible and scalable assistive technologies. This research builds upon previous studies while introducing innovative solutions to overcome the limitations of existing systems, paving the way for a more inclusive and technologically advanced future.

Globally, the challenges faced by visually impaired individuals are multifaceted, ranging from difficulties in mobility to limited access to information. The World Health Organization

(WHO) estimates that over 2.2 billion people live with vision impairments, a significant proportion of who require external assistance for daily tasks. Traditional tools such as white canes and guide dogs have been pivotal in addressing some of these challenges. However, these solutions often fall short in providing comprehensive environmental awareness, especially in dynamic or unfamiliar settings.

The demand for advanced assistive technologies has grown as societies strive for greater inclusivity and accessibility. Over the past decade, artificial intelligence (AI) and computer vision technologies have emerged as transformative tools for enhancing the quality of life for visually impaired individuals. Innovations in deep learning models, such as convolutional neural networks (CNNs), have paved the way for real-time object detection and recognition systems. YOLO (You Only Look Once), a cutting-edge object detection framework, has demonstrated unparalleled speed and accuracy, making it a prime candidate for assistive applications.

Similarly, Optical Character Recognition (OCR) technologies, such as Tesseract, have proven effective in extracting text from images, enabling visually impaired users to read signs, labels, and other textual content. By combining these technologies with natural language processing and text- to-speech engines, developers can create comprehensive systems that bridge the gap between visual input and auditory feedback.

Despite these advancements, several challenges persist in developing reliable assistive technologies. Real-time processing requires computational efficiency to minimize latency, which is crucial for dynamic environments. Environmental factors such as poor lighting, cluttered backgrounds, and overlapping objects further complicate the performance of detection systems. Additionally, user-centric considerations, such as ergonomic design and ease of use, play a critical role in the adoption of these technologies.

The proposed system builds upon these advancements while addressing the limitations of existing solutions. By integrating YOLOv5, an advanced iteration of the YOLO framework, with Tesseract OCR, the system provides real-time object and text detection. Auditory feedback ensures that users receive immediate and context-aware information about their surroundings. Moreover, the modular architecture allows for scalability and customization, enabling deployment across diverse environments, from indoor spaces to outdoor public areas.

Research in this domain has demonstrated the potential for significant societal impact. Studies have shown that AI- driven assistive technologies not only enhance mobility and independence but also contribute to improved mental well- being by reducing dependency on caregivers. The integration of such systems in everyday life can lead to greater inclusivity in education, employment, and public services.

This project represents a step forward in leveraging AI and computer vision to create a practical and accessible solution for visually impaired individuals. By addressing both technical and user-centric challenges, the system aims to set a benchmark for future innovations in assistive technology.

IV Methodology

System Design

The system  architecture comprises five  core layers: input,  processing, output, control/interface, and deployment.

Input Layer: Captures real-time video and optionally audio using an HD camera and microphone. The use of high- resolution cameras ensures the accurate capture of visual data, critical for downstream processing.

Processing Layer:

Preprocessing : Enhances image quality by resizing, noise reduction, and normalization to ensure consistent input for detection algorithms.

Detection: Uses YOLOv8 for object detection and Tesseract OCR for text recognition. YOLOv8’s real-time processing capabilities allow for seamless tracking of multiple objects in dynamic environments.

Post-Processing: Refines detection results, filters out false positives, and prioritizes objects based on user-defined criteria such as proximity or relevance.

Output Layer: Converts detections into audio feedback using TTS. The integration of natural language generation ensures that output descriptions are contextually relevant and user-friendly.

Control/Interface Layer: Provides a user-friendly interface for input and feedback. It allows users to customize detection settings, toggle multilingual support, and adjust feedback sensitivity.

Integration Layer: Supports cloud and edge deployment for scalability. This flexibility ensures that the system can operate on resource-constrained devices as well as in high-performance environments.

Figures, Listings, and Tables

Figures

  • Figure 1:    System Architecture

Illustration showing the layered structure of the system, highlighting input, processing, output, and control mechanisms.

  • Figure 2:    Object Detection Workflow

Flow diagram depicting the preprocessing, detection, and post-processing stages in the object detection module.

Tables

Table 1: System Performance Metrics

Metric

Value

Object

Detection

90%

Accuracy

OCR Accuracy

85%

Frame

Processing Rate

30 fps

Latency Per

Frame

50 ms

Table 2: Hardware Requirements

Com ponent

Specification

CPU

Intel Core i5

RAM

8 GB

Cam era

HD (720p, 30 fps)

Micr ophone

High-quality (20 Hz -

20 kHz)

V Implementation

Object Detection:

The YOLOv8 model processes live video feeds, achieving high-speed detection with GPU acceleration. Key steps include:

  •    Preprocessing video frames with OpenCV to ensure consistent quality.

  •    Detecting objects using YOLOv8 and filtering results with non-maximal suppression to eliminate redundant detections.

  •    Outputting bounding boxes, class labels, and confidence scores to the user interface for further processing.

Text Recognition:

Tesseract OCR processes regions identified as text, extracting and audibly relaying the content. Enhancements include adaptive preprocessing techniques such as contrast adjustment and morphological transformations to improve recognition under variable lighting and complex backgrounds. The integration of multilingual support ensures that the system remains effective across diverse linguistic contexts.

Object Detection

The object detection module is built upon the YOLOv8 framework, which is renowned for its speed and accuracy in real-time scenarios. Key implementation steps include:

Model Initialization:

Pretrained weights are loaded to enable immediate functionality. Custom training is conducted using domain- specific datasets to improve recognition of objects relevant to visually impaired users.

Frame Processing:

Video frames are captured and preprocessed to standardize input dimensions and optimize for GPU acceleration. Techniques such as histogram equalization improve image contrast, aiding in more accurate detections.

Detection and Tracking:

YOLOv8 identifies objects frame-by-frame. Object tracking algorithms, such as SORT (Simple Online and Realtime Tracking), are implemented to maintain continuity and track movement across frames.

Post-Processing:

Detected objects are filtered based on confidence thresholds, and redundant bounding boxes are suppressed. Outputs are tagged with class labels and confidence scores before being passed to the output layer.

Object Detection

Imagine giving the system a pair of vigilant eyes:

Pre-trained Model Foundation: We begin with YOLOv8, a pre-trained model proficient in object detection. This serves as a robust base that we fine-tune to meet specific user needs.

Optimized Image Processing: Frames from the video feed are enhanced for clarity—much like fine-tuning a photograph to highlight details before analyzing it.

Detection Workflow: YOLOv8 scans the visual data in real time, creating bounding boxes around objects and assigning labels. This happens at a blazing speed, ensuring users experience no lag.

Result Refinement: After detection, the system applies additional layers of accuracy checks, such as confidence filtering and non-maximal suppression, to deliver the most relevant insights.

Text Recognition

Turning visual text into audible information is another essential feature:

Text Area Identification: The system identifies areas in the frame likely containing text, much like isolating specific regions of interest on a map.

Text Optimization: Images undergo cleaning steps such as skew correction and binarization to make text more readable. OCR Processing: Tesseract OCR then extracts the text, converting it into a digital format for analysis and speech output.

Audio Feedback: The recognized text is converted to speech, allowing users to instantly understand their surroundings.

Enhancing Scalability and Usability

To cater to varied user needs, the system is designed to adapt to:

Diverse hardware configurations—whether it’s compact wearable devices or cloud-integrated systems.

Ongoing updates to support new features like multilingual detection, advanced object categories, and gesture-based controls.

By ensuring this modular design and scalability, the system remains future-proof and capable of addressing the evolving needs of users, making accessibility tools more dynamic and inclusive.

VI RESULTS AND DISCUSSION

Object Detection

Testing with various scenarios revealed:

  •    Accuracy : YOLOv8 achieved a 90% detection rate, effectively identifying objects in cluttered environments and under varying lighting conditions. However, challenges were noted with smaller objects and fast-moving entities.

  •    Performance : The system processed video at an average of 30 frames per second (fps), ensuring real- time operation without compromising accuracy. GPU acceleration significantly contributed to maintaining this high frame rate.

Text Recognition:

OCR accuracy was 85%, with difficulties in recognizing non- standard fonts, handwritten text, and poorly illuminated regions. Enhancements in preprocessing improved performance but highlighted the need for additional training data and specialized models for non-standard use cases. Future iterations could integrate advanced language models to bridge these gaps.

User Feedback:

Participants in pilot tests highlighted the system’s intuitive interface and the utility of its real-time audio feedback. Visually impaired users reported increased confidence in navigating environments and completing tasks independently. Suggestions for improvement included support for gesture- based controls and integration with wearable devices for hands-free operation.

VII Conclusion

The proposed system demonstrates the feasibility of real-time object and text detection for assistive applications. Key achievements include high detection accuracy, robust real- time processing, and adaptability to diverse conditions. The system’s modular design and scalability ensure its applicability across a wide range of use cases, from personal accessibility tools to industrial automation. By leveraging cutting-edge AI technologies and focusing on user-centric design principles, the system successfully addresses existing limitations in accessibility solutions.

Future work will focus on expanding object categories to include more nuanced items and environments, improving OCR to handle a broader range of languages and scripts, and incorporating advanced features such as contextual understanding, gesture recognition, and augmented reality (AR) integration. Additionally, extensive field testing in diverse real-world scenarios will be essential to fine-tune the system’s performance and ensure it meets the complex needs of visually impaired users globally. By continuing to refine and innovate, the system aims to set a benchmark for assistive technology solutions, paving the way for more inclusive and intelligent applications.

Статья научная