IMPROVING STRUCTURED TEXT RECOGNITION WITH YOLO NEURAL NETWORK

Abstract

This paper explores improving structured text recognition by integrating the YOLO (You Only Look Once) neural network and OCR (Optical Character Recognition) technology. This work aims to explore ways to enhance the efficiency of structured text recognition through the integration of the YOLO model with OCR technology, as well as to develop an automated information system for detecting and subsequently recognizing text objects, which improves the overall efficiency of structured text processing. The developed system is a web application that enables users to upload invoices, store their data, and receive insights into expenses and client interactions. It includes a YOLOv10 neural network trained on a dataset of 500 invoice images, a REST API for user interaction, a user interface for invoice uploading, and a MySQL database to store information about users and their invoices. The authors propose a multithreaded architecture model that utilizes recurrent and two- and three-dimensional convolutional neural networks. The software, which implements algorithms for optical flow calculation and frequency analysis of characters, is developed in Python using Ultralytics, Pytesseract, Python Image Library, and Flask libraries. HTML, CSS, and JavaScript are used for the user interface, and MySQL is the chosen database model. The system’s main feature is the integration of the YOLO and OCR models to ensure accurate and fast recognition of text objects in images. The system architecture follows the MVC (Model-View-Controller) pattern, where the model handles data and logic, the controller acts as an intermediary between the view and model, and the view displays data to users. Each component has its respective roles and functions, making the system well-structured and easy to modify. Additional service layers are used for business logic and routing, and Flask’s Blueprint tool organizes the application into smaller components and URL structures. Overall, the system is well-structured, ensuring efficient data handling and a userfriendly interface for seamless interaction. The analysis of text recognition results demonstrated high OCR accuracy, particularly with structured text, though some limitations were observed, such as disruptions in the original text structure. These drawbacks can be mitigated by combining the YOLO network with OCR technology. Integrating YOLO with OCR enhances the system’s text recognition efficiency, enabling more precise detection of text objects and their subsequent recognition. Despite these achievements, there is room for further improvement, specifically in refining object detection and text recognition algorithms to achieve even greater accuracy and processing speed.