Noise-and Lighting-Robust Handwritten Code Recognition with Syntax-Aware Error Correction for Braille Output

Abstract

Visually impaired students are greatly challenged in accessing programming materials due to the large number of programming materials existing as handwritten images. The variations of lighting conditions, noises, skews, and handwriting styles make it difficult for conventional Optical Character Recognition (OCR) to identify and limit the possibilities of creating accurate Braille transcriptions. In this paper, we propose an approach of converting handwritten programming code into a Braille-accessible format. To achieve high-quality recognition accuracy, the proposed approach utilizes a combination of preprocessing of images, handwriting text recognition, and syntax validation. Furthermore, the system uses a token validation method and analyzes the structure of the source code by utilizing the abstract syntax trees to correct structural errors within the source code. After validation, the output is translated into technical Braille based on unified English Braille computing rules. Outputs include Unicode Braille and Braille-ready formats.