Dissertations / Theses on the topic 'Handwritten'

In this project, I want to produce a software method, which can correct quiz papers.Quizzes are multiple-choice questions and the participant should write her/his personal number on the quiz form. Since the Swedish personal numbers are unique, it is sufficient to recognize them to establish the identity of the quiz-participant.  Recognition of this number is important also for correcting the quiz, since the questions and/or answers can be adapted to each participant.  The issue is thus to recognize hand written numerals on quiz-forms.   I used image processing to find suitable features for automatic classification relying on; Logistic regression, or Neural network.

Teaching effectively requires a clear presentation of the material being taught and interaction with the students. Studies have shown that Tablet PCs provide a good technological support for teaching. The aim of the work presented in this thesis is to design a structure editor of handwritten mathematics that explores the facilities provided by Tablet PCs. The editor is made available in the form of a class library that can be used to extend existing tools. The central feature of the library is the definition of structure for handwritten mathematical expressions which allows syntactic manipulation of expressions. This makes it possible to accurately select, copy and apply algebraic rules, while avoiding the introduction of errors. To facilitate structured manipulation, gestures are used to apply manipulation rules and animations that demonstrate the use of these rules are introduced. Also, some experimental features that can improve the user’s experience and the usability of the library are presented. Furthermore, it is described how to integrate the library into existing tools. In particular, Classroom Presenter, a system developed to create interactive presentations using a Tablet PC, is extended and used to demonstrate how the library’s features can be used in some teaching scenarios. Although there are limitations in the current system, tests performed with teachers and students indicate that it can help to improve the experience of teaching and learning mathematics, particularly calculational mathematics.

Thesis (S.B. and M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.
Includes bibliographical references (leaves 58-59).
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
In recent years, the recognition of handwritten mathematical expressions has received an increasing amount of attention in pattern recognition research. The diversity of approaches to the problem and the lack of a commercially viable system, however, indicate that there is still much research to be done in this area. In this thesis, I will describe an on-line approach for converting a handwritten mathematical expression into an equivalent expression in a typesetting command language such as TEX or MathML, as well as a feedback-oriented user interface which can make errors more tolerable to the end user since they can be quickly corrected. The three primary components of this system are a method for classifying isolated handwritten symbols, an algorithm for partitioning an expression into symbols, and an algorithm for converting a two-dimensional arrangements of symbols into a typeset expression. For symbol classification, a Gaussian classifier is used to rank order the interpretations of a set of strokes as a single symbol. To partition an expression, the values generated by the symbol classifier are used to perform a constrained search of possible partitions for the one with the minimum summed cost. Finally, the expression is parsed using a simple geometric grammar.
by Nicholas E. Matsakis.
S.B.and M.Eng.

L’objectiu d’aquesta tesi és l’extracció d’Informació de documents total o parcialment manuscrits amb una certa estructura. Bàsicament treballem amb dos escenaris d’aplicació diferent. El primer escenari són els documents moderns altament estructurats, com formularis. En aquests documents, la informació semàntica està ja definida en camps, amb una posició concreta al document i l’extracció de la informació és equivalent a una transcripció. El segon escenari son els documents semi-estructurats totalment manuscrits on, a més de transcriure, cal associar un valor semàntic, d’entre un conjunt conegut de valors possibles, a les paraules que es transcriuen. En ambdós casos la qualitat de la transcripció té un gran pes en la precisió del sistema, per això proposem models basats en xarxes neuronals per a transcriure text manuscrit. Per a poder afrontar el repte dels documents semi-estructurats hem generat un benchmark, compost de dataset, una sèrie de tasques definides i una mètrica que es va presentar a la comunitat científica com a una competició internacional. També proposem diferents models basats en Xarxes Neuronals Convolucionals i recurrents, capaços de transcriure i assignar diferent etiquetes semàntiques a cada paraula manuscrita, és a dir, capaços d'extreure informació.
El objetivo de esta tesis es la extracción de Información de documentos total o parcialmente manuscritos, con una cierta estructura. Básicamente trabajamos con dos escenarios de aplicación diferentes. El primer escenario son los documentos modernos altamente estructurados, como los formularios. En estos documentos, la información semántica está pre-definida en campos con una posición concreta en el documento i la extracción de información es equivalente a una transcripción. El segundo escenario son los documentos semi-estructurados totalmente manuscritos, donde, además de transcribir, es necesario asociar un valor semántico, de entre un conjunto conocido de valores posibles, a las palabras manuscritas. En ambos casos, la calidad de la transcripción tiene un gran peso en la precisión del sistema. Por ese motivo proponemos modelos basados en redes neuronales para transcribir el texto manuscrito. Para poder afrontar el reto de los documentos semi-estructurados, hemos generado un benchmark, compuesto de dataset, una serie de tareas y una métrica que fue presentado a la comunidad científica a modo de competición internacional. También proponemos diferentes modelos basados en Redes Neuronales Convolucionales y Recurrentes, capaces de transcribir y asignar diferentes etiquetas semánticas a cada palabra manuscrita, es decir, capaces de extraer información.
The goal of this thesis is information Extraction from totally or partially handwritten documents. Basically we are dealing with two different application scenarios. The first scenario are modern highly structured documents like forms. In this kind of documents, the semantic information is encoded in different fields with a pre-defined location in the document, therefore, information extraction becomes equivalent to transcription. The second application scenario are loosely structured totally handwritten documents, besides transcribing them, we need to assign a semantic label, from a set of known values to the handwritten words. In both scenarios, transcription is an important part of the information extraction. For that reason in this thesis we present two methods based on Neural Networks, to transcribe handwritten text.In order to tackle the challenge of loosely structured documents, we have produced a benchmark, consisting of a dataset, a defined set of tasks and a metric, that was presented to the community as an international competition. Also, we propose different models based on Convolutional and Recurrent neural networks that are able to transcribe and assign different semantic labels to each handwritten words, that is, able to perform Information Extraction.

Thèse (Ph.D.)--École de technologie supérieure, Montréal, 2003.
"Thesis presented to the École de technologie supérieure in partial fulfillment of the thesis requirement for the degree of philosophiae doctor in engineering". La numérotation de cet ouvrage est erronée. Bibliogr.: f. [149]-163. Également disponible en version électronique.

COORDENAÇÃO DE APERFEIÇOAMENTO DO PESSOAL DE ENSINO SUPERIOR
Esta dissertação investiga a utilização de Redes Neurais Artificiais (RNAs) na área de reconhecimento de caracteres, em particular de dígitos manuscritos. Nesta investigação foram utilizadas amostras reais de dígitos isolados e de códigos postais brasileiros relativos e vários escritores. O trabalho consiste de quatro partes principais: o estudo das metodologias de reconhecimento e da semântica e estrutura de representação de caracteres; o desenvolvimento das etapas de pré-processamento dos dígitos; o desenvolvimento das RNAs para o reconhecimento de dígitos manuscritos; e o estudo de casos. No estudo sobre a metodologia de reconhecimento de caracteres fez-se um levantamento preliminar das diversas aplicaões de sistemas OCR (Optical Character Recognition). Enfatizou-se a classificação dos diversos tipos de semânticas existentes de acordo com a aplicação específica, bem como a estrutura geral de um sistema OCR. O estudo também consistiu da análise e apresentação de modelos convencionais e de sistemas inteligentes na implementação da etapa de classificação dos sistemas OCR. O desenvolvimento do pré-processamento dos dígitos envolveu um extenso estudo bibliográfico de diversas metodologias para cada uma de suas etapas. Foram estudados os algoritmos mais empregados nas etapas de pré- processamento de um sistema. OCR: conversão de níveis de cinza para representação binária ( thresholding), filtragem, segmentação e normalização. A partir desse estudo, foram selecionados e desenvolvidos determinados tipos de algoritmos para o pré-processamento. No desenvolvimento de RNAs para o reconhecimento de dígitos manuscritos fez-se uma investigação de diversas metodologias, incluindo as arquiteturas e os algoritmos de aprendizado mais empregados. Neste estudo, constatou-se a predominância do uso do algoritmo de retropropagação do erro (BackPropagation) para o treinamento das redes nas aplicações de reconhecimento de caracteres manuscritos. As arquiteturas propostas neste trabalho foram escolhidas de acordo com dois tipos de aplicados de reconhecimento: reconhecimento de dígitos manuscritos isolados e reconhecimento automático de código postal. No estudo de casos, as RNAs foram modeladas para fazer o reconhecimento automático de código postal. Este estudo consistiu de um conjunto de implementações com o objetivo de testar o desempenho de um sistema OCR baseado em redes neurais. Foram feitos testes com dois tipos de sistemas de reconhecimento por redes neurais: redes totalmente conectadas e redes parcialmente. Para os dois casos foram utilizados amostras reais colhidas de 73 escritores. Os resultados obtidos com os dois tipos de redes foram comparados e comprovaram a superioridade das RNAs com arquitetura parcialmente conectada no reconhecimento de dígitos altamente ruidosos. Comparações também foram feitas com outras técnicas convencionais de reconhecimento, obtendo-se resultados, em muitos casos, superiores.
This dissertation investigates the use of Artificial Neural Networks (ANNs) for character recognition, especially handwritten digits. Real samples of isolated and postal code digits were used from different writers. The dissertation covers four main part: the study of methodologies, semantics and structure on character recognition and its representation; the development of the digits preprocessing phases; the design of ANNs to handwritten digits recognition; and the case studies. The first part of this dissertation studies methodologies, semantics and structures used on character recognition. The result of this study is an overview of the major aplication in OCR (Optical Character Recognition). Different kinds of semantics and their structures were classified according to each specific application. Several conventional models and intelligent systems, used in the classification stage of OCR systems, had also been discussed. The development of the digits preprocessing involved the investigation of different methodologies related to each preprocessing phase. The most used algorithm for each preprocessing phase were considered: thresholding, smoothing, segmentation and normalization. According to this study, specific algorithms were selected and developed. In the design of ANNs for handwritten digits recognition, different methodologies had been investigated, including the architetures and the learning algorithms most used. This overview confirmed the predominance of BackPropagation as the training algorithm for the Neural Network in this application. The architetures proposed in this work had been selected according to two types of applications of character recognition: isolated handwritten digits recognition and postal address code recognition. The case studies consisted of the designing of an ANN to postal address code recognition. The case studies involved testing the system performance for two kinds of ANNs: fully connected networks and partially connected networks. In both cases, samples of 73 writers were used. The results were compared to each other, confirming the superiority of partially connected ANN in handling noisy digits. The ANN perfomance was also compared with the perfomance of other conventional techniques, achieving better results in many cases.

As more and more organizations digitize their records, the need for automatic document processing software increases. In particular, the rise of ‘digital humanities’ precede a new set of problems on how to digitize historical archival material in an efficient and accurate manner. The transcription of archival material to formats fit for research purposes, such as handwritten spreadsheets, is still expensive and plagued by tedious manual labor. Over the decades, research in handwritten text recognition has focused on text line extraction and recognition. In this thesis, we examine document images that contain complex details, contain more categories of text than handwriting, and handwritten text that is not separated easily to lines. The thesis examines the sub-problem of handwritten text segmentation in detail. We propose a broad definition of text segmentation that requires both text detection and text classification, since this enables us to detect multiple kinds of text within the same image. The aim is to design a system which can detect and identify both handwriting and machine-text within the same image. Working with photographs of spreadsheet documents from the years 1871-1951, a topdown layout-agnostic image processing pipeline is developed. Different kinds of preprocessing are examined, to correct illumination and enhance contrast before binarization, and to detect and clear line contours. To achieve text region detection, we evaluate connected components labeling and MSER as region detectors, extracting textual and non-textual sub-images. On detected sub-images, we perform a Bag-of-Visual-Words quantization of k-means clustered feature descriptor vectors and perform categorical classification by training a Naïve Bayesclassifier on the feature distances to the cluster centroids. Results include a novel two-stage illumination correction and contrast enhancement algorithm that improves document quality as a precursor to binarization, increasing the mean grayscale values of an image while retaining low grayscale variance. Region detectors are evaluated on images with different types of preprocessing and the results show that clearing document outlines influences text region detection. Training on a small sample of sub-images, the categorical classification model proves viable for discrimination between machine-text and handwriting, enabling the use of this model for further recognition purposes.

Malgré les prédictions d'un monde sans papier et le développement des documents électroniques, les documents manuscrits ont gardé leur importance et les problèmes de l'identification et de l'authentification des auteurs ont constitué un domaine de recherche actif au cours de ces dernières années. Nous avons développé une méthode efficace pour la reconnaissance automatique de scripteur à partir des images de texte manuscrit offline. Notre méthode repose sur deux aspects différents de l'écriture, la présence des formes redondantes dans l'écriture et des attributs visuels de l'écriture. En nous basant sur l'hypothèse qu'un individu utilise certaines formes plus fréquemment que les autres quand il écrit, nous espérons extraire ces formes en analysant des petits fragments d'écriture et en regroupant les formes similaires dans des classes. Ces classes sont déterminées soit pour chacun des scripteurs séparément ou pour un groupe de scripteurs générant un ensemble universel de formes. L'écriture en question est ensuite comparée à ces classes de formes produites. Ensuite, nous exploitons les deux importants attributs visuels de l'écriture, l'orientation et la courbure, qui permettent de distinguer une écriture d'une autre. Ces attributs sont extraits par le calcul d'un ensemble de caractéristiques à différents niveaux d'observation. Deux écritures sont ensuite comparées en calculant les distances entre leurs caractéristiques respectives. Enfin, nous combinons les deux facettes de l'écriture pour caractériser le scripteur d'un échantillon manuscrit. En utilisant ces caractéristiques, on obtient des taux d'identification qui sont comparables aux meilleurs résultats rapportés à ce jour pour l'identification de scripteur hors ligne
The problem of identifying the writer of a handwritten document image has been an active research area over the last few years and enjoys applications in forensic and historical document analysis. We have developed an effective method for automatic writer identification and verification from unconstrained handwritten text images. Our method relies on two different aspects of writing: the presence of redundant patterns in the writing and its visual attributes. Based on the hypothesis that handwriting carries certain patterns that an individual would use frequently as he writes, we look to extract these patterns by analyzing small writing fragments and grouping similar patterns into clusters. In fact this corresponds more to the redundancy of writing gestures than writing shapes. These clusters are determined either for each of the writers separately or, for a group of writers generating a universal set of patterns. The writing in question is then compared to the produced clusters. We next exploit two important visual attributes of writing, the orientation and curvature, which enable to distinguish one writing from another. These attributes are extracted by computing a set of features from writing samples at different levels of observation. Two writings are then compared by computing distances between their respective features. Finally, we combine the two facets of handwriting to characterize the writer of a handwritten sample. The proposed methodology, evaluated on modern as well as ancient writings exhibited promising results on tasks of writer recognition and handwriting classification

In last years Hidden Markov Models (HMMs) have received significant attention in the task off-line handwritten text recognition (HTR). As in automatic speech recognition (ASR), HMMs are used to model the probability of an observation sequence, given its corresponding text transcription. However, in contrast to what happens in ASR, in HTR there is no standard set of local features being used by most of the proposed systems. In this thesis we propose the use of raw binary pixels as features, in conjunction with models that deal more directly with the binary data. In particular, we propose the use of Bernoulli HMMs (BHMMs), that is, conventional HMMs in which Gaussian (mixture) distributions have been replaced by Bernoulli (mixture) probability functions. The objective is twofold: on the one hand, this allows us to better modeling the binary nature of text images (foreground/background) using BHMMs. On the other hand, this guarantees that no discriminative information is filtered out during feature extraction (most HTR available datasets can be easily binarized without a relevant loss of information). In this thesis, all the HMM theory required to develop a HMM based HTR toolkit is reviewed and adapted to the case of BHMMs. Specifically, we begin by defining a simple classifier based on BHMMs with Bernoulli probability functions at the states, and we end with an embedded Bernoulli mixture HMM recognizer for continuous HTR. Regarding the binary features, we propose a simple binary feature extraction process without significant loss of information. All input images are scaled and binarized, in order to easily reinterpret them as sequences of binary feature vectors. Two extensions are proposed to this basic feature extraction method: the use of a sliding window in order to better capture the context, and a repositioning method in order to better deal with vertical distortions. Competitive results were obtained when BHMMs and proposed methods were applied to well-known HTR databases. In particular, we ranked first at the Arabic Handwriting Recognition Competition organized during the 12th International Conference on Frontiers in Handwriting Recognition (ICFHR 2010), and at the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text organized during the 11th International Conference on Document Analysis and Recognition (ICDAR 2011). In the last part of this thesis we propose a method for training BHMM classifiers using In last years Hidden Markov Models (HMMs) have received significant attention in the task off-line handwritten text recognition (HTR). As in automatic speech recognition (ASR), HMMs are used to model the probability of an observation sequence, given its corresponding text transcription. However, in contrast to what happens in ASR, in HTR there is no standard set of local features being used by most of the proposed systems. In this thesis we propose the use of raw binary pixels as features, in conjunction with models that deal more directly with the binary data. In particular, we propose the use of Bernoulli HMMs (BHMMs), that is, conventional HMMs in which Gaussian (mixture) distributions have been replaced by Bernoulli (mixture) probability functions. The objective is twofold: on the one hand, this allows us to better modeling the binary nature of text images (foreground/background) using BHMMs. On the other hand, this guarantees that no discriminative information is filtered out during feature extraction (most HTR available datasets can be easily binarized without a relevant loss of information). In this thesis, all the HMM theory required to develop a HMM based HTR toolkit is reviewed and adapted to the case of BHMMs. Specifically, we begin by defining a simple classifier based on BHMMs with Bernoulli probability functions at the states, and we end with an embedded Bernoulli mixture HMM recognizer for continuous HTR. Regarding the binary features, we propose a simple binary feature extraction process without significant loss of information. All input images are scaled and binarized, in order to easily reinterpret them as sequences of binary feature vectors. Two extensions are proposed to this basic feature extraction method: the use of a sliding window in order to better capture the context, and a repositioning method in order to better deal with vertical distortions. Competitive results were obtained when BHMMs and proposed methods were applied to well-known HTR databases. In particular, we ranked first at the Arabic Handwriting Recognition Competition organized during the 12th International Conference on Frontiers in Handwriting Recognition (ICFHR 2010), and at the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text organized during the 11th International Conference on Document Analysis and Recognition (ICDAR 2011). In the last part of this thesis we propose a method for training BHMM classifiers using In last years Hidden Markov Models (HMMs) have received significant attention in the task off-line handwritten text recognition (HTR). As in automatic speech recognition (ASR), HMMs are used to model the probability of an observation sequence, given its corresponding text transcription. However, in contrast to what happens in ASR, in HTR there is no standard set of local features being used by most of the proposed systems. In this thesis we propose the use of raw binary pixels as features, in conjunction with models that deal more directly with the binary data. In particular, we propose the use of Bernoulli HMMs (BHMMs), that is, conventional HMMs in which Gaussian (mixture) distributions have been replaced by Bernoulli (mixture) probability functions. The objective is twofold: on the one hand, this allows us to better modeling the binary nature of text images (foreground/background) using BHMMs. On the other hand, this guarantees that no discriminative information is filtered out during feature extraction (most HTR available datasets can be easily binarized without a relevant loss of information). In this thesis, all the HMM theory required to develop a HMM based HTR toolkit is reviewed and adapted to the case of BHMMs. Specifically, we begin by defining a simple classifier based on BHMMs with Bernoulli probability functions at the states, and we end with an embedded Bernoulli mixture HMM recognizer for continuous HTR. Regarding the binary features, we propose a simple binary feature extraction process without significant loss of information. All input images are scaled and binarized, in order to easily reinterpret them as sequences of binary feature vectors. Two extensions are proposed to this basic feature extraction method: the use of a sliding window in order to better capture the context, and a repositioning method in order to better deal with vertical distortions. Competitive results were obtained when BHMMs and proposed methods were applied to well-known HTR databases. In particular, we ranked first at the Arabic Handwriting Recognition Competition organized during the 12th International Conference on Frontiers in Handwriting Recognition (ICFHR 2010), and at the Arabic Recognition Competition: Multi-font Multi-size Digitally Represented Text organized during the 11th International Conference on Document Analysis and Recognition (ICDAR 2011). In the last part of this thesis we propose a method for training BHMM classifiers using discriminative training criteria, instead of the conventionalMaximum Likelihood Estimation (MLE). Specifically, we propose a log-linear classifier for binary data based on the BHMM classifier. Parameter estimation of this model can be carried out using discriminative training criteria for log-linear models. In particular, we show the formulae for several MMI based criteria. Finally, we prove the equivalence between both classifiers, hence, discriminative training of a BHMM classifier can be carried out by obtaining its equivalent log-linear classifier. Reported results show that discriminative BHMMs clearly outperform conventional generative BHMMs.
Giménez Pastor, A. (2014). Bernoulli HMMs for Handwritten Text Recognition [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/37978
TESIS

Existen incontables colecciones de documentos históricos en archivos y librerías repletos de valiosa información para historiadores e investigadores. La extracción de esta información se ha convertido en una de las principales tareas para investigadores del área de análisis de documentos. Hay un interés creciente en digitalizar, conservar y dar acceso a este tipo de documentos. Pero sólo la digitalización no es suficiente para los investigadores. La extracción y/o indexación de la información de estos documentos tiene un creciente interés entre los investigadores. En muchos casos, y en particular en documentos históricos, la completa trascripción de estos documentos es extremadamente difícil debido a dificultades intrínsecas: preservación física pobre, diferentes estilos de escritura, lenguajes obsoletos, etc. La búsqueda de palabras se convierte en una popular y eficiente alternativa a la tran-scripción completa. Este método conlleva una inherente degradación de las imágenes. La búsqueda de palabras se formula holísticamente como una búsqueda visual de una forma dada en un conjunto grande de imágenes, en vez de reconocer el texto y buscar la palabra mediante la comparación de códigos ascii. Pero el rendimiento de los métodos de búsqueda de palabras clásicos puede verse afectado por el nivel de degradación de las imágenes, que en algunos casos pueden ser inaceptables. Por esta razón, proponemos una búsqueda de palabras contextual que utiliza la información contextual/semántica para obtener resultados donde los métodos de búsqueda clásica no lo logran un rendimiento aceptable. El sistema de búsqueda de palabras contextual propuesto en esta tesis utiliza un método de búsqueda de palabras basado en segmentación, y por tanto es necesaria una segmentación de palabras precisa. Documentos históricos manuscritos presentan algunas dificultades que pueden dificultar la extracción de palabras. Proponemos un método de segmentación de palabras que formula el problema como la búsqueda del camino central en el area que hay entre dos líneas consecutivas. Esto se resuelve mediante un problema de grafo transversal. Un algoritmo de búsqueda de caminos es utilizado para encontrar el camino óptimo en el grafo, calculado previamente, entre dos líneas de texto. Una vez las líneas se han extraído, las palabras son localizadas dentro de las líneas de texto utilizando un método del estado del arte para segmentar palabras. Los métodos de búsqueda clásicos pueden mejor utilizando la información contextual de los documentos. Presentamos un nuevo sistema, orientado a documentos manuscritos que presentan una estructura a los largo de sus páginas, para extraer la información uti-lizando información contextual. El sistema es una eficiente herramienta para la transcripción semiautomática que utiliza la información contextual para obtener mejores resultados que los métodos de búsqueda convencionales. La información contextual es descubierta automáticamente reconociendo estructuras repetitivas y categorizando las palabras con su correspondiente clase semántica. Se extraen las palabras más frecuentes de cada clase semántica y así el mismo texto es utilizado para transcribir todas ellas. Los resultados experimentales obtenidos en esta tesis mejoran los resultados de los métodos clásicos de búsqueda de palabras, demostrando idoneidad de la arquitectura propuesta para la búsqueda de palabras en documentos históricos manuscritos utilizando la información contextual.
There are countless collections of historical documents in archives and libraries that contain plenty of valuable information for historians and researchers. The extraction of this information has become a central task among the Document Analysis researches and practitioners. There is an increasing interest to digital preserve and provide access to these kind of documents. But only the digitalization is not enough for the researchers. The extraction and/or indexation of information of this documents has had an increased interest among researchers. In many cases, and in particular in historical manuscripts, the full transcription of these documents is extremely di cult due the inherent de ciencies: poor physical preservation, di erent writing styles, obsolete languages, etc. Word spotting has become a popular an e cient alternative to full transcription. It inherently involves a high level of degradation in the images. The search of words is holistically formulated as a visual search of a given query shape in a larger image, instead of recognising the input text and searching the query word with an ascii string comparison. But the performance of classical word spotting approaches depend on the degradation level of the images being unacceptable in many cases . In this thesis we have proposed a novel paradigm called contextual word spotting method that uses the contextual/semantic information to achieve acceptable results whereas classical word spotting does not reach. The contextual word spotting framework proposed in this thesis is a segmentation-based word spotting approach, so an e cient word segmentation is needed. Historical handwritten documents present some common di culties that can increase the di culties the extraction of the words. We have proposed a line segmentation approach that formulates the problem as nding the central part path in the area between two consecutive lines. This is solved as a graph traversal problem. A path nding algorithm is used to nd the optimal path in a graph, previously computed, between the text lines. Once the text lines are extracted, words are localized inside the text lines using a word segmentation technique from the state of the art. Classical word spotting approaches can be improved using the contextual information of the documents. We have introduced a new framework, oriented to handwritten documents that present a highly structure, to extract information making use of context. The framework is an e cient tool for semi-automatic transcription that uses the contextual information to achieve better results than classical word spotting approaches. The contextual information is automatically discovered by recognizing repetitive structures and categorizing all the words according to semantic classes. The most frequent words in each semantic cluster are extracted and the same text is used to transcribe all them. The experimental results achieved in this thesis outperform classical word spotting approaches demonstrating the suitability of the proposed ensemble architecture for spotting words in historical handwritten documents using contextual information.

Recognizing handwritten mathematical content is a challenging problem, and more so when such content appears in classroom videos. However, given the fact that in such videos the handwritten text and the accompanying audio refer to the same content, a combination of video and audio based recognizer has the potential to significantly improve the content recognition accuracy. This dissertation, using a combination of video and audio based recognizers, focuses on improving the recognition accuracy associated with handwritten mathematical content in such videos. Our approach makes use of a video recognizer as the primary recognizer and a multi-stage assembly, developed as part of this research, is used to facilitate effective combination with an audio recognizer. Specifically, we address the following challenges related to audio-video based handwritten mathematical content recognition: (1) Video Preprocessing - generates a timestamped sequence of segmented characters from the classroom video in the face of occlusions and shadows caused by the instructor, (2) Ambiguity Detection - determines the subset of input characters that may have been incorrectly recognized by the video based recognizer and forwards this subset for disambiguation, (3) A/V Synchronization - establishes correspondence between the handwritten character and the spoken content, (4) A/V Combination - combines the synchronized outputs from the video and audio based recognizers and generates the final recognized character, and (5) Grammar Assisted A/V Based Mathematical Content Recognition - utilizes a base mathematical speech grammar for both character and structure disambiguation. Experiments conducted using videos recorded in a classroom-like environment demonstrate the significant improvements in recognition accuracy that can be achieved using our techniques.

Au cours des dernières années, des progrès considérables ont été accomplis dans le domaine de la reconnaissance de l'écriture manuscrite. Ainsi, il est intéressant de constater que la plupart des systèmes existants s'appuient sur l'utilisation d'un lexique pour effectuer la reconnaissance de mots. Or, dans la plupart des applications le lexique utilisé est de petite ou de moyenne dimension. Bien entendu, la possibilité de traiter efficacement un très grand vocabulaire permettrait d'élargir le champ des applications, mais cette extension du vocabulaire (de quelques dizaines à plus de 80000 mots) a pour conséquence l'explosion de l'espace de recherche et bien souvent la dégradation des taux de reconnaissance. Ainsi, le thème principal de cette thèse de doctorat est la reconnaissance de l'écriture manuscrite dans le cadre de l'utilisation de lexique de très grande dimension. Nous présentons tout d'abord, plusieurs stratégies pour améliorer en termes de vitesse de reconnaissance les performances d'un système de référence. L'objectif sera alors de permettre au système de traiter de très grands lexiques dans un temps raisonnable. Par la suite, nous améliorons les performances en termes de taux de reconnaissance. Pour ce faire, nous utiliserons une approche neuronale afin de vérifier les N meilleurs hypothèses de mots isolés par le système de référence. D'autre part, toutes les caractéristiques du système initial ont été conservées: système omni-scripteurs, écriture sans contraintes, et lexiques générés dynamiquement. Les contributions majeures de cette thèse sont l'accélération d'un facteur 120 du temps de traitement et l'amélioration du taux de reconnaissance d'environ 10% par rapport au système de référence. Le gain en vitesse est obtenu grâce aux techniques suivantes: recherche dans un arbre lexical, réduction des multiples modèles de caractères, techniques de reconnaissance guidée par le lexique avec et sans contraintes, algorithme "level-building" guidé par le lexique, algorithme rapide à deux niveaux pour effectuer le décodage des séquences d'observations et utilisation d'une approche de reconnaissance distribuée. Par ailleurs, la précision du système est améliorée par le post-traitement des N meilleures hypothèses de mots à l'aide d'un module de vérification. Ce module est basé sur l'utilisation d'un réseau de neurones pour vérifier la présence de chacun des caractères segmentés par le système de base. La combinaison des résultats du système de référence et du module de vérification permet alors d'améliorer significativement les performances de reconnaissance. Enfin, une procédure de rejet est mise en place et permet d'atteindre un taux de reconnaissance d'environ 95% en ne rejetant que 30% des exemples.

A number of new techniques and approaches for off-line handwritten character recognition are presented which individually make significant advancements in the field. First. an outline-based vectorization algorithm is described which gives improved accuracy in producing vector representations of the pen strokes used to draw characters. Later. Vectorization and other types of preprocessing are criticized and an approach to recognition is suggested which avoids separate preprocessing stages by incorporating them into later stages. Apart from the increased speed of this approach. it allows more effective alteration of the character images since more is known about them at the later stages. It also allows the possibility of alterations being corrected if they are initially detrimental to recognition. A new feature measurement. the Radial Distance/Sector Area feature. is presented which is highly robust. tolerant to noise. distortion and style variation. and gives high accuracy results when used for training and testing in a statistical or neural classifier. A very powerful classifier is therefore obtained for recognizing correctly segmented characters. The segmentation task is explored in a simple system of integrated over-segmentation. Character classification and approximate dictionary checking. This can be extended to a full system for handprinted word recognition. In addition to the advancements made by these methods. a powerful new approach to handwritten character recognition is proposed as a direction for future research. This proposal combines the ideas and techniques developed in this thesis in a hierarchical network of classifier modules to achieve context-sensitive. off-line recognition of handwritten text. A new type of "intelligent" feedback is used to direct the search to contextually sensible classifications. A powerful adaptive segmentation system is proposed which. when used as the bottom layer in the hierarchical network. allows initially incorrect segmentations to be adjusted according to the hypotheses of the higher level context modules.

The author presents novel algorithms to design unconstrained handwriting recognition systems organized in three parts: In Part One, novel algorithms are presented for processing of Arabic text prior to recognition. Algorithms are described to convert a thinned image of a stroke to a straight line approximation. Novel heuristic algorithms and novel theorems are presented to determine start and end vertices of an off-line image of a stroke. A straight line approximation of an off-line stroke is converted to a one-dimensional representation by a novel algorithm which aims to recover the original sequence of writing. The resulting ordering of the stroke segments is a suitable preprocessed representation for subsequent handwriting recognition algorithms as it helps to segment the stroke. The algorithm was tested against one data set of isolated handwritten characters and another data set of cursive handwriting, each provided by 20 subjects, and has been 91.9% and 91.8% successful for these two data sets, respectively. In Part Two, an entirely novel fuzzy set-sequential machine character recognition system is presented. Fuzzy sequential machines are defined to work as recognizers of handwritten strokes. An algorithm to obtain a deterministic fuzzy sequential machine from a stroke representation, that is capable of recognizing that stroke and its variants, is presented. An algorithm is developed to merge two fuzzy machines into one machine. The learning algorithm is a combination of many described algorithms. The system was tested against isolated handwritten characters provided by 20 subjects resulting in 95.8% recognition rate which is encouraging and shows that the system is highly flexible in dealing with shape and size variations. In Part Three, also an entirely novel text recognition system, capable of recognizing off-line handwritten Arabic cursive text having a high variability is presented. This system is an extension of the above recognition system. Tokens are extracted from a onedimensional representation of a stroke. Fuzzy sequential machines are defined to work as recognizers of tokens. It is shown how to obtain a deterministic fuzzy sequential machine from a token representation that is capable'of recognizing that token and its variants. An algorithm for token learning is presented. The tokens of a stroke are re-combined to meaningful strings of tokens. Algorithms to recognize and learn token strings are described. The. recognition stage uses algorithms of the learning stage. The process of extracting the best set of basic shapes which represent the best set of token strings that constitute an unknown stroke is described. A method is developed to extract lines from pages of handwritten text, arrange main strokes of extracted lines in the same order as they were written, and present secondary strokes to main strokes. Presented secondary strokes are combined with basic shapes to obtain the final characters by formulating and solving assignment problems for this purpose. Some secondary strokes which remain unassigned are individually manipulated. The system was tested against the handwritings of 20 subjects yielding overall subword and character recognition rates of 55.4% and 51.1%, respectively.

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2015.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Pentimento is software developed under the supervision of Fredo Durand in the Computer Graphics Group at CSAIL that focuses on dramatically simplifying the creation of online educational video lectures such as those of Khan Academy. In these videos, the lecture style is that the educator draws on a virtual whiteboard as he/she speaks. Currently, the type of software that the educator uses is very rudimentary in its functionality and only allows for basic functionality such as screen and voice recording. A downside of this approach is that the educator must get it right on the first approach, as there is no ability to simply edit the content taken during a screen capture after the initial recording without using unnecessarily complex video editing software. Even with video editing software, the user is not able to access the original drawing content used to create video. The overall goal of this project is to develop lecture recording software that uses a vector based representation to keep track of the user's sketching, which will allow the user to easily editing the original drawing content retroactively. The goal for my contribution to this project is to implement components for a web-based version of Pentimento. This will allow the application to reach a broader range of users. The goal is to have an HTML5 and Javascript based application that can run on many of popular the web browsers in use today. One of my main focuses in this project is to work on the audio recording and editing component. This includes the working on the user interface component and integrating it with the rest of the parts in the software.
by Jonathan Wang.
M. Eng.

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 81-82).
This thesis presents an interpretation and clustering framework for handwritten student responses on tablet computers. The ink analysis system is able to capture and interpret digital ink strokes for many types of classroom exercises, including graphs, number lines, and fraction shading problems. By approaching the problem with both online and offline ink interpretation methods, relevant information is extracted from sets of ink strokes to produce a representation of a student's answer. A clustering algorithm is then used to group similar student responses. Overall, this approach makes it easier for teachers to view a set of responses and subsequently supply feedback to his or her students.
by Kelsey Leigh Von Tish.
M.Eng.

Thesis (MScEng)--University of Stellenbosch, 2003.
ENGLISH ABSTRACT: Handwritten signatures are provided extensively to verify identity for all types of transactions and documents. However, they are very rarely actually verified. This is because of the high cost of training and employing enough human operators (who are still fallible) to cope with the demand. They are a very well known, yet under-utilised biometric currently performing far below their potential. We present an on-line/dynamic handwritten signature verification system based on Hidden Markov Models, that far out performs human operators in both accuracy and speed. It uses only the local signature features-sampled from an electronic writing tablet-after some novel preprocessing steps, and is a fully automated system in that there are no parameters that need to be manually fine-tuned for different users. Novel verifiers are investigated which attain best equal error rates of between 2% and 5% for different types of high quality deliberate forgeries, and take a fraction of a second to accept or reject an identity claim on a 700 MHz computer.
AFRIKAANSE OPSOMMING: Geskrewe handtekeninge word gereeld gebruik om die identiteit van dokumente en transaksies te bevestig. Aangesien dit duur is in terme van menslike hulpbronne, word die integrit eit daarvan selde nagegaan. Om handtekeninge deur menslike operateurs te verifieër. is ook feilbaar-lOO% akkurate identifikasie is onrealisties. Handtekeninge is uiters akkurate en unieke identifikasie patrone wat in die praktyk nie naastenby tot hul volle potensiaal gebruik word nie. In hierdie navorsing gebruik ons verskuilde Markov modelle om dinamiese handtekeningherkenningstelsels te ontwikkel wat, in terme van spoed en akkuraatheid heelwat meer effektief as operateurs is. Die stelsel maak gebruik van slegs lokale handtekening eienskappe (en verwerkings daarvan) soos wat dit verkry word vanaf 'n elektroniese skryftablet. Die stelsel is ten volle outomaties en geen parameters hoef aangepas te word vir verskillende gebruikers nie. 'n Paar tipes nuwe handtekeningverifieërders word ondersoek en die resulterende gelykbreekpunt vir vals-aanvaardings- en vals-verwerpingsfoute lê tussen 2% en 5% vir verskillende tipes hoë kwaliteit vervalsde handtekeninge. Op 'n tipiese 700 MHz verwerker word die identiteit van 'n persoon ill minder as i sekonde bevestig.

Véritable challenge scientifique, la reconnaissance d’expressions mathématiques manuscrites est un champ très attractif de la reconnaissance des formes débouchant sur des applications pratiques innovantes. En effet, le grand nombre de symboles (plus de 100) utilisés ainsi que la structure en 2 dimensions des expressions augmentent la difficulté de leur reconnaissance. Dans cette thèse, nous nous intéressons à la reconnaissance des expressions mathématiques manuscrites en-ligne en utilisant de façon innovante les réseaux de neurones récurrents profonds BLSTM avec CTC pour construire un système d’analyse basé sur la construction de graphes. Nous avons donc étendu la structure linéaire des BLSTM à des structures d’arbres (Tree-Based BLSTM) permettant de couvrir les 2 dimensions du langage. Nous avons aussi proposé d’ajouter des contraintes de localisation dans la couche CTC pour adapter les décisions du réseau à l’échelle des traits de l’écriture, permettant une modélisation et une évaluation robustes. Le système proposé construit un graphe à partir des traits du tracé à reconnaître et de leurs relations spatiales. Plusieurs arbres sont dérivés de ce graphe puis étiquetés par notre Tree-Based BLSTM. Les arbres obtenus sont ensuite fusionnés pour construire un SLG (graphe étiqueté de traits) modélisant une expression 2D. Une différence majeure par rapport aux systèmes traditionnels est l’absence des étapes explicites de segmentation et reconnaissance des symboles isolés puis d’analyse de leurs relations spatiales, notre approche produit directement un graphe SLG. Notre système sans grammaire obtient des résultats comparables aux systèmes spécialisés de l’état de l’art
As an appealing topic in pattern recognition, handwritten mathematical expression recognition exhibits a big research challenge and underpins many practical applications. Both a large set of symbols (more than 100) and 2-D structures increase the difficulty of this recognition problem. In this thesis, we focus on online handwritten mathematical expression recognition using BLSTM and CTC topology, and finally build a graph-driven recognition system, bypassing the high time complexity and manual work in the classical grammar-driven systems. To allow the 2-D structured language to be handled by the sequence classifier, we extend the chain-structured BLSTM to an original Tree-based BLSTM, which could label a tree structured data. The CTC layer is adapted with local constraints, to align the outputs and at the same time benefit from introducing the additional ’blank’ class. The proposed system addresses the recognition task as a graph building problem. The input expression is a sequence of strokes, and then an intermediate graph is derived considering temporal and spatial relations among strokes. Next, several trees are derived from the graph and labeled with Tree-based BLSTM. The last step is to merge these labeled trees to build an admissible stroke label graph (SLG) modeling 2-D formulas uniquely. One major difference with the traditional approaches is that there is no explicit segmentation, recognition and layout extraction steps but a unique trainable system that produces directly a SLG describing a mathematical expression. The proposed system, without any grammar, achieves competitive results in online math expression recognition domain

En esta tesis se presenta un nuevo marco interactivo y multimodal para la transcripción de Documentos manuscritos. Esta aproximación, lejos de proporcionar la transcripción completa pretende asistir al experto en la dura tarea de transcribir. Hasta la fecha, los sistemas de reconocimiento de texto manuscrito disponibles no proporcionan transcripciones aceptables por los usuarios y, generalmente, se requiere la intervención del humano para corregir las transcripciones obtenidas. Estos sistemas han demostrado ser realmente útiles en aplicaciones restringidas y con vocabularios limitados (como es el caso del reconocimiento de direcciones postales o de cantidades numéricas en cheques bancarios), consiguiendo en este tipo de tareas resultados aceptables. Sin embargo, cuando se trabaja con documentos manuscritos sin ningún tipo de restricción (como documentos manuscritos antiguos o texto espontáneo), la tecnología actual solo consigue resultados inaceptables. El escenario interactivo estudiado en esta tesis permite una solución más efectiva. En este escenario, el sistema de reconocimiento y el usuario cooperan para generar la transcripción final de la imagen de texto. El sistema utiliza la imagen de texto y una parte de la transcripción previamente validada (prefijo) para proponer una posible continuación. Despues, el usuario encuentra y corrige el siguente error producido por el sistema, generando así un nuevo prefijo mas largo. Este nuevo prefijo, es utilizado por el sistema para sugerir una nueva hipótesis. La tecnología utilizada se basa en modelos ocultos de Markov y n-gramas. Estos modelos son utilizados aquí de la misma manera que en el reconocimiento automático del habla. Algunas modificaciones en la definición convencional de los n-gramas han sido necesarias para tener en cuenta la retroalimentación del usuario en este sistema.
Romero Gómez, V. (2010). Multimodal Interactive Transcription of Handwritten Text Images [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8541
Palancia

The use of digital images of handwritten historical documents has increased in recent years. This has been possible through the Internet, which allows users to access a vast collection of historical documents and makes historical and data research more attainable. However, the insurmountable number of images available in these digital libraries is cumbersome for a single user to read and process. Computers could help read these images through methods known as Optical Character Recognition (OCR), which have had significant success for printed materials but only limited success for handwritten ones. Most of these OCR methods work well only when the images have been preprocessed by getting rid of anything in the image that is not text. This preprocessing step is usually known as binarization. The binarization of images of historical documents that have been affected by degradation and that are of poor image quality is difficult and continues to be a focus of research in the field of image processing. We propose two novel approaches to attempt to solve this problem. One combines recursive Otsu thresholding and selective bilateral filtering to allow automatic binarization and segmentation of handwritten text images. The other adds background normalization and a post-processing step to the algorithm to make it more robust and to work even for images that present bleed-through artifacts. Our results show that these techniques help segment the text in historical documents better than traditional binarization techniques.

Thesis (MEng)--University of Stellenbosch, 2000.
ENGLISH ABSTRACT: Handwritten signature verification (HSV) is the process through which handwritten signatures are analysed in an attempt to determine whether the person who made the signature is who he claims to be. Banks and other financial institutions lose billions of rands annually to cheque fraud and other crimes that are preventable with the aid of good signature verification techniques. Unfortunately, the volume of cheques that are processed precludes a thorough HSV process done in the traditional manner by human operators. It is the aim of this research to investigate new methods to compare signatures automatically, to eventually speed up the HSV process and improve on the accuracy of existing systems. The new technology that is investigated is the use of the so-called hidden Markov models (HMMs). It is only quite recently that the computing power has become commonly available to make the real-time use of HMMs in pattern recognition a possibility. Two demonstration programs, SigGrab and Securitlheque, have been developed that make use of this technology, and show excellent improvements over other techniques and competing products. HSV accuracies in excess of99% can be attained.
AFRIKAANSE OPSOMMING: Handgeskrewe handtekening verifikasie (HHV) is die proses waardeur handgeskrewe handtekeninge ondersoek word in 'n poging om te bevestig of die persoon wat die handtekening gemaak het werklik is wie hy voorgee om te wees. Banke en ander finansiele instansies verloor jaarliks biljoene rande aan tjekbedrog en ander misdrywe wat voorkom sou kon word indien goeie metodes van handtekening verifikasie daargestel kon word. Ongelukkig is die volume van tjeks wat hanteer word so groot, dat tradisionele HHV deur menslike operateurs 'n onbegonne taak is. Dit is die doel van hierdie navorsmg om nuwe metodes te ondersoek om handtekeninge outomaties te kan vergelyk en so die HHV proses te bespoedig en ook te verbeter op die akkuraatheid van bestaande stelsels. Die nuwe tegnologie wat ondersoek is is die gebruik van die sogenaamde verskuilde Markov modelle (VMMs). Dit is eers redelik onlangs dat die rekenaar verwerkingskrag algemeen beskikbaar geraak het om die intydse gebruik van VMMs in patroonherkenning prakties moontlik te maak. Twee demonstrasieprogramme, SigGrab en SecuriCheque, is ontwikkel wat gebruik maak van hierdie tegnologie en toon uitstekende verbeterings teenoor ander tegnieke en kompeterende produkte. 'n Akkuraatheid van 99% of hoer word tipies verkry.

This thesis describes an important step towards the standardization of the research on Optical Character Recognition (OCR) in Farsi language. It includes the development of several novel and standard Farsi handwritten databases, consisting of Farsi isolated digits, isolated letters, numerical strings, legal amounts on cheques, dates, and English isolated digits. Despite conventional research and an Internet search, to the best of our knowledge, no publicly accessible handwritten Farsi database exists that is available to researchers. In a character recognition system, three data sets are usually required: (1) Training set for training the classifier using designed features, (2) Verifying set for checking and adjusting the designed system, (3) Testing set to finally measure the performance of the system. To cover all the specified requirements, all our databases contain complete sets of training, testing, and verifying samples. Data entry forms were used for collecting handwritings. To process those forms, some form processing techniques were used to automate the process of extracting images of different fields in the forms, and to segment the numerical strings into isolated digits. Included in this thesis, is the implementation of a recognition system for recognizing our handwritten Farsi isolated digits database which may be used for comparison with the results of future research. For this recognition system, we used three feature sets including outer profiles, crossing counts and projection histograms; and for classification we used Support Vector Machines with an RBF kernel which gave us a recognition rate of 97.46% on our Testing Set. We also applied a rejection method to our system, which could improve the error rate by 1.18% by a rejection rate of 2.94%

Thesis (M.S.)--University of Wisconsin--Madison, 1991.
Typescript. eContent provider-neutral record in process. Description based on print version record. Includes bibliographical references (leaf 22).

碩士
國立成功大學
工程科學系
84
Owing to large amount of Chinese character set, the recognition of handwritten Chinese character is difficult. In order to overcome this problem, many kinds of preclassification schemes have been proposed. Because the Chinese character is composed of main radicals, subradicals and remaining strokes, the recognition methods based on radicals embedded in Chinese characters are intuitive and commonly used. In this paper we modify the 46 set of radicals defined in Ching- Song input method to satisfy the requirements of optical character recognition and introduce the structural information of radicals. After the preprocessing and segment extraction steps, the proposed dividing algorithm is used to separate the thinned character image into several independent substructures of strokes, according to the connectivity among strokes. Under the guidance of structural model, that is composed of 1-D relation string and structural information of radicals, the predefined stroke sequence is searched from the candidate strokes by a depth first searching method. Not only can the method we proposed above recognize the radical sets embedded in a character, but also accomplish the preclassification work at the same time. Our method has following advantages: (1)only a little amount of defined radical needed and less strokes for each radical; (2)the recognition speed is fast; (3)the structural information of radicals is salient and stable.

碩士
大同大學
資訊工程研究所
90
In preparing the entrance examination, the registration number of examinee on the answer sheet should be sealed in advance. After the teacher graded the score on the answer sheet, the operator takes the seal away and records the score into the database. In order to reduce the processing time and cost, we propose a score recording system which uses the bar code to represent the registration number of examinee and recognizes and records the handwritten score automatically. In this thesis, we use an approach which extracts areas having high density of mono-oriented gradient to locate the position of bar code. We use multiple features to recognize the score, and offer a confidence index sound to inform the operator how the system rate the confidence of recognition. According to the confidence index, the operator can determine if confirmation for the score on the answer sheet is required. Experimental results show our system not only can locate bar code accurately, but also can achieve the demand on recognizing the score. Keywords: handwritten numeral recognition, bar code location

Optical character recognition (OCR) plays a very vital role in today’s modern world. OCR can be useful for solving many complex problems and thus making human’s job easier. In OCR we give a scanned digital image or handwritten text as the input to the system. OCR can be used in postal department for sorting of the mails and in other offices. Much work has been done for English alphabets but now a day’s Indian script is an active area of interest for the researchers. Devanagari is on such Indian script. Research is going on for the recognition of alphabets but much less concentration is given on numerals. Here an attempt was made for the recognition of Devanagari numerals. The main part of any OCR system is the feature extraction part because more the features extracted more is the accuracy. Here two methods were used for the process of feature extraction. One of the method was moment based method. There are many moment based methods but we have preferred the Tchebichef moment. Tchebichef moment was preferred because of its better image representation capability. The second method was based on the contour curvature. Contour is a very important boundary feature used for finding similarity between shapes. After the process of feature extraction, the extracted feature has to be classified and for the same Artificial Neural Network (ANN) was used. There are many classifier but we preferred ANN because it is easy to handle and less error prone and apart from that its accuracy is much higher compared to other classifier. The classification was done individually with the two extracted features and finally the features were cascaded to increase the accuracy.

Devanagari, the most accepted script in India and Hindi is the only dialect which is widely spoken and written, so Handwritten Hindi character Recognition is done. Optical Character Recognition (OCR) is used for pattern recognition, it can be online or offline. Handwritten text is electronically converted into machine learning language. Handwritten character Recognition has many applications like cheque reader,passport reader,address reader,specific tasks readers. Devanagari is troublesome because the characters present in a words are somewhat similar to other character or connected words may have problem in recognition as number of modifiers are present. The major challenge faced was removal of header line as header line cannot be always straight as it varies from person to person. The characters which are handwritten will not always have sharp corners, the header lines present will not be perfectly straight and the curves which are present will not be so smooth. Handwritten character recognition undergo three major steps (i) Pre-Processing (ii) Feature Extraction(iii)Classification. Pre processing is the first step which deals with binarization, noise removal, morphological operations and segmentation. Segmentation is major part in character recognition. Words are segmented into single single characters and these segmented characters are used for feature extraction. In second step Histogram of Oriented Gradients (HOG) is used as extraction of feature in an image so as to obtain the feature vector .Object detection can be easily done by using HOG in image processing and computer vision. HOG has intensity values which is obtained by gradient computation and will give rough idea of shape or pattern of an image. Last step concludes with classification, for classifying the samples Support Vector machine is implemented. SVM is basically used as binary classifier but in this project it has been used as Multiclass Classifier (One Vs. All). SVM constructs a hyper plane as data points are mapped into higher D-dimensional space. Non Linear SVM includes various kernels like polynomial kernel, radial basis kernel for mapping the data into higher D-dimensional space. The performance analysis is efficient for the kernels which are used. Accuracy rate can be improved for segmentation by using various other methods for segmentation. It can be extended to work on degraded text or broken characters and conversion of text to speech. Online recognition of character can be done.

In this report, we review different techniques for handwritten numeral recognition. More importantly we develop and test a hand-written numeral recognition system using multiwavelets. Given a black-and-white numeral, we first trace the contour of the numeral. Secondly we normalize and resample the contour points. Thirdly we perform multiwavelet orthonormal shell expansion on the contour points and we get several resolution levels and the average. We use the multiwavelet coefficients as the features to recognize the hand-written numerals. We use the L1 distance as a measure and the nearest neighbour rule as classifier for the recognition. The experimental result shows that it is a feasible way to use multi-wavelet features in handwritten numeral recognition.

碩士
國立高雄第一科技大學
電腦與通訊工程系
90
This thesis brings up the implementation of handwritten ID number recognition system by the application of plastic perceptron neural network（PPNN）. The applied structure of PPNN in this thesis is improved from the learning algorithm and network structure of back-propagation neural network （BPNN）in artificial neural networks. The problems of traditional BPNN such as longer learning period, not prone to convergence, re-training while delete or add new patterns make the realization of real time BPNN system impossible. The proposed methods are combined with the parallel distributive process concept and modification of the BPNN structure could accelerate the learning speed and solve the re-training problem. The character segmentation, noise removal and extraction of feature are also discussed. Adequate extracted feature make recognition of character easier. The adoption of white run-length and pixel density could clearly display the structural and integral of the character respectively, and facilitate to make higher recognition accuracy.

Most software programs that deal with mathematical objects require input expressions to be linearized using somewhat awkward and unfamiliar string-based syntax. It is natural to desire a method for inputting mathematics using the same two-dimensional syntax employed with pen and paper, and the increasing prevalence of pen- and touch-based interfaces causes this topic to be of practical as well as theoretical interest. Accurately recognizing two-dimensional mathematical notation is a difficult problem that requires not only theoretical advancement over the traditional theories of string-based languages, but also careful consideration of runtime efficiency, data organization, and other practical concerns that arise during system construction. This thesis describes the math recognizer used in the MathBrush pen-math system. At a high level, the two-dimensional syntax of mathematical writing is formalized using a relational grammar. Rather than reporting a single recognition result, all recognizable interpretations of the input are simultaneously represented in a data structure called a parse forest. Individual interpretations may be extracted from the forest and reported one by one as the user requests them. These parsing techniques necessitate robust tree scoring functions, which themselves rely on several lower-level recognition processes for stroke grouping, symbol recognition, and spatial relation classification. The thesis covers the recognition, parsing, and scoring aspects of the MathBrush recognizer, as well as the algorithms and assumptions necessary to combine those systems and formalisms together into a useful and efficient software system. The effectiveness of the resulting system is measured through two accuracy evaluations. One evaluation uses a novel metric based on user effort, while the other replicates the evaluation process of an international accuracy competition. The evaluations show that not only is the performance of the MathBrush recognizer improving over time, but it is also significantly more accurate than other academic recognition systems.

碩士
大同工學院
資訊工程研究所
81
This paper presents a methodology for classifying syntactic patterns is using a feature matching against a set of proto- otypes. The prototypes are first classified and arranged into a hierarchical structure that facilitates this matching. Image of characters are described by a sequence of features extracted from the chain codes of their contours. A rotatio- nally invariant string distance measure is defined that com- pared two feature strings. The methodology discussed in this paper is compared to a nearest neighbor classifier that use 2,010 prototypes. The proposed technique can get a recognit- ion rate of greater than 97 percent, and the recognition sp- eed is 0.5 sec/char.

碩士
國立中央大學
資訊工程研究所
98
This paper proposes a video-based handwritten signature verification framework. When acquiring signature information, we use a webcam in substitution for a digitizing tablet. Because webcams are more prevalent and cheaper than digitizing tablets, using webcams as sensors can reduce the cost. In addition, the features extracted using a webcam also contain more information. In tradition handwritten signature verification, features extracted using a digitizing tablet are mainly trajectories. But for the features extracted using a webcam, we can acquire pen grasping posture information of the subscriber in addition to the trajectories of the signature. Therefore, in the proposed framework, we perform video-based handwritten signature verification using two different types of feature information. For the first type of feature, we perform curvelet transform on the subscriber’s writing trajectory to obtain static information. The second type of feature is dynamic information which is the pen grasping posture of the subscriber. The dynamic feature is represented by motion energy image (MEI). We cascade the classifiers using static information and dynamic information to perform handwritten signature verification. The proposed video-based handwritten signature verification framework achieves a low false acceptance rate of 0% and false rejection rate 0.5% for our handwritten signature database without imitation signatures. For the database with imitation signatures, the proposed framework can also achieve a low false acceptance rate of 0.05%.

碩士
國立臺灣師範大學
工業教育學系在職進修碩士班
91
Recognition of off-line handwritten numerals has been the subject of research for many years. Since handwritten numerals widely vary in their shapes, recognizing them has been difficult and challenging. Although a high level of recognition has been achieved, the shortcomings of time-consuming learning and recognition still persist. The present research focuses on overcoming these defects, while maintaining a high recognition level. The research discussed in the present paper makes use of the MNIST database for learning and testing. For feature extraction, statistic features are used in the present research. Employing statistic features is saddled with the difficulty of a high number of dimensions, yet the present research, by using 130 dimensions, is able to distinguish between ten classifications. To make character recognition more effective, in the present research transformation by Fisher''s LDF (linear discriminant function) is applied to input characters. As experiments have shown, after transformation of non-clustered features (without learning) a level of recognition of 92.6% is achieved. In the present research, the method of WGLVQ, which is based on GLVQ (generalized learning vector quantization), is employed. Better convergence is achieved by GLVQ, and it is able to improve for LVQ. Experiments conducted within the current research have shown that both LVQ and GLVQ, applied to recognizing handwritten numerals, have quite good convergence behavior, also confirming the effectiveness of feature processing presented here. In the present research, the methods of LVQ and GLVQ are enhanced by weighting, yielding novel methods of WLVQ and WGLVQ. Therein, in every learning step, not only directions classifying reference vectors are adjusted, but also weights of every vector. With every step the weights of less-weighted vectors decrease, resulting in more pronounced distinctions of light and heavy weights. According to experiments, both WLVQ and WGLVQ exhibit more effective character recognition. With classification by WGLVQ and including learning, in an open test a level of recognition of 97.6% is achieved. With 16 clusters for each class, the recognition level rises to 98.2%. This result trails the level of 99.3% attained by Ernst using classification by LIRA, but while recognizing 10000 samples takes 30 minutes for the LIRA’s classification, the present approach allows recognition of 10000 samples in 1 - 2 minutes. The present research offers a more practical approach.

abstract: Handwritten documents have gained popularity in various domains including education and business. A key task in analyzing a complex document is to distinguish between various content types such as text, math, graphics, tables and so on. For example, one such aspect could be a region on the document with a mathematical expression; in this case, the label would be math. This differentiation facilitates the performance of specific recognition tasks depending on the content type. We hypothesize that the recognition accuracy of the subsequent tasks such as textual, math, and shape recognition will increase, further leading to a better analysis of the document. Content detection on handwritten documents assigns a particular class to a homogeneous portion of the document. To complete this task, a set of handwritten solutions was digitally collected from middle school students located in two different geographical regions in 2017 and 2018. This research discusses the methods to collect, pre-process and detect content type in the collected handwritten documents. A total of 4049 documents were extracted in the form of image, and json format; and were labelled using an object labelling software with tags being text, math, diagram, cross out, table, graph, tick mark, arrow, and doodle. The labelled images were fed to the Tensorflow’s object detection API to learn a neural network model. We show our results from two neural networks models, Faster Region-based Convolutional Neural Network (Faster R-CNN) and Single Shot detection model (SSD).
Dissertation/Thesis
Masters Thesis Computer Science 2018

碩士
淡江大學
電機工程學系研究所
86
In this thesis , we propose a new scheme for off-line recognition of totallyun constrained handwritten characters using SOM/LVQ neural networks and extractio ngfeature vectors by kirsch masks method. In the learning phase, the SOM neura l networks is used to cluster the feature vectors into several classes. In the recognitionstage, the learning results of the neural networks are utilized to identify the inputdata. In order to seek the optium cluster set, the resultin g clusters from the SOMneural networks need to be refined such that the hetero geneity among different targetscan be increased. This is done by introducing a supervised refining algorithm. We have chosen the supervied version of Kohone n''s model known as the Learning vector Quantization to refine selected feature s.The proposed scheme consists of two stages: a feature extraction stage for e xtractingfour-directional local feature vectors with Kirsch masks and one glob al feature vectorform compution the density over small regions of the image, a nd a classification stagefor recognizing characters with SOM neural networks. We first use the Kohonen clusteringnetworks (SOM) to represent the training da ta with minimum quantization error whilemaximizes the within-target homogeneit y. We then use the LVQ to learn the between-targetheterogeneity. It is done by collecting those selected neurons as the inital cluster centers for the LVQ t o learn their class boundaries. This is maximize the probability of correct cl assification. In order to verify the performance of the proposed approach, 630 0 handwritten characters written by 70 persons were collected as the database, 2000characters are used as the training set and the other 4300 characters as the testing set.Some experimental results are conducted to show the feasibilit y of our proposed method.

In this thesis, a classifier based on local sub-unit level and global character level representations of a character, using stroke direction and order variations independent features, is developed for recognition of Devanagari online handwritten characters. It is shown that online character corresponding to Devanagari ideal character can be analyzed and uniquely represented in terms of homogeneous sub-structures called the sub-units. These sub-units can be extracted using direction property of online strokes in an ideal character. A method for extraction of sub-units from a handwritten character is developed, such that the extracted sub-units are similar to the sub-units of the corresponding ideal character. Features are developed that are independent of variations in order and direction of strokes in characters. The features are called histograms of points, orientations, and dynamics of orientations (HPOD) features. The method for extraction of these features spatially maps co-ordinates of points and orientations and dynamics of orientations of strokes at these points. Histograms of these mapped features are computed in di erent regions into which the spatial map is divided. HPOD features extracted from the sub-units represent the character locally; and those extracted from the character as a whole represent it globally. A classifier is developed that models handwritten characters in terms of the joint distribution of the local and global HPOD features of the characters and the number of sub-units in the characters. The classifier uses latent variables to model the structure of the the sub-units. The parameters of the model are estimated using the maximum likelihood method. The use of HPOD features and the assumption of independent generation of the sub-units given the number of sub-units, make the classifier independent of variations in the direction and order of strokes in characters. This sub-unit based classifier is called SUB classifier. Datasets for training and testing the classifiers consist of handwritten samples of Devanagari vowels, consonants, half consonants, nasalization sign, vowel omission sign, vowel signs, consonant with vowel sign, conjuncts, consonant clusters, and three more short strokes with di erent shapes. In all, there are 96 di erent characters or symbols that have been considered for recognition. The average number of samples per character class in the training and the test sets are, respectively, 133 and 29. The smallest and the largest dimensions of the extracted feature vectors are, respectively, 258 and 786. Since the size of the training set per class is not large compared to the dimension of the extracted feature vectors, the training set is small from the perspective of training any classifier. classifiers that can be trained on a small data set are considered for performance comparison with the developed classifier. Second order statistics (SOS), sub-space (SS), Fisher discriminant (FD), feedforward neural network (FNN), and support vector machines (SVM) are the other classifiers considered that are trained with the other features like spatio-temporal (ST), discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete wavelet transform (DWT), spatial (SP), and histograms of oriented gradients (HOG) features extracted from the samples of the training set. These classifiers are tested with these features extracted from the samples of the test set. SVM classifier trained with DFT features has the highest accuracy of 90.2% among the accuracies of the other classifiers trained with the other features extracted from the test set. The accuracy of SUB classifier trained with HPOD features is 92.9% on the test set which is the highest among the accuracies of all the classifiers. The accuracies of the classifiers SOS, SS, FD, FNN, and SVM increase when trained with HPOD features. The accuracy of SVM classifier trained with HPOD features is 92.9%, which is the highest among the accuracies of the other classifiers trained with HPOD features. SUB classifier using HPOD features has the highest accuracy among the considered classifiers trained with the considered features on the same training set and tested on the same test set. The better character discriminative capability of the designed HPOD features is re ected by the increase in the accuracies of the other classifiers when trained with these features