This paper presents an approach for carrying out gesture recognition for the Brazilian Sign Language Manual Alphabet. The gestural patterns are treated as a combination of three primitives, or cheremes-hand configuration, hand orientation and hand movement. The recognizer is built in a modular architecture composed by inductive reasoning modules, which use the artificial neural network Fuzzy Learning Vector Quantization; and rule-based modules. This architecture has been tested and results are presented here. Some strengths of such approach are: robustness of recognition, portability to similar contexts, extensibility of the dataset to be recognize and reduction of the vocabulary recognition problem to the recognition of its primitives.