Abstract—Designing a structure of Recurrent Neural Network (RNN) can be performed either manually or automatically using a Neural Architecture Search (NAS). Manually designing an RNN structure can be a time consuming and error-prone process, whereas NAS uses optimization algorithms such as Evolutionary Algorithm (EA) to find the optimal structure. As one of the NAS methods, a Neuro Evolution of Augmenting Topology (NEAT) searches a neural network structure constructively by adding a new neuron through mutation, which is time-consuming for the RNN structures with large neurons. Genetic Algorithm (GA) is one of the EA that is commonly used to solve optimization problems. Fix length chromosome representation dominates GA filed, this representation suitable for fix length solution. Finding the optimal structure of RNN, the number of nodes and its connections cannot be determined. In this research, a Variable-Length Chromosome GA (VLC-GA) is exploited to represent RNN structure with a different number of nodes. VLCGA able to evolve the structure of RNN constructively and destructively. Therefore, the processing time for a larger structure can be reduced. Evaluation for a language modeling task shows that it is capable of reaching a lower perplexity than NAS.