In this paper we have considered training of recurrent neural network as nonlinear Bayesian estimation. The nonlinear Bayesian estimator is implemented as Gaussian Sum filter. The major drawback of the proposed algorithm is exponential growth of the number of components in the posterior density of the recurrent neural networks state vector.  In order to prevent exponential explosion, we have implemented an adaptive clustering algorithm which clusters the components of the mixture and replaces each cluster with a single Gaussian.