In recent years, the multi-cores and General-Purpose GPU (GPGPU) architectures have become general platforms for various of parallel applications, with lots of parallel algorithms being proposed for this interesting persperctive. In this report, we study and develop a particular kind of artificial neural network (ANN), in hopfield model, to solve some optimization problems, since it has a highly parallel nature. Section 1 introduces the context of the problem and the linked topics of this internship. In Section 2, we synthesize some previous work in this domain. In section 3, we study the parallel mechanism and propose the algorithm to realize this neural network. Section 4 interprets the method we use to map the neural network structure. Section 5 discusses the implementation details of simulations of hopfield networks on GPU. In section 6, we comment on the results and the performances compared with the CPU implementation.