Self Modeling and Gait Control of Quadruped Robot Using Q-Learning Based Particle Swarm Optimization

Abstract

In the realm of the living creature human and animals create their body schema by the learning they gather while they interact with the real world. They can also remodel the schema if they have any uncertain changes in their body. This kind of robustness is still not achieved by any machine or artificial system. Researchers are trying to build the machines resilient so that machines can explore the unknown space. In this paper, we used Particle Swarm Optimization (PSO) which a population based algorithm to allow a quadruped robot to learn its body schema using a gyroscopic sensor and real world interaction. We added Q- Value based learning (Q-Learning),s an actor-critic scheme to aid PSO to learn faster and avoid being trap in local optima. Robot creates an imaginary model of its own body which include imaginary gaits using a very little prior knowledge. The robot aims to use the gaits to achieve stability and predictive movements. I can also detect changes in its body and adopt the changes, which leads to a damage diagnosis system. We tested the algorithm using graphics simulator and verified using a 3D printed quadruped robot with 12 actuators.