Self Modeling and Gait Control of Quadruped Robot Using Q-Learning Based Particle Swarm Optimization

Abstract

—In the realm of the living creature human and
animals create their body schema by the learning they gather
while they interact with the real world. They can also remodel
the schema if they have any uncertain changes in their body.
This kind of robustness is still not achieved by any machine or
artificial system. Researchers are trying to build the machines
resilient so that machines can explore the unknown space. In
this paper, we used Particle Swarm Optimization (PSO) which
a population based algorithm to allow a quadruped robot to
learn its body schema using a gyroscopic sensor and real world
interaction. We added Q- Value based learning (Q-Learning),s
an actor-critic scheme to aid PSO to learn faster and avoid
being trap in local optima. Robot creates an imaginary model
of its own body which include imaginary gaits using a very
little prior knowledge. The robot aims to use the gaits to
achieve stability and predictive movements. I can also detect
changes in its body and adopt the changes, which leads to a
damage diagnosis system. We tested the algorithm using
graphics simulator and verified using a 3D printed quadruped
robot with 12 actuators.