Some executed projects

Robot learning using DDQN and NEAT for a 2 DOF pointer robot

May 1^st, 2020

Just for fun, I wanted to build a robot that learns by experience. I designed and built a small robot that learns to control a laser beam using Deep Reinforcement Learning, Neuroevolution, and Computer Vision. The 2 DOF robot learned to point its laser beam to reach a target located at the center of two marks. It received the graphic information from a smartphone’s camera. Then, it identified the beam and marks position in real-time using computer vision. The high-level processing is done on a computer and the Arduino acts as the robot low-level controller. For the communication between these devices, I used my PyDuino Bridge Library, which is freely available for the community.

The algorithms tested were the Double Deep Q-Learning (DDQN) and NeuroEvolution of Augmenting Topologies (NEAT). A better performance was obtained using the first algorithm, whose results are shown below.

Physical setup of the pointer robot during its first trials — **First image:** Physical setup of the robot while executing its first trials (please ignore the little hole in my wall, as the robot does). **Second image:** Robot learning to move the beam to the desired position, which is the midpoint of the segment that links the red and green bottle caps (that target point is marked with a blue dot).

Pointer robot learning to move the laser beam to the midpoint between two bottle caps — **First image:** Physical setup of the robot while executing its first trials (please ignore the little hole in my wall, as the robot does). **Second image:** Robot learning to move the beam to the desired position, which is the midpoint of the segment that links the red and green bottle caps (that target point is marked with a blue dot).