You can click on any of the images to see a bigger version.
Just a brief note for some work in progress. The computer is actually playing the game. Sorry for the poor video.
The setup includes a Jetson TX2, a USB camera, an old iPhone, a stepper motor on the back of the iphone, a servo motor used to start the game, a solenoid used for the "fire" button, and a USB motor controller. Click on the image below for a bit clearer image.
I have the hardware built and running and enough software to do imitation learning. The robot plays the game with my corrections as feedback. The Jetson records the actions, corrections, and the camera feed. The data is exported to a Caffe setup running on the TX2 for training. The trained network is used for the next round. The robot slowly gets better so I have less and less to do. This process is slow and manually intensive but it works.
Next step is to convert the network to Tensorflow and try an Advantage Actor Critic (A2C) reinforcement setup. I do have a second network running in Tensor RT that can read the score off of the top left corner of the iphone so I can feed the scores back into A2C as a reward.
|
You can click on any of the images to see a bigger version. |