Robotic Arm Control using Image Processing in MATLAB

When i was in third year (Kathmandu University, Undergraduate Course, EE) our group choose "Robotic Arm control using Image Processing" as our third year project. It was little bit challenge for us, but we finally did it. The basic idea was to pick up selected colored object(red, green, blue) from the board and drop it to the destination container. So our first task was to distinguished objects i.e. whether it was red, blue or green. We developed an image processing algorithm that distinguished red, blue or green colors from captured image of an object. So we captured image (camera was placed vertically upward) of that object placed in the board and fed that image into computer. Then processing algorithm in MATLAB processes the image to select which one to pick. 

 figure: Project platform

But that was no at all. We had to find the position of object; so we developed another algorithm to find center of any circular object. After finding the center of image, we used matching algorithm that matches the center pixel of required circular image with the coordinate of platform. 

Software and hardware part were done; Now we had to move our arm. We needed electrical/logical computer interface to control the arm of robot. We used parallel port that had to control four motors of robotic arm. Our interface circuit was shown below:

figure: interface circuit

The pins D0, D1, D1, D3 and D4 is from parallel ports. 74368 IC is for parallel port protection and logic inversion. There are two 3 by 8 decoders. Combination two outputs of different decoders with relay switching control the robotic arm. In circuit, Y0 of two decoder is controlling output1 (o/p1) and in similar way, Y1 is controlling o/p2, Y2 is controlling o/p3 and Y4 is controlling o/p4. The table showing input output circuit relation is as below.

D0	D1	D2	D3	D4	Motor\Direction
1	0	0	0	0	1st\right
0	1	0	0	0	1st\left
1	0	0	0	1	2nd\right
0	1	0	0	1	2nd\left
1	0	0	1	0	3rd\right
0	1	0	1	0	3rd\left
1	0	0	1	1	4th\right
0	1	0	1	1	4th\left
1	0	1	0	0	5th\right
0	1	1	0	0	5th\left

table: Logic of circuit diagram

The output D0 to D4 are suitable controlled by parallel port using predefined set of algorithm. So, to pick up any object from the desired location, we should give required logic from D0 to D4 for a predefined time. At a time one motor would move. So there was sequence of processes that would move different motors one after another.

  

figure: Robotic arm of project

We had set our logic in such a way that 'base' motor moves first in the desired line, than combination of 'Elbow' and 'Shoulder' takes arm to the destination. Then gripper grips the object and 'Elbow' and 'Shoulder' takes off the object. Then 'Base' moves to the container, and 'gripper' drops the object in the container. 

We had made this whole process to be controlled in MATLAB GUI. Our GUI looked like as shown below.

figure: GUI of robotic arm control

We was three control schemes, one is 'autonomous control' where we had to insert which color object to pick up and it would automatically pick up that object and drop down in the container. 'Manual control' scheme allowed control of Robotic arm by keyboard keys (keys from 0 to 9). 'Grid Control' was to pick up and drop down the object by giving its coordinates (A1,A2). Here you can watch the video of this control system.

video: Picking Green ball and dropping it in container

The project was a like dream project for me. I really enjoyed doing this, and learned a lot.