Robot Grasping - Setting Reseting Environment

Motivation

In today’s fast-moving production environments, picking and packing operations demand a lot from human operators including uninterrupted speed, reliability, inspection, sorting, accuracy, and dexterity. In these repetitive tasks, robots become superior and cheap with the fact that they can complete these tasks consistently at high speed without the need for breaks.

We have seen picking and placing robots taking place at tasks like wholesale distribution, assembly, medical application, and even tasks in hazardous environments. In this final project, we have chosen the empty container task to validate and strengthen our understanding of the intelligent robot for picking and placing solutions.

Problem Statment

The aim of our project was to tackle real-life scenarios and challenges observed for grasping objects by a robotic arm. The most important requirement is to perform a successful grasp irrespective of the orientation of the object. We devised our problem statement to achieve a successful grasp by using vision feedback and implementing Q-learning algorithm for cases where cameras are not available.
We simulated our algorithms in Coppelia Sim on Franka Panda 7DOF robotic manipulator. We also implemented basic maneuvers on an actual robot.

Vision Based Approach

We present an overall overview of the solutions we implemented to solve the picking and placing task. Firstly, we obtain the noisy pose information about the source and target bins from the pose server. Then, we tried to locate the object using the cameras and if the object is detected to be too close to the edge, we pushed it toward the middle of the container to make the gripping task easier. And then, we obtain

the orientation of the object and tried to grasp it. By confirming the gripping state by depth information, we then move the arm and object to the target bin and release the gripper. We repeat the above operations until there is no more object in the source bin and we then reset the environment by interchange the definition of source and target bins.

Moving object to bin

We accessed the position of the bins from the noisy pose sensor and moved the arm above the bin to get a good vantage point of all the objects placed in the bin.

Object detection and segmentation

After the robot moves to the vantage point of the bin, it uses object segmentation based on colors and only captures small objects close to the center view in the image view and ignores segments that are too big or close to edges of the view. In the above figure, the robot is able to filter just only the objects (grayscale images). If there are found objects, the robot will move the next stage. If not, the whole process is completed.
After object detection, the robot finds the contour of the objects (red lines on the left images), and locate the one closest to the center of the image view to pick up and draw a bounding box for it.

Aligning gripper to object

With the bounding box of the object from the previous step, the robot will first move close to the bounding box so that the center of the bounding box is at the center of the image view. Then the gripper rotates so that the bounding box is vertically aligned with the image view as in the figure above. The gripper will do final adjustment if the box is not at the center after the rotation, then it executes the grasping from above to the object. After grasping, the gripper pulls back to the vintage point and checks whether the object is in the gripper using depth images. If the object is in the gripper, it moves to the target bin location and drops the object. If the grasping fails, it will retry again with the horizontal alignment of the bounding box to the gripper and repeats the cycle of alignments until it can grasp it.

Container edge detection

We tried to detect the edge of the container firstly using canny edge detection and shape detection, but the performance is poor with the fact that the container edge is often occluded when the arm is moving around. Then, we tried to use the segmentation mask provided by the RL bench and use breadth-first search from the object center location to identify the closest edge around it, with its orientation relative to the object. And if the squared pixel-wise euclidean distance is too large, we will then enter the gripper state to push the object into the middle

Pushing objects at edges

An edge detection algorithm detects the edges of the object and the bin. It calculates the minimum distance between the edges in pixel values and also the direction of the edge with respect to the object. If this distance is less than the programmed threshold, the arm performs the push maneuver. During this motion, the arm moves towards the object as mentioned above but instead of grasping, it just places its gripper on the object. Once this is achieved, the joint torques and increases to put normal force on the object. Then the arm is slowly moved in the direction opposite to that of the edge thus moving the object away from the edge.

Reinforcement Learning Approach

The reward function of RLBench is very sparse that it only gives a reward of 1 when the whole task is complete. The robot cannot learn anything useful with it. As a result, we had to implement a custom reward function that gives the robot a reward for every action so that it would learn better. Our learning criteria is

• Distance between the objects and the target bin.

• Distance between the gripper and the objects.

• Desired orientation of the gripper.

• Gripper state (reward if the gripper is closed).

• Reward for object in gripper.

With these criteria and Q-Learning, the robot is able to learn to move close to the object that is closest to the target bin. However, it stucks at that location without being able to properly grasp the object probably due to hitting the local maxima point.

Future Work

As currently, our robot can only move close to an object with Q-Learning, our next goal is to remodel our reward function so that the robot can grasp the object and move to the target bin to drop it

When the object is near the edge, the arm tries to push it away. However, sometimes it is not able to reach the object as the bin edge comes in collision with the gripper. A mechanism to orient the gripper ideally parallel to edge should be implemented to avoid the collision. Another solution is to approach the object from the inner side of the bin to avoid the edges all together.

The arm tries to drag the object away from the edge once it applies the normal force. But sometimes the object slips away and is not dragged successfully. Implementing a feedback loop to check if the object is getting pushed successfully will result in a more robust push.

We could also use vision sensor to detect the bin location which is currently obtained from the pose server to make the solution purely vision-based. This could be implemented by leveraging multiple object and shape detection methods to find the largest rectangular shape in the segmentation hierarchy.