Some executed projects

Monocular computer vision for metal-block centering on a CNC gripper

May 16^th, 2024

I was contacted by an Italian metal-forming company to develop a computer vision system capable of determining whether a metal block was correctly centered between two bolts of a CNC gripper using a single camera.

I extended the original requirement beyond a simple aligned/misaligned classification. Using geometric projection principles inspired by epipolar geometry, I developed a method that estimates the degree of misalignment as a percentage. This continuous measurement provides substantially more information than a binary decision and enables the system to recalibrate itself with minimal human intervention.

The processing pipeline begins with a neural network that detects seven characteristic corners defining the position and orientation of the metal block, together with the two bolts used as alignment references. Based on these detected points, a geometric formulation projects the observed block position to estimate its desired centered location, shown in blue in the visualizations. The geometric formulation was developed independently from scratch, drawing on concepts related to epipolar and projective geometry and without LLM-based assistance.

The images below show the performance of the system on photographs collected directly from the production environment at the factory.

Computer vision shift estimation on factory photograph 1 of a metal block held by a CNC gripper — System performance on some of the images collected at the factory. The blue overlay represents the estimated desired centered position of the metal block, while red indicates a shift to the left of the target position and green a shift to the right.

Computer vision shift estimation on factory photograph 2 of a metal block held by a CNC gripper — System performance on some of the images collected at the factory. The blue overlay represents the estimated desired centered position of the metal block, while red indicates a shift to the left of the target position and green a shift to the right.

To evaluate the complete pipeline under real-time conditions, I also built a physical mockup reproducing the relevant geometry of the two reference bolts. This allowed me to test the CNN-based corner detection, real-time computation of the desired block position, and vision-based estimation of the displacement between the current and target positions using a live camera feed. Two examples of these experiments are shown below.

Real-time shift estimation of a metal block on a physical mockup, first video example — Demonstration of the computer vision system performing real-time shift estimation from a video stream captured on a geometrically similar physical mockup. The second video was compressed to reduce its size, which caused a slight loss of quality.

Real-time shift estimation of a metal block on a physical mockup, second video example — Demonstration of the computer vision system performing real-time shift estimation from a video stream captured on a geometrically similar physical mockup. The second video was compressed to reduce its size, which caused a slight loss of quality.

The prototype was successfully developed and validated using the dataset collected at the Italian factory. I did not continue with the deployment of the project because I moved to Prague to start studying my master's degree.