Bir 3D sensör nedir?
Bir 3D sensör, bir objenin veya sahnenin mekansal yapısını ve yüzeyini algılar. x ve y yönündeki iki boyutlu yer bilgilerinin yanı sıra üçüncü bir uzay yönündeki (z) mesafeyi veya derinliği de belirler. Bu sayede örneğin bir derinlik haritası veya nokta bulutu şeklinde üç boyutlu veriler elde edilir. Buna dayanarak objelerin boyutları, konumları, hacimleri ve yüzey geometrisi belirlenebilir.
3D görüntü işleme ne zaman kullanılır?
2D görüntü işlemede parlaklık, renk ve kontur gibi özellikler tek bir düzlem içinde kaydedilir. Bir objenin konumu, yüksekliği veya şekli hakkında ek bilgilere ihtiyaç duyulduğunda 3D sensörler kullanılır. Ölçüm verileri 3D obje algılama, hacim belirleme, robotik yönlendirme ve otomatik kalite kontrolü sağlar. Tipik uygulama alanları arasında Bin Picking, Depalleting, robotik Pick-and-Place ve 3D yüzey denetimi yer alır.
2D görüntü algılama
3D Image Capture
Typical Applications with 3D Sensors
Bin Picking
In bin picking, a robot picks unordered objects from a bin or crate. A 3D sensor captures the position and orientation and generates a 3D point cloud of its visible surface. The robot system can then determine the suitable gripping position and remove the components in a targeted manner. This method is particularly suitable for objects with complex geometries or varying positions.
Paletten alma
3D sensörler, kartonların, konteynerlerin veya ürünlerin paletten alınmasını destekler. Mekansal veriler sayesinde yükseklik farkları, obje konumları ve istifleme desenleri algılanır, böylece objeler robot tarafından hassas bir şekilde alınabilir. Bu sayede düzensiz veya karışık yüklemeler de güvenilir şekilde işlenebilir.
3D yüzey denetimi
Çukurlar, deformasyonlar, montaj hataları ve yükseklik farkları, renk veya kontrasttan bağımsız olarak 3D yüzey denetimi ile hassas bir şekilde tespit edilebilir. Bu sayede üç boyutlu ölçüm verileri, otomatik kalite kontrolleri, boyutsal doğruluk ve eksiksizlik kontrolleri için uygundur.
3D görüntü kaydının temelleri
3D sensörler, ölçüm verilerini 3D nokta bulutu olarak sunar ve sensör modeline bağlı olarak ek yoğunluk değerleri verebilir. Endüstriyel 3D görüntü işlemede 2D/3D profil sensörleri ve anlık görüntü sensörleri kullanılır.
2D/3D profil sensörleri, ölçüm objesine bir lazer çizgisi yansıtır ve münferit 2D yükseklik profillerini algılar. Sensör ve obje arasındaki göreceli hareket sayesinde, birbiri ardına kaydedilen profiller eksiksiz bir 3D nokta bulutu oluşturur. Bu yöntem özellikle konveyör bantları veya üretim hatları gibi hareketli objelerin sürekli ölçümü için uygundur.
3D sensörler tüm ölçüm hacmini tek bir kayıtta algılar. Eksiksiz bir 3D nokta bulutu oluşturmak için, kayıt sırasında obje hareket etmemelidir. Bu yöntem, özellikle statik objelerin yüksek çözünürlüklü ölçümleri ve en ince yüzey ayrıntılarının tespiti için uygundur.
Coordinate Systems for Processing 3D Data
Coordinate systems are of central importance for the spatial processing of 3D data. The measuring points recorded by the 3D sensor refer to the sensor’s coordinate system. The sensor coordinate system defines the position of each measurement point in relation to the position and orientation of the sensor and forms the basis for further processing of the depth data.
For robots, plants or image processing systems to use the collected data, it must be transferred to a global or plant-related coordinate system. This allows object positions, movements and gripping points to be clearly determined within the application.
Hand-Eye Calibration for Robot Vision
For robot vision applications, the coordinate systems of the 3D sensor and the robot must be linked to each other. The hand-eye calibration determines the geometric transformation between the two systems. This allows the robot to interpret the 3D positions recorded by the sensor in its own coordinate system and move to them in a targeted manner.
Depending on the application, a distinction is made between two variants:
Eye-to-Hand
In the eye-to-hand procedure, the sensor is permanently mounted in the working area and observes the robot’s working range. The calibration determines its position and orientation relative to the robot base.
Eye-in-Hand
With eye-in-hand, on the other hand, the sensor is located directly on the robot arm and moves with it. Its position relative to the tool center point (TCP) is determined here.
Calibration takes place via defined reference objects or calibration patterns, which are recorded by both the robot system and the 3D sensor. The transformation between the coordinate systems is calculated from the captured positions. The detected image coordinates can then be transferred from the sensor coordinate system to robot coordinates. Even small deviations in the calibration can lead to position errors in the gripper control or path planning.
Hand-Eye Calibration with M3DS Series Sensors
Various project and calibration examples are available on GitHub for integration into robotics and image processing systems. These include reference implementations for hand-eye calibrations and transformations of 3D coordinate systems. This allows typical integration and calibration processes to be traced and the examples provided can be used directly for your own application.
Working Distance and Measuring Range of 3D Sensors
Working Distance
Measuring Range
The measuring range of a triangulation-based 3D sensor is shaped like a pyramid stump. As the distance to the sensor increases, the measuring field in the x- and y-direction increases. This allows larger objects or scenes to be captured in the far range.
At the same time, the measuring points are spread over a larger area, so that the distance between the neighboring points increases. At close range, the point density of a 3D point cloud is therefore higher and fine structures can be captured in more detail.
The depth resolution in the z-direction also depends on the working distance. As the distance increases, the geometric sensitivity of the triangulation decreases, so that small height differences in the near range can be detected better.
How Is a 3D Point Cloud Created?
3D sensors use structured light to project light patterns onto an object. The deformations of these patterns give conclusions about the contour of the object. A camera then captures the deformation of the pattern on the surface. From their displacement, the sensor uses optical triangulation to calculate the depth information and generates a 3D point cloud.
Step 1 – Projection of a light pattern
The built-in light engine projects sequentially defined light patterns onto the surface of an object. The pattern serves as a defined reference structure for the subsequent depth calculation. wenglor’s 3D sensors use blue light, as its shorter wavelength enables higher optical precision and at the same time is less sensitive to ambient light. This allows even fine surface structures to be captured in a stable and high-contrast manner.
Step 2 – Deformation of the Light Pattern on the Object
If the structured light pattern hits an object, it is distorted spatially depending on its geometry. Differences in height, edges and indentations cause the projected stripes in the camera image to shift. The deformation of the pattern contains information about the shape and depth of the object.
Smooth, textured or highly reflective surfaces can affect the quality of the pattern image and thus also affect the subsequent depth calculation.
Step 3 – Camera Captures the Light Pattern
The camera then captures the projected light patterns from a specified angle. During capture, the system analyzes the exact position and deformation of the projected patterns in the image. A complete data set of the scene is created from several recorded individual images. How clearly the patterns can be seen in the camera image depends, among other things, on the exposure time, surface reflection, ambient light and optical resolution of the sensor.
Step 4 – Calculate the Depth Value by Triangulation
The camera and the projector view the measurement object from different angles. Together with a surface point, they form a geometric triangle. Since the positions of the camera and projector are known to each other, the sensor can calculate the depth value of each individual measurement point from the shift of the recorded patterns.
This process is called optical triangulation and forms the basis for the spatial reconstruction of the visible object surface.
Step 5 – Creating the Point Cloud
Based on the camera calibration, the sensor generates a 3D point cloud from the calculated depth values. Each measuring point is described by its x-, y- and z-coordinates and thus has a unique position in the space. This point cloud thus represents the surface structure of the object true to detail.
Further information such as intensity or so-called confidence values can also be stored. While intensity data provides information about the brightness or reflectance properties, the confidence value indicates the quality or reliability of an individual measurement point.
Processing 3D Data
In structured light projection, different reconstruction methods can be used to generate 3D data. The stripe method and the phase method differ in the type of projected patterns and their evaluation. Both methods use gray code sequences, which allow the projected light patterns to be clearly assigned to the captured image points. This assignment forms the basis for calculating the depth values. Which method is suitable for a measurement task depends, among other things, on the required recording time, the condition of the object surface and reflections or ambient light.
Stripe and Phase Method Comparison
Stripe Method
With the stripe method, clearly separated and high-contrast light stripes of different widths are projected onto the object surface. Due to the high contrast, the projected patterns can be seen particularly well on dark or highly reflective surfaces. At the same time, the binary patterns allow very short exposure times, making the method particularly suitable for demanding surfaces.
Phase Method
The phase method projects sinusoidal light stripe patterns with different phase shifts onto the object surface. Instead of individual stripes, the continuous brightness distribution of the projected patterns is evaluated. This allows the position of a measurement point to be determined with sub-pixel accuracy, enabling particularly smooth, high-resolution and detailed 3D data.
Visualization of the 3D Point Cloud
False Color Representation
In the case of false color representation, different colors are assigned to the depth values of the 3D point cloud. This visualizes height differences and object contours, allowing quick interpretation of the measurement data.
Intensity Image
In addition to the depth values, the sensor can generate an intensity image. This corresponds to a classic gray value recording and contains information about the reflection properties of the surface.
Confidence Image
The confidence image shows the quality or reliability of individual measurement points within the point cloud. This allows areas with uncertain depth calculations or poor signal quality to be detected and evaluated in a targeted manner.
Recording Parameters of 3D Sensors
Exposure Time
Gain
Subsampling
Accuracy specifications according to VDI/VDE
Stack size
The stack size indicates how many individual images or pattern projections are used for a 3D image. A larger stack size provides more image information and can reduce measurement noise. At the same time, the recording time is extended, which reduces the maximum achievable capture speed.
Other 3D Technologies
Passive stereo
In this procedure, two cameras view the same object at an angle. By capturing the same features from both cameras, the spatial position of the feature in the room can be calculated from their image shift. The difficulty here is identifying the same point with both cameras. This method is suboptimal when viewing a low-contrast surface such as a white wall, for example.
Active stereo
The structure is the same as that of the passive stereo. In addition, a pattern (e.g. randomly distributed points) is projected onto the surface. This makes it easier to assign a point from both cameras.
Time-of-flight
In this procedure, the distance between the object and the sensor is determined based on the transit time. The sensor emits light pulses that hit an object and are reflected by it. The distance is determined depending on the duration of the reflection of the light pulses. This allows depth information such as structures or distances of objects to be determined.
Laser triangulation
In laser triangulation, a laser projects a line onto the object that is captured by a camera at a defined angle. The height of the object surface can be determined by the shift of the laser line in the camera image. As only one profile is captured at a time, either the object or the sensor must be moved to generate complete 3D data. The method is characterized by high accuracy, but is dependent on a defined relative movement during recording.
Comparison of Technologies
| Structured light | Passive stereo | Active stereo | Time of flight | |
|---|---|---|---|---|
| Resolution |  |
 |
|
 |
| Accuracy | |
|
|
|
| Ambient light | |
|
|
|
| Reading speed |  |
|
|
|
| Low-contrast objects | |
|
|
|
| Obstruction/shading | |
|
|
|
|
Resolution
|
|||
|---|---|---|---|
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
|
Accuracy
|
|||
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
|
Ambient light
|
|||
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
|
Reading speed
|
|||
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
|
Low-contrast objects
|
|||
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
|
Obstruction/shading
|
|||
|
Structured light
|
Passive stereo
|
Active stereo
|
Time of flight
|
Integration in Industrial Systems
For a 3D sensor to be used in industrial applications, it must be able to be seamlessly integrated into existing automation, robotics and image processing systems. In addition to the actual 3D data acquisition, standardized interfaces, software libraries and flexible integration options play a central role here. Modern 3D sensors offer extensive software and communication interfaces for this purpose, which enable easy integration into a wide range of system environments.
