High-speed parallel confocal detection method with an inverse pinhole array for three-dimensional color imaging

Abstract

Reflectance confocal microscopy is a widely used non-destructive optical imaging technique for three-dimensional (3D) surface measurement. An optically sectioned image with a high spatial resolution can be acquired by the pinhole structure. In confocal microscopy, a stack of two-dimensional (2D) images along the axial position is required to reconstruct 3D image. Thus, the speed of 3D volumetric imaging can be limited by the 2D beam scanning and the axial scanning that are generally achieved by the mechanical movement of the scanners. The parallelization of the beam with a pinhole array instead of transverse point scanning can be considered to achieve high-speed volumetric imaging. Since this approach can acquire the multiple points simultaneously, the mechanical scanning on the transverse direction can be removed. Also, the focus tunable lens (FTL) is a good alternative for mechanical axial scanning. A deformable surface of the FTL can regulate the focal length of the optical system by the electrical current with high-speed without mechanical scanning.

Previously, a direct-view confocal microscope with a FTL to produce a monochrome 3D surface profile of a specimen without any mechanical scanning was developed. To improve the signal-to-noise ratio (SNR) of the 2D image, the configurations of the optical system was improved by rejecting the internally reflected signals from the pinhole array. The color imaging with a white light-emitting diode (LED) and a color camera enables the expression of color information onto the surface profile. The filtering method both on the transverse and on the axial directions were applied to improve the image quality by reducing the effect of the cross-talk and the shot noise of the camera.

Here, a high-speed color 3D measurement method based on direct-view confocal microscopy with an inverse pinhole array which has an inverse characteristic of the beam transmission compared to the conventional pinhole array is proposed to solve the problem of the low transmission and the finite sampling points. The transmission of the beam can be raised up to about 3.3-folds theoretically when the imaging with the inverse pinhole array. This enables acquiring the images with higher brightness, or reducing the acquisition time with same brightness. Since the 4 different filtering masks are applied, 4-folds of height information and 3-folds of color information can be acquired with the inverse pinhole array compared to the imaging system with conventional pinhole array. The proposed method produces a color 3D image of a specimen using the 180 2D color images with an acquisition time of 1 second. The 3D image can be reconstructed by overlaying the color information onto the height map. The feasibility to application of the proposed method was demonstrated by imaging the various specimens. This high-speed non-contact color 3D measurement technique can be applicable for biomedical imaging, industrial measurement, and restoration fields where the acquisition of the rapid 3D surface and the color information are required.

|반사 공초점 현미경은 3차원 표면 측정을 위한 비파괴 광학 영상화 기법으로 널리 활용되고 있으며, 핀홀 구조에 의해 공간해상도가 높은 광학 단면 이미지을 얻을 수 있다. 공초점 현미경에서 3D 이미지를 획득하기 위해서는 축방향을 따라 획득한 2차원 이미지의 스택이 필요하기 때문에, 3차원 볼륨 이미징의 속도는 일반적으로 스캐너의 기계적 움직임에 의해 달성되는 2차원 빔 스캐닝 및 축방향 스캐닝에 의해 제한될 수 있다. 고속 볼륨 이미징을 달성하기 위하여 횡방향 스캐닝 대신 핀홀 어레이를 이용한 빔의 병렬화 방법이 고안되었으며, 이는 여러 포인트의 신호를 동시에 획득할 수 있어 횡방향의 기계적 스캐닝을 제거할 수 있다. 또한, 초점 가변 렌즈는 축방향 스캐닝을 대체할 수 있는 좋은 대안으로, 초점 가변 렌즈의 변형 가능한 표면은 전류에 의해 기계적 스캐닝 없이 고속으로 광학 시스템의 초점 위치를 조절할 수 있다.

선행연구로, 기계적 스캐닝 없이 시편의 단색조의 3차원 표면 프로파일을 생성하는 초점 가변 렌즈를 이용한 직시 공초점 현미경이 개발되었다. 2차원 이미지의 신호 대 잡음비를 개선하기 위해, 핀홀 에러이로부터 내부적으로 반사되는 신호를 제거할 수 있도록 광학 시스템의 구조를 개선하고, 백색 발광 다이오드와 컬러 카메라를 사용하여 컬러 이미징을 적용하여 3차원 표면 프로파일에 컬러 정보를 표현하는 연구가 진행되었다. 또한 횡방향과 축방향에 대해 각각 필터링을 적용하여 크로스 토크의 영향과 카메라의 샷 노이즈를 줄임으로써 이미지의 품질을 향상시킬 수 있었다.

본 연구에서 기존 연구의 낮은 투과율 및 샘플링 포인트의 제한 문제를 해결하기 위해 종래의 핀홀 어레이와 비교하여 빔 투과에 대해 역 특성을 갖는 역 핀홀 어레이를 이용한 직시 공초점 현미경에 기반한 고속 컬러 3차원 측정 방법이 제안되었다. 역 핀홀 어레이로 이미징을 할 때 빔의 투과율은 이론적으로 약 3.3배까지 상승되었으며, 이를 통해 더 밝은 이미지를 획득하거나, 동일한 밝기의 이미지를 획득하기 위한 시간을 줄일 수 있다. 4개의 상이한 필터링 마스크를 이용한 필터링의 적용으로, 역 핀홀 어레이를 통해 종래의 핀홀 어레이를 이용한 이미징 시스템에 비해 4배의 높이 정보와 3배의 컬러 정보를 획득할 수 있다. 제안된 방법은 1초동안 180장의 2차원 컬러 이미지를 획득하여 시편의 3차원 컬러 이미지를 생성할수 있으며, 색상 정보는 높이 맵에 오버레이하여 3차원 이미지를 표현한다. 시편의 이미징을 통해 제안된 방법의 적용 가능성을 확인하였다. 제안된 비접촉 방식의 고속 컬러 3차원 측정 기술은 빠른 속도의 3차원 표면 정보 및 색상 정보 표현이 필요한 생체 의료 영상, 산업용 측정, 복원 분야 등에 적용할 수 있을 것으로 기대된다.; Reflectance confocal microscopy is a widely used non-destructive optical imaging technique for three-dimensional (3D) surface measurement. An optically sectioned image with a high spatial resolution can be acquired by the pinhole structure. In confocal microscopy, a stack of two-dimensional (2D) images along the axial position is required to reconstruct 3D image. Thus, the speed of 3D volumetric imaging can be limited by the 2D beam scanning and the axial scanning that are generally achieved by the mechanical movement of the scanners. The parallelization of the beam with a pinhole array instead of transverse point scanning can be considered to achieve high-speed volumetric imaging. Since this approach can acquire the multiple points simultaneously, the mechanical scanning on the transverse direction can be removed. Also, the focus tunable lens (FTL) is a good alternative for mechanical axial scanning. A deformable surface of the FTL can regulate the focal length of the optical system by the electrical current with high-speed without mechanical scanning.

Previously, a direct-view confocal microscope with a FTL to produce a monochrome 3D surface profile of a specimen without any mechanical scanning was developed. To improve the signal-to-noise ratio (SNR) of the 2D image, the configurations of the optical system was improved by rejecting the internally reflected signals from the pinhole array. The color imaging with a white light-emitting diode (LED) and a color camera enables the expression of color information onto the surface profile. The filtering method both on the transverse and on the axial directions were applied to improve the image quality by reducing the effect of the cross-talk and the shot noise of the camera.

Here, a high-speed color 3D measurement method based on direct-view confocal microscopy with an inverse pinhole array which has an inverse characteristic of the beam transmission compared to the conventional pinhole array is proposed to solve the problem of the low transmission and the finite sampling points. The transmission of the beam can be raised up to about 3.3-folds theoretically when the imaging with the inverse pinhole array. This enables acquiring the images with higher brightness, or reducing the acquisition time with same brightness. Since the 4 different filtering masks are applied, 4-folds of height information and 3-folds of color information can be acquired with the inverse pinhole array compared to the imaging system with conventional pinhole array. The proposed method produces a color 3D image of a specimen using the 180 2D color images with an acquisition time of 1 second. The 3D image can be reconstructed by overlaying the color information onto the height map. The feasibility to application of the proposed method was demonstrated by imaging the various specimens. This high-speed non-contact color 3D measurement technique can be applicable for biomedical imaging, industrial measurement, and restoration fields where the acquisition of the rapid 3D surface and the color information are required.