2D/3D Vision-Based Mango's Feature Extraction and Sorting

Abstract

This paper describes a vision system that can extract 2D and 3D visual properties of mango such as size (length, width, and thickness), projected area, volume, and surface area from images and use them in sorting. The 2D/3D visual properties are extracted from multiple view images of mango. The images are first segmented to extract the silhouette regions of mango. The 2D visual properties are then measured from the top view silhouette as explained by Yimyam et al. (2005). The 3D mango volume reconstruction is done using volumetric caving on multiple silhouette images. First the cameras are calibrated to obtain the intrinsic and extrinsic camera parameters. Then the 3D volume voxels are crafted based on silhouette images of the fruit in multiple views. After craving all silhouettes, we obtain the coarse 3D shape of the fruit and then we can compute the volume and surface area. We then use these features in automatic mango sorting which we employ a typical backpropagation neural networks. In this research, we employed the system to evaluate visual properties of a mango cultivar called "Nam Dokmai". There were two sets total of 182 mangoes in three various sizes sorted by weights according to a standard sorting metric for mango export. Two experiments were performed. One is for showing the accuracy of our vision-based feature extraction and measurement by comparing results with the measurements using various instruments. The second experiment is to show the sorting accuracy by comparing to human sorting. The results show the technique could be a good alternative and more feasible method for sorting mango comparing to human's manual sorting.