Convolutional neural networks to assess bergamot essential oil content in the field from smartphone images

The essential oil (EO) extracted from bergamot peel (Citrus bergamia, Risso et Poiteau) is appreciated in perfumery and gastronomy. Notably, 90 % of the bergamot EO production is concentrated in the Province of Reggio Calabria (Southern Italy) under a protected designation of origin (PDO). The early estimation of EO content in fruits is fundamental to help farmers in their decision at harvesting period. The application of advanced modelling techniques based on artificial intelligence and digital device technology can contribute to this goal. This study proposes a method to estimate the EO content of fruits in the field using classification and regression models based on a deep learning approach in two cultivars: cv. “Fantastico” and cv. “Femminello”. The first step was to capture images of the fruit in the Red, Green, and Blue colours (RGB) using a mid-range smartphone camera and a portable inspection chamber designed and developed for this study. The acquisition of the images was carried out in the field. The fruits were collected and transported to the laboratory, where the EO was extracted using steam hydrodistillation. Custom-built convolutional neural networks (CNN) and three transfer learning models (VGG-16, VGG-19, and Xception architectures) were trained and applied for classification (among different discrete levels of oil content) and regression (to predict the EO content). The classification results showed an accuracy of 0.795 and 0.797 on the test samples of the two cultivars separately, while the best regression model achieved a minimum mean squared error of 0.12 and 0.04 for each cultivar, respectively. The results showed the effectiveness of the approach tested and how modelling each variety independently can lead to better performance for the CNNs tested