• 中文核心期刊
  • CSCD来源期刊
  • 中国科技核心期刊
  • CA、CABI、ZR收录期刊

融合SKNet与MobilenetV3的芒果叶片病虫害分类方法

沈熠辉, 何惠彬, 陈小宇, 颜胜男

沈熠辉,何惠彬,陈小宇,等. 融合SKNet与MobilenetV3的芒果叶片病虫害分类方法 [J]. 福建农业学报,2024,39(5):584−592. DOI: 10.19303/j.issn.1008-0384.2024.05.010
引用本文: 沈熠辉,何惠彬,陈小宇,等. 融合SKNet与MobilenetV3的芒果叶片病虫害分类方法 [J]. 福建农业学报,2024,39(5):584−592. DOI: 10.19303/j.issn.1008-0384.2024.05.010
SHEN Y H, HE H B, CHEN X Y, et al. Integrated SKNet/Mobilenet V3 Classification of Mango Leaf Diseases and Infestations [J]. Fujian Journal of Agricultural Sciences,2024,39(5):584−592. DOI: 10.19303/j.issn.1008-0384.2024.05.010
Citation: SHEN Y H, HE H B, CHEN X Y, et al. Integrated SKNet/Mobilenet V3 Classification of Mango Leaf Diseases and Infestations [J]. Fujian Journal of Agricultural Sciences,2024,39(5):584−592. DOI: 10.19303/j.issn.1008-0384.2024.05.010

融合SKNet与MobilenetV3的芒果叶片病虫害分类方法

基金项目: 福建省技术创新重点攻关及产业化项目(2023G015)
详细信息
    作者简介:

    沈熠辉(1991 —),男,硕士,助教,主要从事人工智能应用、农业数据处理研究,E-mail:1723378551@qq.com

  • 中图分类号: TQ639.8;TP183

Integrated SKNet/Mobilenet V3 Classification of Mango Leaf Diseases and Infestations

  • 摘要:
      目的  针对芒果叶片病虫害缺少数据集和识别准确率低的问题,筛选构建芒果叶片病虫害分类模型,以提高芒果叶病虫害分类准确率。
      方法  提出使用去噪扩散模型进行病虫害数据增强,同时联合SKNet与MobilenetV3模型的芒果叶片病虫害分类方法。首先使用去噪扩散模型对数据集进行扩充,再采用多尺度结构相似性指标对生成的病虫害图像与拍摄的病虫害图像之间的相似程度进行评估,接着对DDIM与DCGAN网络训练和生成效果进行比对。在MobilenetV3模型中,将SE注意力模块替换为SKNet模块进行构建网络模型。
      结果  使用DDIM生成的所有类型的病虫害图像与拍摄的病虫害图像的MS-SSIM指标均大于0.63,且都高于DCGAN。相较于其他注意力模块,联合SKNet与MobilenetV3的分类效果最佳,在98%以上。对添加CA、CBAM、ECA注意力模块进行平滑类激活图可视化,对比其他注意力模块,使用SKNet注意力分布区域更为集中在病虫害叶片上。
      结论  该方法在病虫害叶片检测上具有良好的应用前景,能提升病虫害识别效率与精度,减少检测成本,同时可应用于移动式或者嵌入式设备。
    Abstract:
      Objective  Leaf diseases and infestations on mango trees were classified for database establishment and precision identification by combining the Mobilenet V3 model with Selective Kernel Network (SKNet).
      Method  To improve the accuracy of disease and infestation classification on mango plants, data augmentation was firstly conducted. A denoising diffusion model was applied to expand the dataset followed by using a multi-scale structural similarity index to examine the similarity between the virtually generated and the camera-captured images of the diseases or infestations. Then, the training and generation effects of DDIM and DCGAN networks were compared. In the Mobilenet V3 model, the SE attention module was replaced with SKNet to construct the final platform.
      Results  The MS-SSIM index of all types of DDIM images was greater than 0.63, which was higher than that of DCGAN. The classification accuracy of 98% delivered by merging SKNet with Mobilenet V3 was the best performance. Furthermore, combination of the two programs afforded more focus on the diseased leaves than did other smooth grade activation visualization by adding CA, CBAM, or ECA.
      Conclusion   The newly developed classification method by integrating SKNet and Mobilenet V3 performed satisfactorily in distinguishing various diseased or infested mango leaves. The application not only significantly improved the efficiency and accuracy of disease identification but also reduced the epidemic monitoring costs by easily incorporating it with mobile or embedded devices.
  • 图  1   病虫害叶片数据集样本

    Figure  1.   Sample of diseased mango leaf images dataset

    图  2   加噪与去噪示意图

    Figure  2.   Schematic diagram of noise addition and removal

    图  3   融合SKNet的bneck模型结构

    ⊕:元素相加运算;⊗:元素相乘运算。

    Figure  3.   SKNet-integrated bneck model

    ⊕: Element-wise addition; ⊗: element-wise multiplication.

    图  4   学习率变化曲线

    Figure  4.   Change on learning rate

    图  5   0~1000步学习率曲线

    Figure  5.   Learning rate from 0 to 1000 step

    图  6   DDIM生成各类芒果叶片病虫害图

    Figure  6.   DDIM-generated mango leaf diseases

    图  7   迭代次数0~80下数据增强前后损失值与准确率

    Figure  7.   Loss and accuracy before and after data augmentation over Epochs 0–80

    图  8   原始图像集(上)、DCGAN数据增强(中)、DDIM数据增强(下)分类混淆矩阵

    Figure  8.   Classification confusion matrix on original (top), DCGAN data augmented (middle), and DDIM data augmented (bottom) image sets

    图  9   不同注意力模块的MobilenetV3模型平滑类激活图

    Figure  9.   Smooth Grad-CAM of Mobilenet V3 model introduced with different attention modules

    表  1   MobilenetV3模型结构

    Table  1   Structure of Mobilenet V3

    输入尺寸
    Input shape
    操作算子
    Operation
    扩展尺寸
    Expand size
    输出通道
    Output channel
    SE模块
    SE module
    激活函数
    Activation function
    步长
    Stride
    2242 conv2d,3×3 16 HS 2
    1122×16 bneck,3×3 16 16 RE 2
    562×16 bneck,3×3 72 24 RE 2
    282×24 bneck,3×3 88 24 RE 1
    282×24 bneck,5×5 96 40 HS 2
    142×40 bneck,5×5 240 40 HS 1
    142×40 bneck,5×5 240 40 HS 1
    142×40 bneck,5×5 120 48 HS 1
    142×48 bneck,5×5 144 48 HS 1
    142×48 bneck,5×5 288 96 HS 2
    72×96 bneck,5×5 576 96 HS 1
    72×96 bneck,5×5 576 96 HS 1
    72×96 conv2d,1×1 576 HS 1
    72×576 pool,7×7 1
    12×576 conv2d1×1,NBN 1280 HS 1
    12×1024 conv2d1×1,NBN 7 1
    conv2d—二维卷积;bneck—瓶颈模块;pool—池化层;NBN—不使用批量归一化;HS—硬切线激活函数;RE—修正线性单元;√—使用SE模块。
    conv2d: 2D convolution; bneck: bottleneck module; pool: pooling layer; NBN: no batch normalization; HS: hard swish activation function; RE: ReLU (rectified linear unit); √: SE module applied.
    下载: 导出CSV

    表  2   图片生成前后数据量比较

    Table  2   Data volumes before and after image generation

    类型
    Type
    原始图数量
    Original dataset
    增强后图像数量
    Augmented dataset
    炭疽病
    Colletotrichum gloeosporioides
    320 1800
    细菌性角斑病
    Xanthomonas campestris pv. mangiferaeindicae
    324 1800
    切叶象甲 Myllocerus viridanus 320 1800
    枯萎病 Fusarium oxysporum 320 1800
    瘿蚊 Erosomyia mangiferae 544 1800
    白粉病 Oidium mangiferae 320 1800
    煤污病 Capnodium mangiferae 320 1800
    下载: 导出CSV

    表  3   DDIM与DCGAN网络模型训练指标对比

    Table  3   Network model training metrics of DDIM and DCGAN

    指标 Index DDIM DCGAN
    模型大小 Model size/MB 117 89
    训练时间 Training time /h 48 36
    收敛速度(训练轮次) Convergence speed (epochs) 19 57
    训练中损失函数的标准偏差 Standard deviation 0.05 0.15
    总耗时 Total time/s 392.6 457.2
    下载: 导出CSV

    表  4   生成的病虫害图像与拍摄的病虫害图像MS-SSIM值

    Table  4   MS-SSIM values on virtually generated and camera-captured images of diseased leaves

    图像类型 Image type DDIM DCGAN
    炭疽病 Colletotrichum gloeosporioides 0.6312 0.5992
    细菌性角斑病
    Xanthomonas campestris pv. mangiferaeindicae
    0.7298 0.6912
    切叶象甲 Myllocerus viridanus 0.6754 0.6413
    枯萎病 Fusarium oxysporum 0.7123 0.6805
    瘿蚊 Erosomyia mangiferae 0.7459 0.6990
    白粉病 Oidium mangiferae 0.7211 0.6853
    煤污病 Capnodium mangiferae 0.6671 0.6396
    下载: 导出CSV

    表  5   引入注意力模块的MobilenetV3模型试验对比

    Table  5   Experimental results on Mobilenet V3 model introduced with attention modules

    算法
    Algorithm
    准确率
    Accuracy/%
    参数量
    Params/M
    乘加运算数
    MACs/G
    MobilenetV3(SE) 97.24 2.54 0.06
    MobilenetV3+CA 95.75 2.18 0.06
    MobilenetV3+CBAM 97.69 2.59 0.09
    MobilenetV3+ECA 97.51 2.08 0.06
    MobilenetV3+SKNet 98.21 2.58 0.06
    下载: 导出CSV
  • [1] 杨芝霓. 芒果主要采后病害的病原鉴定及炭疽病的生物防治[D]. 南宁: 广西大学.

    YANG Z N. Pathogen identification of mango post havest disease and biological control of anthracnose[D]. Nanning: Guangxi University. (in Chinese)

    [2] 秦丰, 刘东霞, 孙炳达, 等. 基于图像处理技术的四种苜蓿叶部病害的识别 [J]. 中国农业大学学报, 2016, 21(10):65−75.

    QIN F, LIU D X, SUN B D, et al. Recognition of four different alfalfa leaf diseases based on image processing technology [J]. Journal of China Agricultural University, 2016, 21(10): 65−75. (in Chinese)

    [3] 柴阿丽, 李宝聚, 石延霞, 等. 基于计算机视觉技术的番茄叶部病害识别 [J]. 园艺学报, 2010, 37(9):1423−1430.

    CHAI A L, LI B J, SHI Y X, et al. Recognition of tomato foliage disease based on computer vision technology [J]. Acta Horticulturae Sinica, 2010, 37(9): 1423−1430. (in Chinese)

    [4]

    SHI B Z, ZHOU X L, QIN Z K, et al. Corn ear quality recognition based on DCGAN data enhancement and transfer learning[C]//The 4th International Conference on Electronics, Communications and Control Engineering. Seoul Republic of Korea. ACM, 2021.

    [5]

    AZIZI S, KORNBLITH S, SAHARIA C, et al. Synthetic data from diffusion models improves imagenet classification [J]. arXiv preprint arXiv:2304.08466, 2023.

    [6] 哈马友吉, 任万春, 张秤, 等. 基于轻量级网络MobileNet V2的二极管玻壳缺陷识别 [J]. 传感器与微系统, 2022, 41(4):153−155,160.

    HAMA Y J, REN W C, ZHANG C, et al. Defect recognition of diode glass shells based on lightweight network MobileNet V2 [J]. Transducer and Microsystem Technologies, 2022, 41(4): 153−155,160. (in Chinese)

    [7] 王志强, 于雪莹, 杨晓婧, 等. 基于WGAN和MCA-MobileNet的番茄叶片病害识别 [J]. 农业机械学报, 2023, 54(5):244−252.

    WANG Z Q, YU X Y, YANG X J, et al. Tomato leaf diseases recognition based on WGAN and MCA-MobileNet [J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(5): 244−252. (in Chinese)

    [8]

    LI X, WANG W H, HU X L, et al. Selective kernel networks[J]. IEEE, 2020. DOI: 10.1109/CVPR.2019.00060.

    [9]

    AHMED S I, IBRAHIM M, NADIM M, et al. MangoLeafBD: A comprehensive image dataset to classify diseased and healthy mango leaves [J]. Data in Brief, 2023, 47: 108941. DOI: 10.1016/j.dib.2023.108941

    [10]

    ZHANG Q S, TAO M L, CHEN Y X. gDDIM: Generalized denoising diffusion implicit models [J]. arXiv preprint arXiv: 2206.05564v1, 2022.

    [11] 闫志浩, 周长兵, 李小翠. 生成扩散模型研究综述 [J]. 计算机科学, 2024, 51(1):273−283.

    YAN Z H, ZHOU C B, LI X C. Survey on generative diffusion model [J]. Computer Science, 2024, 51(1): 273−283. (in Chinese)

    [12]

    WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. Pacific Grove, CA, USA. IEEE, 2003: 1398-1402.

    [13]

    HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications [J]. arXiv preprint arXiv:1704.04861, 2017.

    [14]

    SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT. IEEE, 2018: 4510-4520.

    [15]

    HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019. Seoul, Korea (South). IEEE, 2019: 1314-1324.

    [16] 刘勇国, 高攀, 兰荻, 等. ECA-SKNet: 玉米单倍体种子的卷积神经网络识别模型 [J]. 电子科技大学学报, 2023, 52(6):866−871.

    LIU Y G, GAO P, LAN D, et al. ECA-SKNet: Convolutional neural network identification model for corn haploid seeds [J]. Journal of University of Electronic Science and Technology of China, 2023, 52(6): 866−871. (in Chinese)

    [17] 何翔. 基于DCGANs的半片光伏组件电致发光图像增强技术 [J]. 应用光学, 2023, 44(2):314−322. DOI: 10.5768/JAO202344.0202003

    HE X. Electroluminescence image enhancement technology of half-cut photovoltaic module based on DCGANs [J]. Journal of Applied Optics, 2023, 44(2): 314−322. (in Chinese) DOI: 10.5768/JAO202344.0202003

    [18]

    HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2021. Nashville, TN, USA. IEEE, 2021: 13713−13722.

    [19]

    WANG Q L, WU B G, ZHU P F, et al. ECA-net: Efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020. Seattle, WA, USA. IEEE, 2020: 11534-11542.

    [20]

    WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.

  • 期刊类型引用(5)

    1. 叶炜,颜沛沛,王培育,李尊文,郭莺,张剑亮,林敏水,江金兰. 金线兰茎腐病拮抗木霉菌鉴定及抑菌和促生效果评价. 福建农业学报. 2024(11): 1256-1264 . 本站查看
    2. 叶炜,颜沛沛,王培育,罗维鸿,林敏水,沈绍榕,江金兰. 金线兰茎腐病的病原菌鉴定与防治药剂的筛选. 亚热带农业研究. 2023(01): 1-9 . 百度学术
    3. 柴琴琴,曾建,张勋. 基于贝叶斯优化卷积神经网络的金线莲伪品鉴别. 浙江农业学报. 2022(02): 391-396 . 百度学术
    4. 彭琴,邹福贤,许少华,张勋,徐伟,林羽,陈抒云,许文. 基于化学计量学的指纹图谱法鉴别金线莲及其混伪品. 药学研究. 2021(03): 149-159 . 百度学术
    5. 王澄林,张勋,陈抒云,黄泽豪,朱志灵,林羽. 金线莲总黄酮的最佳超声提取工艺. 福建中医药. 2017(01): 40-42 . 百度学术

    其他类型引用(1)

图(9)  /  表(5)
计量
  • 文章访问数:  196
  • HTML全文浏览量:  75
  • PDF下载量:  51
  • 被引次数: 6
出版历程
  • 收稿日期:  2024-04-17
  • 修回日期:  2024-05-04
  • 网络出版日期:  2024-06-25
  • 刊出日期:  2024-05-27

目录

    /

    返回文章
    返回