Metraipolis › archive › dc_e9ee5723b044

HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment — arXiv:2611.09224

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09224
License (as archived)
CC0-1.0
Archived
2026-11-09T22:34:55.806Z
SHA-256 of the archived text
5ecd36170f4128805dcab442908f9aa8cef960371d6eac3934c6376666a5c9a8
Stable URL of this page
https://metraipolis.com/documento/dc_e9ee5723b044

How to cite

HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment — arXiv:2611.09224 — arXiv (en), https://arxiv.org/abs/2611.09224 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e9ee5723b044 (Metraipolis archive dc_e9ee5723b044, sha256 5ecd36170f4128805dcab442908f9aa8cef960371d6eac3934c6376666a5c9a8)
BibTeX
@misc{dce9ee5723b044,
  title = {HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment — arXiv:2611.09224},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09224},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e9ee5723b044; sha256 5ecd36170f4128805dcab442908f9aa8cef960371d6eac3934c6376666a5c9a8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09224. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated fish feeding intensity assessment (FFIA) is important for precision aquaculture and feed cost optimization. However, unimodal methods are vulnerable to acoustic noise and visual disturbances, while existing multimodal frameworks often rely on computationally intensive 3D convolutions or Transformers, limiting their use on edge devices. To address these limitations, we propose HMMF-Net, a lightweight multimodal architecture that combines early multi-frame visual encoding and multi-axis acoustic modeling through a hierarchical bidirectional messenger cross-modal fusion (BMCF) mechanism. The visual branch captures temporal variations in feeding behavior by stacking temporally distant frames as a multi-channel input to an adapted 2D EfficientNetB0 backbone. The acoustic branch employs adaptive multi-axis residual units (AMARU) to extract temporal, frequency, and local time-frequency representations from log-mel spectrograms. BMCF progressively exchanges information between the two modalities at intermediate representation levels through compact messenger tokens, reducing the computational overhead of dense cross-modal attention. Evaluated on the AV-FFIA dataset using five-fold cross-validation, HMMF-Net achieves 96.77% accuracy and 96.88% F1-score on the held-out test folds, with 3.987M parameters and 1.002 GFLOPs. Compared with the best-performing visual configuration (95.49% accuracy), HMMF-Net reduces the classification error by 28.38%. Deployment on an NVIDIA Jetson Orin Nano using TensorRT with FP32 precision maintains comparable accuracy, with an inference time of 11.54 ms per 2-second sample and a model size of 18.81 MB. These results demonstrate that HMMF-Net combines effective multimodal representation learning with computational efficiency, enabling accurate and practical FFIA on resource-constrained edge platforms.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.09224 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:30:47.323Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).