Cross-Domain Transfer Learning: Enhancing Deep Neural Networks for Low-Resource Environments
Abstract
Deep neural networks (DNNs) have achieved remarkable success in various domains; however, their performance often relies heavily on large-scale, high-quality labeled datasets, which are scarce in low-resource environments. Cross-domain transfer learning has emerged as a promising technique for adapting pre-trained models from data-rich source domains to low-resource target domains to address this limitation. This study explores innovative strategies to enhance the performance and applicability of DNNs through cross-domain transfer learning, focusing on challenges such as domain disparity, data scarcity, and computational constraints. We evaluate several transfer learning approaches, including feature-based transfer, parameter fine-tuning, and adversarial domain adaptation, across diverse healthcare, agriculture, and natural language processing applications. Experimental results demonstrate significant improvements in model accuracy and generalization in low-resource environments, with accuracy gains of up to 20% compared to models trained from scratch. Additionally, we analyze the impact of transfer learning on reducing training time and computational requirements, making it a viable solution for resource-constrained settings. Despite its potential, the study highlights critical challenges, including negative transfer, model interpretability, and ethical considerations in domain transfer. Addressing these issues, we propose a framework for selecting optimal source domains and enhancing model robustness through hybrid techniques and unsupervised learning. This research emphasizes the transformative potential of cross-domain transfer learning in bridging the gap between data-rich and low-resource environments, paving the way for more equitable and efficient applications of deep learning technologies worldwide.
Downloads
References
M. H. IBRAHIM, “WBA-DNN: A hybrid weight bat algorithm with deep neural network for classification of poisonous and harmful wild plants,” Comput. Electron. Agric., vol. 190, p. 106478, Nov. 2021, doi: https://doi.org/10.1016/j.compag.2021.106478.
K. Xu, A. M. Tartakovsky, J. Burghardt, and E. Darve, “Learning viscoelasticity models from indirect data using deep neural networks,” Comput. Methods Appl. Mech. Eng., vol. 387, p. 114124, Dec. 2021, doi: https://doi.org/10.1016/j.cma.2021.114124.
F. Yu et al., “Characterizing and understanding deep neural network batching systems on GPUs,” BenchCouncil Trans. Benchmarks, Stand. Eval., vol. 3, no. 4, p. 100151, Dec. 2023, doi: https://doi.org/10.1016/j.tbench.2024.100151.
G. Omondi and T. O. Olwal, “Variational autoencoder-enhanced deep neural network-based detection for MIMO systems,” e-Prime - Adv. Electr. Eng. Electron. Energy, vol. 6, p. 100335, Dec. 2023, doi: https://doi.org/10.1016/j.prime.2023.100335.
Y. Liu, H. Liang, and S. Zhao, “A lightweight visual mamba network for image recognition under resource-limited environments,” Appl. Soft Comput., vol. 167, 2024, doi: https://doi.org/10.1016/j.asoc.2024.112294.
Z. Li, V. Chang, H. Hu, M. Fu, J. Ge, and F. Piccialli, “Optimizing makespan and resource utilization for multi-DNN training in GPU cluster,” Futur. Gener. Comput. Syst., vol. 125, pp. 206–220, Dec. 2021, doi: https://doi.org/10.1016/j.future.2021.06.021.
B. Huang, X. Huang, X. Liu, C. Ding, Y. Yin, and S. Deng, “Adaptive partitioning and efficient scheduling for distributed DNN training in heterogeneous IoT environment,” Comput. Commun., vol. 215, pp. 169–179, Feb. 2024, doi: https://doi.org/10.1016/j.comcom.2023.12.034.
L. Cheng, H. Qi, R. Ma, X. Kong, Y. Zhang, and Y. Zhu, “FS-PTL: A unified few-shot partial transfer learning framework for partial cross-domain fault diagnosis under limited data scenarios,” Knowledge-Based Syst., vol. 305, p. 112658, Dec. 2024, doi: https://doi.org/10.1016/j.knosys.2024.112658.
A. Maity and G. Saha, “Enhancing cross-domain robustness in phonocardiogram signal classification using domain-invariant preprocessing and transfer learning,” Comput. Methods Programs Biomed., vol. 257, p. 108462, Dec. 2024, doi: https://doi.org/10.1016/j.cmpb.2024.108462.
I. Anam, N. Arafat, M. S. Hafiz, J. R. Jim, M. M. Kabir, and M. F. Mridha, “A systematic review of UAV and AI integration for targeted disease detection, weed management, and pest control in precision agriculture,” Smart Agric. Technol., vol. 9, p. 100647, Dec. 2024, doi: https://doi.org/10.1016/j.atech.2024.100647.
M. A. Ali, A. K. Sharma, and R. K. Dhanaraj, “Heterogeneous features and deep learning networks fusion-based pest detection, prevention and controlling system using IoT and pest sound analytics in a vast agriculture system,” Comput. Electr. Eng., vol. 116, p. 109146, May 2024, doi: https://doi.org/10.1016/j.compeleceng.2024.109146.
Y. Zhang and C. Lv, “TinySegformer: A lightweight visual segmentation model for real-time agricultural pest detection,” Comput. Electron. Agric., vol. 218, p. 108740, Mar. 2024, doi: https://doi.org/10.1016/j.compag.2024.108740.
Y. Liu and S.-E. Fang, “Cross-domain structural damage identification using transfer learning strategy,” Eng. Struct., vol. 311, p. 118171, Jul. 2024, doi: https://doi.org/10.1016/j.engstruct.2024.118171.
Y. M. Saluky, “A Review: Application of AIOT in Smart Cities in Industry 4.0 and Society 5.0,” nternational J. Smart Syst., vol. 1, no. 1, pp. 1–4, 2023.
B. Fu, A. Hadid, and N. Damer, “Generative AI in the context of assistive technologies: Trends, limitations and future directions,” Image Vis. Comput., p. 105347, Nov. 2024, doi: https://doi.org/10.1016/j.imavis.2024.105347.
F. Terranova et al., “Windy events detection in big bioacoustics datasets using a pre-trained Convolutional Neural Network,” Sci. Total Environ., vol. 949, p. 174868, Nov. 2024, doi: https://doi.org/10.1016/j.scitotenv.2024.174868.
K. El-Demerdash, R. A. El-Khoribi, M. A. Ismail Shoman, and S. Abdou, “Deep learning based fusion strategies for personality prediction,” Egypt. Informatics J., vol. 23, no. 1, pp. 47–53, Mar. 2022, doi: https://doi.org/10.1016/j.eij.2021.05.004.
Y. Jin, X. Song, Y. Yang, X. Hei, N. Feng, and X. Yang, “An improved multi-channel and multi-scale domain adversarial neural network for fault diagnosis of the rolling bearing,” Control Eng. Pract., vol. 154, p. 106120, Jan. 2025, doi: https://doi.org/10.1016/j.conengprac.2024.106120.
Y. Zhao, W. Huang, W. Liu, and X. Yao, “Negatively correlated ensemble against transfer adversarial attacks,” Pattern Recognit., p. 111155, Nov. 2024, doi: https://doi.org/10.1016/j.patcog.2024.111155.
W. Yang, “Comparing self-generated noise reduction efforts and tolerance of neighbours’ noise in multi-residential buildings,” J. Build. Eng., vol. 98, p. 111495, Dec. 2024, doi: https://doi.org/10.1016/j.jobe.2024.111495.
B. Sun, W. Gan, R. Ma, P. Feng, and J. Chu, “SnapE-ResNet: A novel electronic nose classification algorithm for gas data collected by open sampling systems,” Sensors Actuators A Phys., vol. 379, p. 115978, Dec. 2024, doi: https://doi.org/10.1016/j.sna.2024.115978.
G. He, C. Lin, J. Ren, and P. Duan, “Predicting the emergence of disruptive technologies by comparing with references via soft prompt-aware shared BERT,” J. Informetr., vol. 18, no. 4, p. 101596, Nov. 2024, doi: https://doi.org/10.1016/j.joi.2024.101596.
V. G., V. V. Rani, S. Ponnada, and J. S., “A hybrid EfficientNet-DbneAlexnet for brain tumor detection using MRI images,” Comput. Biol. Chem., vol. 115, p. 108279, Apr. 2025, doi: https://doi.org/10.1016/j.compbiolchem.2024.108279.
