{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T12:04:08Z","timestamp":1774440248007,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2019,6,6]],"date-time":"2019-06-06T00:00:00Z","timestamp":1559779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Since the state-of-the-art deep learning algorithms demand a large training dataset, which is often unavailable in some domains, the transfer of knowledge from one domain to another has been a trending technique in the computer vision field. However, this method may not be a straight-forward task considering several issues such as original network size or large differences between the source and target domain. In this paper, we perform transfer learning for semantic segmentation of off-road driving environments using a pre-trained segmentation network called DeconvNet. We explore and verify two important aspects regarding transfer learning. First, since the original network size was very large and did not perform well for our application, we proposed a smaller network, which we call the light-weight network. This light-weight network is half the size to the original DeconvNet architecture. We transferred the knowledge from the pre-trained DeconvNet to our light-weight network and fine-tuned it. Second, we used synthetic datasets as the intermediate domain before training with the real-world off-road driving data. Fine-tuning the model trained with the synthetic dataset that simulates the off-road driving environment provides more accurate results for the segmentation of real-world off-road driving environments than transfer learning without using a synthetic dataset does, as long as the synthetic dataset is generated considering real-world variations. We also explore the issue whereby the use of a too simple and\/or too random synthetic dataset results in negative transfer. We consider the Freiburg Forest dataset as a real-world off-road driving dataset.<\/jats:p>","DOI":"10.3390\/s19112577","type":"journal-article","created":{"date-parts":[[2019,6,7]],"date-time":"2019-06-07T03:56:31Z","timestamp":1559879791000},"page":"2577","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving"],"prefix":"10.3390","volume":"19","author":[{"given":"Suvash","family":"Sharma","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6774-4851","authenticated-orcid":false,"given":"John E.","family":"Ball","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5708-766X","authenticated-orcid":false,"given":"Bo","family":"Tang","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0707-9252","authenticated-orcid":false,"given":"Daniel W.","family":"Carruth","sequence":"additional","affiliation":[{"name":"Center for Advanced Vehicular Systems, Mississippi State University, Starkville, MS 39762, USA"}]},{"given":"Matthew","family":"Doude","sequence":"additional","affiliation":[{"name":"Center for Advanced Vehicular Systems, Mississippi State University, Starkville, MS 39762, USA"}]},{"given":"Muhammad Aminul","family":"Islam","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,6]]},"reference":[{"key":"ref_1","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the NIPS\u201912 the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada."},{"key":"ref_2","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"155","DOI":"10.3389\/fpls.2019.00155","article-title":"Deep learning-based segmentation and quantification of cucumber Powdery Mildew using convolutional neural network","volume":"10","author":"Lin","year":"2019","journal-title":"Front. Plant Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1002\/rob.21699","article-title":"Image segmentation for fruit detection and yield estimation in apple orchards","volume":"34","author":"Bargoti","year":"2017","journal-title":"J. Field Robot."},{"key":"ref_6","unstructured":"Ciresan, D., Giusti, A., Gambardella, L.M., and Schmidhuber, J. (2012, January 3\u20136). Deep neural networks segment neuronal membranes in electron microscopy images. Proceedings of the NIPS\u201912 the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kola\u0159\u00edk, M., Burget, R., Uher, V., \u0158\u00edha, K., and Dutta, M.K. (2019). Optimized High Resolution 3D Dense-U-Net Network for Brain and Spine Segmentation. Appl. Sci., 9.","DOI":"10.3390\/app9030404"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, Y., Ren, Q., Geng, J., Ding, M., and Li, J. (2018). Efficient Patch-Wise Semantic Segmentation for Large-Scale Remote Sensing Images. Sensors, 18.","DOI":"10.3390\/s18103232"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Pan, X., Gao, L., Zhang, B., Yang, F., and Liao, W. (2018). High-Resolution Aerial Imagery Semantic Labeling with Dense Pyramid Network. Sensors, 18.","DOI":"10.3390\/s18113774"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Papadomanolaki, M., Vakalopoulou, M., and Karantzalos, K. (2019). A Novel Object-Based Deep Learning Framework for Semantic Segmentation of Very High-Resolution Remote Sensing Data: Comparison with Convolutional and Fully Convolutional Networks. Remote Sens., 11.","DOI":"10.3390\/rs11060684"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1915","DOI":"10.1109\/TPAMI.2012.231","article-title":"Learning hierarchical features for scene labeling","volume":"35","author":"Farabet","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gupta, S., Girshick, R., Arbel\u00e1ez, P., and Malik, J. (2014). Learning rich features from RGB-D images for object detection and segmentation. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10584-0_23"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hariharan, B., Arbel\u00e1ez, P., Girshick, R., and Malik, J. (2014). Simultaneous detection and segmentation. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10584-0_20"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_15","unstructured":"Badrinarayanan, V., Kendall, A., and Cipolla, R. (2015). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Noh, H., Hong, S., and Han, B. (2015, January 7\u201313). Learning deconvolution network for semantic segmentation. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.178"},{"key":"ref_17","unstructured":"Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv."},{"key":"ref_18","unstructured":"Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_21","unstructured":"Long, M., Cao, Y., Wang, J., and Jordan, M.I. (2015). Learning transferable features with deep adaptation networks. arXiv."},{"key":"ref_22","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). How transferable are features in deep neural networks?. Advances in Neural Information Processing Systems 27, Curran Associates, Inc."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Van Opbroek, A., Ikram, M.A., Vernooij, M.W., and de Bruijne, M. (2012, January 1). Supervised image segmentation across scanner protocols: A transfer learning approach. Proceedings of the International Workshop on Machine Learning in Medical Imaging, Nice, France.","DOI":"10.1007\/978-3-642-35428-1_20"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1109\/TMI.2014.2366792","article-title":"Transfer learning improves supervised image segmentation across imaging protocols","volume":"34","author":"Ikram","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 13\u201316). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wei, L., Runge, L., and Xiaolei, L. (2018, January 9\u201311). Traffic sign detection and recognition via transfer learning. Proceedings of the 2018 Chinese Control And Decision Conference (CCDC), Shenyang, China.","DOI":"10.1109\/CCDC.2018.8408160"},{"key":"ref_27","unstructured":"Ying, W., Zhang, Y., Huang, J., and Yang, Q. (2018, January 10\u201315). Transfer learning via learning to transfer. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_28","unstructured":"Xiao, H., Wei, Y., Liu, Y., Zhang, M., and Feng, J. (2017). Transferable Semi-supervised Semantic Segmentation. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hong, S., Oh, J., Lee, H., and Han, B. (2016, January 27\u201330). Learning transferrable knowledge for semantic segmentation with deep convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.349"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nigam, I., Huang, C., and Ramanan, D. (2018, January 12\u201315). Ensemble Knowledge Transfer for Semantic Segmentation. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00168"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_32","unstructured":"Bengio, Y. (2011, January 2). Deep learning of representations for unsupervised and transfer learning. Proceedings of the UTLW\u201911 the 2011 International Conference on Unsupervised and Transfer Learning Workshop, Washington, DC, USA."},{"key":"ref_33","unstructured":"Baldi, P. (2012, January 27). Autoencoders, unsupervised learning, and deep architectures. Proceedings of the ICML Workshop on Unsupervised and Transfer Learning, Edinburgh, Scotland."},{"key":"ref_34","unstructured":"Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Maturana, D., Chou, P.W., Uenoyama, M., and Scherer, S. (2018). Real-time semantic mapping for autonomous off-road navigation. Field and Service Robotics, Springer.","DOI":"10.1007\/978-3-319-67361-5_22"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Adhikari, S.P., Yang, C., Slot, K., and Kim, H. (2018). Accurate Natural Trail Detection Using a Combination of a Deep Neural Network and Dynamic Programming. Sensors, 18.","DOI":"10.3390\/s18010178"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Holder, C.J., Breckon, T.P., and Wei, X. (2016, January 8\u201316). From on-road to off: transfer learning within a deep convolutional neural network for segmentation and classification of off-road scenes. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46604-0_11"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, K., and Sun, J. (2015, January 7\u201312). Convolutional neural networks at constrained time cost. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299173"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20137). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM international conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_41","unstructured":"Valada, A., Oliveira, G., Brox, T., and Burgard, W. (2016, January 3\u20136). Deep Multispectral Semantic Scene Understanding of Forested Environments using Multimodal Fusion. Proceedings of the 2016 International Symposium on Experimental Robotics (ISER 2016), Tokyo, Japan."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hudson, C.R., Goodin, C., Doude, M., and Carruth, D.W. (2018, January 23\u201325). Analysis of Dual LIDAR Placement for Off-Road Autonomy Using MAVS. Proceedings of the 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA), Kosice, Slovakia.","DOI":"10.1109\/DISA.2018.8490620"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Goodin, C., Sharma, S., Doude, M., Carruth, D., Dabbiru, L., and Hudson, C. (2019). Training of Neural Networks with Automated Labeling of Simulated Sensor Data, Society of Automotive Engineers. SAE Technical Paper.","DOI":"10.4271\/2019-01-0120"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2577\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:56:36Z","timestamp":1760187396000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2577"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,6]]},"references-count":43,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["s19112577"],"URL":"https:\/\/doi.org\/10.3390\/s19112577","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,6]]}}}