{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T08:06:52Z","timestamp":1753862812348,"version":"3.41.2"},"reference-count":42,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Comp Applic In Engineering"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper presents the development of a teaching approach for Reinforcement Learning (RL) for students at the Faculty of Electrical Engineering, University of Ljubljana. The approach is designed to introduce students to the basic concepts, approaches, and algorithms of RL through examples and experiments in both simulation environments and on a real robot. The approach includes practical programs written in Python and presents various RL algorithms. The Q\u2010learning algorithm is introduced and a deep Q network is implemented to introduce the use of neural networks in deep RL. The software is user\u2010friendly and allows easy modification of learning parameters, reward functions, and algorithms. The approach was tested successfully on a Franka Emika Panda robot, where the robot manipulator learned to move to a randomly generated target position, shoot a real ball into the goal, and push various objects into target position. The goal of the presented teaching approach is to serve as a study aid for future generations of students of robotics to help them better understand the basic concepts of RL and apply them to a wide variety of problems.<\/jats:p>","DOI":"10.1002\/cae.22780","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T16:04:49Z","timestamp":1720454689000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Teaching approach for deep reinforcement learning of robotic strategies"],"prefix":"10.1002","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9359-0297","authenticated-orcid":false,"given":"Janez","family":"Podobnik","sequence":"first","affiliation":[{"name":"Laboratory of Robotics University of Ljubljana, Faculty of Electrical Engineering Ljubljana Slovenia"}]},{"given":"Ana","family":"Udir","sequence":"additional","affiliation":[{"name":"Laboratory of Robotics University of Ljubljana, Faculty of Electrical Engineering Ljubljana Slovenia"}]},{"given":"Marko","family":"Munih","sequence":"additional","affiliation":[{"name":"Laboratory of Robotics University of Ljubljana, Faculty of Electrical Engineering Ljubljana Slovenia"}]},{"given":"Matja\u017e","family":"Mihelj","sequence":"additional","affiliation":[{"name":"Laboratory of Robotics University of Ljubljana, Faculty of Electrical Engineering Ljubljana Slovenia"}]}],"member":"311","published-online":{"date-parts":[[2024,7,8]]},"reference":[{"key":"e_1_2_10_2_1","unstructured":"M.Abadi A.Agarwal P.Barham E.Brevdo Z.Chen C.Citro G. S.Corrado A.Davis J.Dean M.Devin S.Ghemawat I.Goodfellow A.Harp G.Irving M.Isard Y.Jia R.Jozefowicz L.Kaiser M.Kudlur J.Levenberg D.Mane R.Monga S.Moore D.Murray C.Olah M.Schuster J.Shlens B.Steiner I.Sutskever K.Talwar P.Tucker V.Vanhoucke V.Vasudevan F.Viegas O.Vinyals P.Warden M.Wattenberg M.Wicke Y.Yu andX.Zheng TensorFlow: Large\u2010scale machine learning on heterogeneous systems arXiv(2015)."},{"key":"e_1_2_10_3_1","unstructured":"J.AdamsandS.Turner Problem solving and creativity for undergraduate engineers: Process or product?EE 2008 \u2010 International Conference on Innovation Good Practice and Research in Engineering Education 2008."},{"key":"e_1_2_10_4_1","first-page":"288","volume-title":"Proceedings of EdMedia + Innovate Learning 2008","author":"Alimisis D.","year":"2008"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781003142706"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10798-016-9385-9"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2013.10.020"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10763-017-9799-1"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3068769"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/cae.22637"},{"key":"e_1_2_10_12_1","first-page":"3\u20134","article-title":"Robotics in education","volume":"81","author":"Curto B.","year":"2015","journal-title":"J. Intell. Robot. Syst"},{"key":"e_1_2_10_13_1","unstructured":"A.Dobrosovestnova Constructivism in educational robotics. interpretations and challenges Ph.D. thesis Universit\u00e4t Wien 2019."},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/cae.22732"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-022-01790-2"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3027152"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.3390\/computers10090110"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1002\/cae.22601"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1080\/10494820.2018.1525411"},{"key":"e_1_2_10_21_1","unstructured":"G.Martius Real robot challenge [online](2022).https:\/\/real-robot-challenge.com"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_10_23_1","unstructured":"OpenAI Gym documentation [online](2023).https:\/\/www.gymlibrary.dev accessed 30 January 2023."},{"volume-title":"Educational psychology: developing learners","year":"2016","author":"Ormrod J. E.","key":"e_1_2_10_24_1"},{"issue":"268","key":"e_1_2_10_25_1","first-page":"1","article-title":"Stable\u2010baselines3: reliable reinforcement learning implementations","volume":"22","author":"Raffin A.","year":"2021","journal-title":"J. Mach. Learn. Res"},{"key":"e_1_2_10_26_1","first-page":"153","volume-title":"Proc. Int. Worksh. Teach. Robot. Teach. Robot. Integr. Robot. School Curriculum","author":"Ruzzenente M.","year":"2012"},{"key":"e_1_2_10_27_1","first-page":"1","article-title":"Active learning methodologies and industry 4.0 skills development \u2010 a systematic review of the literature","volume":"2","author":"Santana A. L. M.","year":"2020","journal-title":"2020 XV Conferencia Latinoamericana de Tecnologias de Aprendizaje (LACLO)"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-021-00592-x"},{"volume-title":"Learning theories an educational perspective","year":"2012","author":"Schunk D. H.","key":"e_1_2_10_29_1"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2020.14"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.3390\/en16031512"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2017.05.001"},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2020.00008"},{"key":"e_1_2_10_34_1","doi-asserted-by":"crossref","unstructured":"R.SuenagaandK.Morioka Development of a web\u2010based education system for deep reinforcement learning\u2010based autonomous mobile robot navigation in real world 2020 IEEE\/SICE Int. Symp. Syst. Integr. (SII) 2020 pp.1040\u20131045.","DOI":"10.1109\/SII46433.2020.9025980"},{"volume-title":"Reinforcement learning: an introduction","year":"2018","author":"Sutton R. S.","key":"e_1_2_10_35_1"},{"key":"e_1_2_10_36_1","first-page":"317","article-title":"An open robotics environment motivates students to learn the key concepts of artificial neural networks and reinforcement learning","author":"Toivonen T.","year":"2017","journal-title":"Robotics in Education"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-18141-3_9"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.3390\/educsci12050305"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1186\/s41239-019-0171-0"},{"issue":"3","key":"e_1_2_10_40_1","article-title":"Reinforcement learning for robot research: a comprehensive review and open issues","volume":"18","author":"Zhang T.","year":"2021","journal-title":"Int. J. Adv. Robotic Syst"},{"key":"e_1_2_10_41_1","first-page":"737","volume-title":"2020 IEEE Symp. Series Comput. Intell. (SSCI)","author":"Zhao W.","year":"2020"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.1002\/cae.22363"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10763-018-09939-y"}],"container-title":["Computer Applications in Engineering Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cae.22780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T06:07:12Z","timestamp":1731391632000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cae.22780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,8]]},"references-count":42,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10.1002\/cae.22780"],"URL":"https:\/\/doi.org\/10.1002\/cae.22780","archive":["Portico"],"relation":{},"ISSN":["1061-3773","1099-0542"],"issn-type":[{"type":"print","value":"1061-3773"},{"type":"electronic","value":"1099-0542"}],"subject":[],"published":{"date-parts":[[2024,7,8]]},"assertion":[{"value":"2024-04-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e22780"}}