Papers

Papers related to machine learning, deep learning, and reinforcement learning.

A good way to track the paper reading may be through Google Docs?

• LLMs/FMs: Google doc [Lastly updated: 1/6/2026]
• Computer Vision: Google doc [Lastly updated: 1/6/2026]
• (Multi-agent) Reinforcement Learning: Google doc [Lastly updated: 4/19/2025]
• Autonomous Driving: Google doc [Lastly updated: 10/06/2024]
• Smart Agriculture: Google doc [Lastly updated: 1/6/2026]
• Robotics: Google doc [Lastly updated: 1/6/2026]
• Power Systems: Google doc [Lastly updated: 10/06/2024]
• Time-series Models: Google doc [Lastly updated: 10/06/2024]
• Good Tutorials: Google doc [Lastly updated: 1/6/2026]
• General Research Ideas: Google doc [Lastly updated: 10/06/2024]

Fresh Papers

• Li, Yuecheng, Hongwen He, Amir Khajepour, Yong Chen, Weiwei Huo, and Hao Wang. "Deep reinforcement learning for intelligent energy management systems of hybrid-electric powertrains: Recent advances, open issues, and prospects." IEEE Transactions on Transportation Electrification (2024).
• Shi, Zhonghao, Ellen Landrum, Amy O. Connell, Mina Kian, Leticia Pinto-Alva, Kaleen Shrestha, Xiaoyuan Zhu, and Maja J. Matarić. "How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Survey." arXiv preprint arXiv:2404.00938 (2024).
• Zhuang, Weiming, Chen Chen, and Lingjuan Lyu. "When foundation model meets federated learning: Motivations, challenges, and future directions." arXiv preprint arXiv:2306.15546 (2023).
• Li, Xinran, and Jun Zhang. "Context-aware Communication for Multi-agent Reinforcement Learning." arXiv preprint arXiv:2312.15600 (2023).
• Gao, Tianyu, Xingcheng Yao, and Danqi Chen. "Simcse: Simple contrastive learning of sentence embeddings." arXiv preprint arXiv:2104.08821 (2021).
• Guan, Cong, et al. "Efficient Multi-agent Communication via Self-supervised Information Aggregation." Advances in Neural Information Processing Systems 35 (2022): 1020-1033.
• Yuan, William, et al. "Transformer in Reinforcement Learning for Decision-Making: A Survey." (2023).
• He, Sihong, et al. "Data-driven distributionally robust electric vehicle balancing for autonomous mobility-on-demand systems under demand and supply uncertainties." IEEE Transactions on Intelligent Transportation Systems (2023).
• Chafii, Marwa, et al. "Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networks." arXiv preprint arXiv:2309.06021 (2023).
• Faghri, Fartash, et al. "Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement." arXiv preprint arXiv:2303.08983 (2023).
• Wang, Letian, et al. "Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors." arXiv preprint arXiv:2305.04412 (2023).
• Jiang, Qingsong, et al. "Deep-reinforcement-learning-based water diversion strategy." Environmental Science and Ecotechnology (2023): 100298.
• Prudencio, Rafael Figueiredo, Marcos ROA Maximo, and Esther Luna Colombini. "A survey on offline reinforcement learning: Taxonomy, review, and open problems." IEEE Transactions on Neural Networks and Learning Systems (2023).
• Chen, Wubing. "Learning Multi-intersection Traffic Signal Control via Coevolutionary Multi-Agent Reinforcement Learning." (2023).
• Chen, Xianda, et al. "FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling." arXiv preprint arXiv:2306.05381 (2023).
• Yadavalli, Sushma Reddy, Lokesh Chandra Das, and Myounggyu Won. "RLPG: Reinforcement Learning Approach for Dynamic Intra-Platoon Gap Adaptation for Highway On-Ramp Merging." arXiv preprint arXiv:2212.03497 (2022).

Contents

• Large Language Models

  • LLM Survey

• Reinforcement Learning

  • Survey
  • Value-based
  • Policy-based
  • Model Based
  • Offline RL
  • Imitation Learning
  • Semi-supervised
  • Reward Shaping
  • Inverse Reinforcement Learning
  • Transfer Learning
  • Diffusion Models RL
  • Foundation Models in RL
  • Transformer
  • Federated Reinforcement Learning
  • Constrained Policy Optimization
  • Control
  • Applications
    • Intelligent Transportation Systems
    • Gaming
    • Robotics
    • Quantum Control
    • UAVs
    • Environment

• Multi Agent Reinforcement Learning

  • Survey
  • Value-based MARL
  • Policy-based MARL
  • Communication
  • Parameter Sharing
  • Constrained Policy Optimisation MARL
  • Graph Convolutional Reinforcement Learning
  • Offline MARL
  • Attention
  • Multi-agent Imitation Learning
  • Simulator
  • Traffic Applications
    • Autonomous Driving
    • Traffic Signal Control

• Machine Learning

  • Multiple Instance Learning
  • Continual Learning
  • Privacy
  • Large Language Models

• Computer Vision

  • Foundation Models
  • Semi Supervised Learning
  • Image Classification
  • Object Detection
  • Image Segmentation
  • Panoptic Segmentation
  • GANs
  • Diffusion Models
  • Image Synthsis
  • Transformers
  • Domain Adaptation
  • Others

• Meta Learning

  • Meta Learning
  • Meta MARL
  • Offline Meta
  • Traffic Applications

• Power System

  • Smart Buildings
  • Voltage and Frequency Control
  • Energy Trading
  • Testbed
  • Load Control

• Precision Agriculture

  • Weed Control
  • Fruit Detection
  • Data Augmentation
  • Meat Science
  • Agricultural Robots
  • Plant Phenotyping
  • GANs in Agriculture
  • Label-efficient Learning in Agriculture

• Robotics

  • Soft Robots

• Tricks

  • Ensemble
  • Curriculum Learning
  • Contrastive Learning
  • Auxiliary Task
  • Noise

• Impressive Works

  • Multi-agent Systems
  • Smart Agriculture
  • Reinforcement Learning

Large Language Models

LLM Survey

• Shi, Zhonghao, Ellen Landrum, Amy O. Connell, Mina Kian, Leticia Pinto-Alva, Kaleen Shrestha, Xiaoyuan Zhu, and Maja J. Matarić. "How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Survey." arXiv preprint arXiv:2404.00938 (2024).

Reinforcement Learning

Survey

Value-based

• [HER] Andrychowicz, Marcin, et al. "Hindsight experience replay." Advances in neural information processing systems 30 (2017).
• Yang, Zhihan, and Hai Nguyen. "Recurrent off-policy baselines for memory-based continuous control." arXiv preprint arXiv:2110.12628 (2021).
• [DRQN]: Hausknecht, Matthew, and Peter Stone. "Deep recurrent q-learning for partially observable mdps." arXiv preprint arXiv:1507.06527 (2015).
• [Esemble] Lan, Qingfeng, et al. "Maxmin q-learning: Controlling the estimation bias of q-learning." arXiv preprint arXiv:2002.06487 (2020).
• [Esemble] Chen, Xinyue, et al. "Randomized ensembled double q-learning: Learning fast without a model." arXiv preprint arXiv:2101.05982 (2021).
• [Esemble] Hiraoka, Takuya, et al. "Dropout Q-Functions for Doubly Efficient Reinforcement Learning." arXiv preprint arXiv:2110.02034 (2021).

Policy-based

• Xu, Mengda, Manuela Veloso, and Shuran Song. "ASPiRe: Adaptive Skill Priors for Reinforcement Learning." arXiv preprint arXiv:2209.15205 (2022).
• (Auxiliary tasks) Jaderberg, Max, et al. "Reinforcement learning with unsupervised auxiliary tasks." arXiv preprint arXiv:1611.05397 (2016).

Offline RL

• [Survey]: Prudencio, Rafael Figueiredo, Marcos ROA Maximo, and Esther Luna Colombini. "A survey on offline reinforcement learning: Taxonomy, review, and open problems." IEEE Transactions on Neural Networks and Learning Systems (2023).

• [Survey]: Levine, Sergey, et al. "Offline reinforcement learning: Tutorial, review, and perspectives on open problems." arXiv preprint arXiv:2005.01643 (2020).

• (BCQ): Fujimoto, Scott, David Meger, and Doina Precup. "Off-policy deep reinforcement learning without exploration." International Conference on Machine Learning. PMLR, 2019.

• (BEAR) Kumar, Aviral, et al. "Stabilizing off-policy q-learning via bootstrapping error reduction." arXiv preprint arXiv:1906.00949 (2019).

• Chen, Lili, et al. "Decision transformer: Reinforcement learning via sequence modeling." arXiv preprint arXiv:2106.01345 (2021).

• Janner, Michael, Qiyang Li, and Sergey Levine. "Reinforcement Learning as One Big Sequence Modeling Problem." arXiv preprint arXiv:2106.02039 (2021).

• Fujimoto, Scott, and Shixiang Shane Gu. "A Minimalist Approach to Offline Reinforcement Learning." arXiv preprint arXiv:2106.06860 (2021).

• Mandlekar, Ajay, et al. "Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data." 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020

Offline-to-Online

• Nair, Ashvin, et al. "AWAC: Accelerating Online Reinforcement Learning with Offline Datasets." (2020).
• Lee, Seunghyun, et al. "Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble." arXiv preprint arXiv:2107.00591 (2021).

Model Based

• Kurutach, Thanard, et al. "Model-ensemble trust-region policy optimization." arXiv preprint arXiv:1802.10592 (2018).
• Matsushima, Tatsuya, et al. "Deployment-efficient reinforcement learning via model-based offline optimization." arXiv preprint arXiv:2006.03647 (2020).
• Zhang, Marvin, et al. "Solar: Deep structured representations for model-based reinforcement learning." International Conference on Machine Learning. PMLR, 2019.
• Kaiser, Lukasz, et al. "Model-based reinforcement learning for atari." arXiv preprint arXiv:1903.00374 (2019).

Uncertainty Estimate

• Yu, Tianhe, et al. "Mopo: Model-based offline policy optimization." arXiv preprint arXiv:2005.13239 (2020).
• (LOMPO) Rafailov, Rafael, et al. "Offline reinforcement learning from images with latent space models." Learning for Dynamics and Control. PMLR, 2021.

Imitation Learning

• Wang, Letian, et al. "Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors." arXiv preprint arXiv:2305.04412 (2023).
• Chen, Dian, et al. "Learning by cheating." Conference on Robot Learning. PMLR, 2020.
• Lynch, Corey, et al. "Learning latent plans from play." Conference on Robot Learning. PMLR, 2020.
• (BCQ) Torabi, Faraz, Garrett Warnell, and Peter Stone. "Behavioral cloning from observation." arXiv preprint arXiv:1805.01954 (2018).
• (ILPO) Edwards, Ashley, et al. "Imitating latent policies from observation." International Conference on Machine Learning. PMLR, 2019.

Semi-supervised

• Park, Jongjin, et al. "SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning." arXiv preprint arXiv:2203.10050 (2022).
• Finn, Chelsea, et al. "Generalizing skills with semi-supervised reinforcement learning." arXiv preprint arXiv:1612.00429 (2016).

Hierarchical Reinforcement Learning

• Nachum, Ofir, et al. "Data-efficient hierarchical reinforcement learning." arXiv preprint arXiv:1805.08296 (2018).

Reward Shaping

• Ng, Andrew Y., Daishi Harada, and Stuart Russell. "Policy invariance under reward transformations: Theory and application to reward shaping." Icml. Vol. 99. 1999.

Inverse Reinforcement Learning

• (FORM) Jaegle, Andrew, et al. "Imitation by Predicting Observations." International Conference on Machine Learning. PMLR, 2021.
• Christiano, Paul F., et al. "Deep reinforcement learning from human preferences." Advances in neural information processing systems 30 (2017).

Transfer Learning

• Cang, Catherine, et al. "Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL." arXiv preprint arXiv:2106.09119 (2021).

Diffusion Models RL

• Wang, Zhendong, Jonathan J. Hunt, and Mingyuan Zhou. "Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning." arXiv preprint arXiv:2208.06193 (2022).
• Janner, Michael, et al. "Planning with Diffusion for Flexible Behavior Synthesis." arXiv preprint arXiv:2205.09991 (2022).

Foundation Models in RL

• Di Palo, Norman, et al. "Towards A Unified Agent with Foundation Models." Workshop on Reincarnating Reinforcement Learning at ICLR 2023. 2023.

Hybrid Action Space

• Li, Boyan, et al. "Hyar: Addressing discrete-continuous action reinforcement learning via hybrid action representation." arXiv preprint arXiv:2109.05490 (2021).
• Neunert, Michael, et al. "Continuous-discrete reinforcement learning for hybrid control in robotics." Conference on Robot Learning. PMLR, 2020.

Transformer

• Mao, Hangyu, et al. "Transformer in Transformer as Backbone for Deep Reinforcement Learning." arXiv preprint arXiv:2212.14538 (2022).

Federated Reinforcement Learning

• [Survey] Beltrán, Enrique Tomás Martínez, et al. "Decentralized Federated Learning: Fundamentals, State-of-the-art, Frameworks, Trends, and Challenges." arXiv preprint arXiv:2211.08413 (2022).
• [Survey] Qi, Jiaju, et al. "Federated reinforcement learning: Techniques, applications, and open challenges." arXiv preprint arXiv:2108.11887 (2021).

Constrained Policy Optimization

• Achiam, Joshua, et al. "Constrained policy optimization." International conference on machine learning. PMLR, 2017.

Control

• Amos, Brandon, et al. "Differentiable mpc for end-to-end planning and control." Advances in neural information processing systems 31 (2018).

Applications

Intelligent Transportation Systems

• • Chen, Xianda, et al. "FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling." arXiv preprint arXiv:2306.05381 (2023).
• Yadavalli, Sushma Reddy, Lokesh Chandra Das, and Myounggyu Won. "RLPG: Reinforcement Learning Approach for Dynamic Intra-Platoon Gap Adaptation for Highway On-Ramp Merging." arXiv preprint arXiv:2212.03497 (2022).
• Cao, Zhong, et al. "Continuous improvement of self-driving cars using dynamic confidence-aware reinforcement learning." Nature Machine Intelligence 5.2 (2023): 145-158.
• Huang, Wenhui, et al. "Goal-guided Transformer-enabled Reinforcement Learning for Efficient Autonomous Navigation." arXiv preprint arXiv:2301.00362 (2023).
• Liu, Haochen, et al. "Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving." arXiv preprint arXiv:2208.12263 (2022).
• Mavrogiannis, Angelos, Rohan Chandra, and Dinesh Manocha. "B-GAP: Behavior-Guided Action Prediction for Autonomous Navigation." arXiv preprint arXiv:2011.03748 (2020).

Gaming

• Zha, Daochen, et al. "DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning." arXiv preprint arXiv:2106.06135 (2021).

Robotics

• Haarnoja, Tuomas, et al. "Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning." arXiv preprint arXiv:2304.13653 (2023).
• Evans, Benjamin, et al. "Accelerating Online Reinforcement Learning via Supervisory Safety Systems." arXiv preprint arXiv:2209.11082 (2022).
• Lee, Joonho, et al. "Learning quadrupedal locomotion over challenging terrain." Science robotics 5.47 (2020): eabc5986.

Environment

• Jiang, Qingsong, et al. "Deep-reinforcement-learning-based water diversion strategy." Environmental Science and Ecotechnology (2023): 100298.

Quantum Control

• Ma, Hailan, et al. "Curriculum-based deep reinforcement learning for quantum control." IEEE Transactions on Neural Networks and Learning Systems (2022).

UAVs

• Joshi, Bhaskar, Dhruv Kapur, and Harikumar Kandath. "Sim-to-Real Deep Reinforcement Learning based Obstacle Avoidance for UAVs under Measurement Uncertainty." arXiv preprint arXiv:2303.07243 (2023).

Multi Agent Reinforcement Learning

Survey MARL

• Yuan, William, et al. "Transformer in Reinforcement Learning for Decision-Making: A Survey." (2023).
• Da Silva, Felipe Leno, and Anna Helena Reali Costa. "A survey on transfer learning for multiagent reinforcement learning systems." Journal of Artificial Intelligence Research 64 (2019): 645-703.
• Wong, Annie, et al. "Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches." arXiv preprint arXiv:2106.15691 (2021).

Value-based MARL

• Guan, Cong, et al. "Efficient Multi-agent Communication via Self-supervised Information Aggregation." Advances in Neural Information Processing Systems 35 (2022): 1020-1033.
• VDN (2017): Sunehag, Peter, et al. "Value-decomposition networks for cooperative multi-agent learning." arXiv preprint arXiv:1706.05296 (2017).
• QMIX (2018): Rashid, Tabish, et al. "QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning." arXiv preprint arXiv:1803.11485 (2018).
• DIAL (2016): Foerster, Jakob, et al. "Learning to communicate with deep multi-agent reinforcement learning." Advances in neural information processing systems. 2016.
• CommNet (2016): Sukhbaatar, Sainbayar, and Rob Fergus. "Learning multiagent communication with backpropagation." Advances in neural information processing systems. 2016.
• IAC (2021): Ma, Xiaoteng, et al. "Modeling the Interaction between Agents in Cooperative Multi-Agent Reinforcement Learning." arXiv preprint arXiv:2102.06042 (2021).

Policy-based MARL

• Wen, Muning, et al. "Multi-Agent Reinforcement Learning is a Sequence Modeling Problem." arXiv preprint arXiv:2205.14953 (2022).
• Yu, Chao, et al. "The surprising effectiveness of ppo in cooperative, multi-agent games." arXiv preprint arXiv:2103.01955 (2021).
• Kuba, Jakub Grudzien, et al. "Trust region policy optimisation in multi-agent reinforcement learning." arXiv preprint arXiv:2109.11251 (2021).
• Kuba, Jakub Grudzien, et al. "Settling the variance of multi-agent policy gradients." Advances in Neural Information Processing Systems 34 (2021): 13458-13470.
• ConsensusNet (2018): Zhang, Kaiqing, et al. "Fully decentralized multi-agent reinforcement learning with networked agents." arXiv preprint arXiv:1802.08757 (2018).
• MAAC: Iqbal, Shariq, and Fei Sha. "Actor-attention-critic for multi-agent reinforcement learning." International Conference on Machine Learning. PMLR, 2019.
• NeurComm: Chu, Tianshu, Sandeep Chinchali, and Sachin Katti. "Multi-agent Reinforcement Learning for Networked System Control." arXiv preprint arXiv:2004.01339 (2020).

Communication

• Li, Xinran, and Jun Zhang. "Context-aware Communication for Multi-agent Reinforcement Learning." arXiv preprint arXiv:2312.15600 (2023).
• Chafii, Marwa, et al. "Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networks." arXiv preprint arXiv:2309.06021 (2023).
• Zhu, Changxi, Mehdi Dastani, and Shihan Wang. "A survey of multi-agent reinforcement learning with communication." arXiv preprint arXiv:2203.08975 (2022).
• [MASIA] Guan, Cong, et al. "Efficient Multi-agent Communication via Self-supervised Information Aggregation." Advances in Neural Information Processing Systems 35 (2022): 1020-1033.
• Kim, Woojun, Jongeui Park, and Youngchul Sung. "Communication in multi-agent reinforcement learning: Intention sharing." International Conference on Learning Representations. 2021.
• [NeurComm] Chu, Tianshu, Sandeep Chinchali, and Sachin Katti. "Multi-agent reinforcement learning for networked system control." arXiv preprint arXiv:2004.01339 (2020).
• [IC3Net] Singh, Amanpreet, Tushar Jain, and Sainbayar Sukhbaatar. "Learning when to communicate at scale in multiagent cooperative and competitive tasks." arXiv preprint arXiv:1812.09755 (2018).
• [COMA] Foerster, Jakob, et al. "Counterfactual multi-agent policy gradients." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018.
• [NMARL] Zhang, Kaiqing, et al. "Fully decentralized multi-agent reinforcement learning with networked agents." International Conference on Machine Learning. PMLR, 2018.
• [DIAL] Foerster, Jakob, et al. "Learning to communicate with deep multi-agent reinforcement learning." Advances in neural information processing systems 29 (2016).
• [CommNet] Sukhbaatar, Sainbayar, and Rob Fergus. "Learning multiagent communication with backpropagation." Advances in neural information processing systems 29 (2016).

Parameter Sharing

• Gupta, Jayesh K., Maxim Egorov, and Mykel Kochenderfer. "Cooperative multi-agent control using deep reinforcement learning." International Conference on Autonomous Agents and Multiagent Systems. Springer, Cham, 2017.
• Lin, Kaixiang, et al. "Efficient large-scale fleet management via multi-agent deep reinforcement learning." Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018.

Constrained Policy Optimisation MARL

• Gu, Shangding, et al. "Multi-agent constrained policy optimisation." arXiv preprint arXiv:2110.02793 (2021).

Graph Convolutional Reinforcement Learning

• Kortvelesy, Ryan, Steven Morad, and Amanda Prorok. "Permutation-Invariant Set Autoencoders with Fixed-Size Embeddings for Multi-Agent Learning." arXiv preprint arXiv:2302.12826 (2023).
• Jiang, Jiechuan, et al. "Graph convolutional reinforcement learning." arXiv preprint arXiv:1810.09202 (2018).
• Dong, Jiqian, et al. "A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network." arXiv preprint arXiv:2010.05437 (2020).

Offline MARL

• Pan, Ling, et al. "Plan better amid conservatism: Offline multi-agent reinforcement learning with actor rectification." International Conference on Machine Learning. PMLR, 2022.
• Yang, Yiqin, et al. "Believe what you see: Implicit constraint approach for offline multi-agent reinforcement learning." Advances in Neural Information Processing Systems 34 (2021): 10299-10312.

Attention

• Guo, Xudong, Daming Shi, and Wenhui Fan. "Scalable Communication for Multi-Agent Reinforcement Learning via Transformer-Based Email Mechanism." arXiv preprint arXiv:2301.01919 (2023).
• Qi, Shuhan, et al. "Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning." IEEE Transactions on Neural Networks and Learning Systems (2022).

Multi-agent Imitation Learning

• Wang, Hongwei, et al. "Multi-Agent Imitation Learning with Copulas." arXiv preprint arXiv:2107.04750 (2021).

Simulator

• Peng, Bei, et al. "Facmac: Factored multi-agent centralised policy gradients." Advances in Neural Information Processing Systems 34 (2021): 12208-12221.

Traffic Applications

Autonomous Driving

• Li, Meng, et al. "Enhancing Cooperation of Vehicle Merging Control in Heavy Traffic Using Communication-Based Soft Actor-Critic Algorithm." IEEE Transactions on Intelligent Transportation Systems (2022).
• Zhang, Jiawei, et al. "Multi-Agent DRL-Based Lane Change With Right-of-Way Collaboration Awareness." IEEE Transactions on Intelligent Transportation Systems (2022).
• self-play: Tang, Yichuan. "Towards learning multi-agent negotiations via self-play." Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. 2019.

Traffic Signal Control

Machine Learning

Multiple Instance Learning

• Amores, Jaume. "Multiple instance classification: Review, taxonomy and comparative study." Artificial intelligence 201 (2013): 81-105.

Continual Learning

• Wang, Liyuan, et al. "A Comprehensive Survey of Continual Learning: Theory, Method and Application." arXiv preprint arXiv:2302.00487 (2023).

Privacy

• Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016.

Large Language Models

• Bai, Yuntao, et al. "Constitutional AI: Harmlessness from AI Feedback." arXiv preprint arXiv:2212.08073 (2022).
• Huang, Shaohan, et al. "Language is not all you need: Aligning perception with language models." arXiv preprint arXiv:2302.14045 (2023).

Computer Vision

Foundation Models

• Zhuang, Weiming, Chen Chen, and Lingjuan Lyu. "When foundation model meets federated learning: Motivations, challenges, and future directions." arXiv preprint arXiv:2306.15546 (2023).
• Moor, Michael, et al. "Foundation models for generalist medical artificial intelligence." Nature 616.7956 (2023): 259-265.

Semi Supervised Learning

• Zhang, Jifan, et al. "LabelBench: A Comprehensive Framework for Benchmarking Label-Efficient Learning." arXiv preprint arXiv:2306.09910 (2023).
• Chen, Hao, et al. "SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning." arXiv preprint arXiv:2301.10921 (2023).
• Wang, Yidong, et al. "Freematch: Self-adaptive thresholding for semi-supervised learning." arXiv preprint arXiv:2205.07246 (2022).
• Zhang, Bowen, et al. "Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling." Advances in Neural Information Processing Systems 34 (2021): 18408-18419.
• Sohn, Kihyuk, et al. "Fixmatch: Simplifying semi-supervised learning with consistency and confidence." Advances in neural information processing systems 33 (2020): 596-608.

Image Classification

Object Detection

• Ali, Mansoor, Gilberto Ochoa-Ruiz, and Sharib Ali. "A semi-supervised Teacher-Student framework for surgical tool detection and localization." arXiv preprint arXiv:2208.09926 (2022).

Image Segmentation

Panoptic Segmentation

• Li, Yanwei, et al. "Fully convolutional networks for panoptic segmentation with point-based supervision." IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
• Shen, Yunhang, et al. "Toward joint thing-and-stuff mining for weakly supervised panoptic segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
• Kirillov, Alexander, et al. "Panoptic feature pyramid networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
• Li, Qizhu, Anurag Arnab, and Philip HS Torr. "Weakly-and semi-supervised panoptic segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.
• Kirillov, Alexander, et al. "Panoptic segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.

Diffusion Models

• Chai, Shang, Liansheng Zhuang, and Fengying Yan. "LayoutDM: Transformer-based Diffusion Model for Layout Generation." arXiv preprint arXiv:2305.02567 (2023).
• Bao, Fan, et al. "All are Worth Words: a ViT Backbone for Score-based Diffusion Models." arXiv preprint arXiv:2209.12152 (2022).
• You, Zebin, et al. "Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels." arXiv preprint arXiv:2302.10586 (2023).
• Bansal, Arpit, et al. "Cold diffusion: Inverting arbitrary image transforms without noise." arXiv preprint arXiv:2208.09392 (2022).
• Sohl-Dickstein, Jascha, et al. "Deep unsupervised learning using nonequilibrium thermodynamics." International Conference on Machine Learning. PMLR, 2015.
• Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in Neural Information Processing Systems 33 (2020): 6840-6851.
• Song, Jiaming, Chenlin Meng, and Stefano Ermon. "Denoising diffusion implicit models." arXiv preprint arXiv:2010.02502 (2020).
• Nichol, Alexander Quinn, and Prafulla Dhariwal. "Improved denoising diffusion probabilistic models." International Conference on Machine Learning. PMLR, 2021.
• Dhariwal, Prafulla, and Alexander Nichol. "Diffusion models beat gans on image synthesis." Advances in Neural Information Processing Systems 34 (2021): 8780-8794.
• Ho, Jonathan, et al. "Cascaded Diffusion Models for High Fidelity Image Generation." J. Mach. Learn. Res. 23 (2022): 47-1.

GANs

Image Synthsis

• Saseendran, Amrutha, Kathrin Skubch, and Margret Keuper. "Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
• Sylvain, Tristan, et al. "Object-centric image generation from layouts." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 3. 2021.
• Koturwar, Saiprasad, Soma Shiraishi, and Kota Iwamoto. "Robust multi-object detection based on data augmentation with realistic image synthesis for point-of-sale automation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.
• Hinz, Tobias, Stefan Heinrich, and Stefan Wermter. "Generating multiple objects at spatially distinct locations." arXiv preprint arXiv:1901.00686 (2019).

Transformers

• Rao, Yongming, et al. "Dynamicvit: Efficient vision transformers with dynamic token sparsification." Advances in neural information processing systems 34 (2021): 13937-13949.

Domain Adaptation

• Tzeng, Eric, et al. "Adversarial discriminative domain adaptation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
• Ganin, Yaroslav, et al. "Domain-adversarial training of neural networks." The journal of machine learning research 17.1 (2016): 2096-2030.

Others

• Faghri, Fartash, et al. "Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement." arXiv preprint arXiv:2303.08983 (2023).

Meta Learning

Meta Learning

• (MAML): Finn, Chelsea, Pieter Abbeel, and Sergey Levine. "Model-agnostic meta-learning for fast adaptation of deep networks." International Conference on Machine Learning. PMLR, 2017.
• (Reptile): Nichol, Alex, Joshua Achiam, and John Schulman. "On first-order meta-learning algorithms." arXiv preprint arXiv:1803.02999 (2018).
• PEARL: Rakelly, Kate, et al. "Efficient off-policy meta-reinforcement learning via probabilistic context variables." International conference on machine learning. PMLR, 2019.
• MAML++: Antoniou, Antreas, Harrison Edwards, and Amos Storkey. "How to train your MAML." arXiv preprint arXiv:1810.09502 (2018).
• MQL: Fakoor, Rasool, et al. "Meta-q-learning." arXiv preprint arXiv:1910.00125 (2019).

Meta MARL

• Parisotto, Emilio, et al. "Concurrent meta reinforcement learning." arXiv preprint arXiv:1903.02710 (2019).
• Chen, Long, et al. "Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization." IEEE Transactions on Neural Networks and Learning Systems (2021).
• Munir, Md Shirajum, et al. "Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems." IEEE Transactions on Network and Service Management (2021).
• Gupta, Abhinav, Angeliki Lazaridou, and Marc Lanctot. "Meta Learning for Multi-agent Communication." Learning to Learn-Workshop at ICLR 2021. 2021.

Offline Meta

• Mitchell, Eric, et al. "Offline Meta-Reinforcement Learning with Advantage Weighting." arXiv preprint arXiv:2008.06043 (2020).
• Li, Lanqing, Rui Yang, and Dijun Luo. "FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization." arXiv preprint arXiv:2010.01112 (2020).

Imitation Learning

• Duan, Yan, et al. "One-shot imitation learning." arXiv preprint arXiv:1703.07326 (2017).
• James, Stephen, Michael Bloesch, and Andrew J. Davison. "Task-embedded control networks for few-shot imitation learning." Conference on Robot Learning. PMLR, 2018.

Traffic Applications

• Jaafra, Yesmina, et al. "Meta-Reinforcement Learning for Adaptive Autonomous Driving." (2019)
• Ye, Fei, et al. "Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles." arXiv preprint arXiv:2008.12451 (2020).
• Hu, Ye, et al. "Distributed multi-agent meta learning for trajectory design in wireless drone networks." IEEE Journal on Selected Areas in Communications (2021).

Power System

Smart Buildings

• Vázquez-Canteli, José R., et al. "Citylearn v1. 0: An openai gym environment for demand response with deep reinforcement learning." Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation. 2019.
• Zhang, Huiliang, Di Wu, and Benoit Boulet. "MetaEMS: A Meta Reinforcement Learning-based Control Framework for Building Energy Management System." arXiv preprint arXiv:2210.12590 (2022).
• Z. Nagy, G. Henze, S. Dey et al., Ten questions concerning reinforcement learning for building energy management, Building and Environment (2023), doi: https://doi.org/10.1016/j.buildenv.2023.110435.

Voltage and Frequency Control

• Cao, Di, et al. "A multi-agent deep reinforcement learning based voltage regulation using coordinated PV inverters." IEEE Transactions on Power Systems 35.5 (2020): 4120-4123.
• Wang, Minrui, et al. "Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer." arXiv preprint arXiv:2206.03721 (2022).
• Gao, Yuanqi, Wei Wang, and Nanpeng Yu. "Consensus multi-agent reinforcement learning for volt-var control in power distribution networks." IEEE Transactions on Smart Grid 12.4 (2021): 3594-3604.
• Wang, Jianhong, et al. "Multi-agent reinforcement learning for active voltage control on power distribution networks." Advances in Neural Information Processing Systems 34 (2021): 3271-3284.
• Zhang, Qianzhi, et al. "Multi-agent safe policy learning for power management of networked microgrids." IEEE Transactions on Smart Grid 12.2 (2020): 1048-1062.

Energy Trading

• Qiu, Dawei, et al. "Mean-Field Multi-Agent Reinforcement Learning for Peer-to-Peer Multi-Energy Trading." IEEE Transactions on Power Systems (2022).
• Chen, Tianyi, et al. "Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning." IEEE Transactions on Smart Grid 13.1 (2021): 715-727.
• Ye, Yujian, et al. "A scalable privacy-preserving multi-agent deep reinforcement learning approach for large-scale peer-to-peer transactive energy trading." IEEE transactions on smart grid 12.6 (2021): 5185-5200.

Testbed

• Meinecke, Steffen, et al. "Simbench—a benchmark dataset of electric power systems to compare innovative solutions based on power flow analysis." Energies 13.12 (2020): 3290.

Load Control

• Qin, Zhaoming, et al. "Privacy preserving load control of residential microgrid via deep reinforcement learning." IEEE Transactions on Smart Grid 12.5 (2021): 4079-4089.

Precision Agriculture

Weed Control

• Dang, Fengying, et al. "YOLOWeeds: A novel benchmark of YOLO object detectors for multi-class weed detection in cotton production systems." Computers and Electronics in Agriculture 205 (2023): 107655.
• Steininger, Daniel, et al. "The CropAndWeed Dataset: A Multi-Modal Learning Approach for Efficient Crop and Weed Manipulation." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.
• Rahman, Abdur, Yuzhen Lu, and Haifeng Wang. "Performance Evaluation of Deep Learning Object Detectors for Weed Detection for Cotton." Smart Agricultural Technology (2022): 100126.
• Du, Yayun, et al. "Deep-cnn based robotic multi-class under-canopy weed control in precision farming." 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022.
• Chen, Dong, et al. "Performance evaluation of deep transfer learning on multi-class identification of common weed species in cotton production systems." Computers and Electronics in Agriculture 198 (2022): 107091.
• Dang, Fengying, et al. "DeepCottonWeeds (DCW): A Novel Benchmark of YOLO Object Detectors for Weed Detection in Cotton Production Systems." 2022 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers, 2022.

Plant Disease

• Paymode, Ananda S., and Vandana B. Malode. "Transfer Learning for Multi-Crop Leaf Disease Image Classification using Convolutional Neural Network VGG." Artificial Intelligence in Agriculture 6 (2022): 23-33.

Fruit Detection

• Li, Kangshun, et al. "A fast and lightweight detection algorithm for passion fruit pests based on improved YOLOv5." Computers and Electronics in Agriculture 204 (2023): 107534.

Fruit Flowers

• Siddique, Abubakar, Amy Tabb, and Henry Medeiros. "Self-supervised Learning for Panoptic Segmentation of Multiple Fruit Flower Species." arXiv preprint arXiv:2209.04618 (2022).

Plant Phenotyping

• Roggiolani, Gianmarco, et al. "Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain." arXiv preprint arXiv:2210.07879 (2022).

GANs in Agriculture

• Please refer to Awesome GANs in Agriculture

Label-efficient Learning in Agriculture

• Please refer to Label-efficient_in_Agriculture

Data Augmentation

• Trabucco, Brandon, et al. "Effective Data Augmentation With Diffusion Models." arXiv preprint arXiv:2302.07944 (2023).
• [Survey] Lu, Yuzhen, et al. "Generative adversarial networks (GANs) for image augmentation in agriculture: A systematic review." Computers and Electronics in Agriculture 200 (2022): 107208.
• [Survey] Xu, Mingle, et al. "A Comprehensive Survey of Image Augmentation Techniques for Deep Learning." arXiv preprint arXiv:2205.01491 (2022).

Meat Science

• Lee, Hyo-Jun, et al. "MSENet: Marbling score estimation network for automated assessment of Korean beef." Meat Science 188 (2022): 108784.

Agricultural Robots

• Hu, Chengsong, et al. "Algorithm and System Development for Robotic Micro-Volume Herbicide Spray Towards Precision Weed Management." IEEE Robotics and Automation Letters 7.4 (2022): 11633-11640.

Robotics

Soft Robots

• Liu, Wenbo, et al. "Touchless interactive teaching of soft robots through flexible bimodal sensory interfaces." Nature communications 13.1 (2022): 1-14.
• Xiao, Xuesu, et al. "Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation." arXiv preprint arXiv:2209.10780 (2022).
• Gasoto, Renato, et al. "A validated physical model for real-time simulation of soft robotic snakes." 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019.
• Liu, Xuan, et al. "Learning to locomote with artificial neural-network and cpg-based control in a soft snake robot." 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.
• Liu, Xuan, Cagdas Onal, and Jie Fu. "Reinforcement Learning of a CPG-regulated Locomotion Controller for a Soft Snake Robot." arXiv preprint arXiv:2207.04899 (2022).
• Ji, Guanglin, et al. "Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learning." IEEE Robotics and Automation Letters 6.4 (2021): 7461-7468.
• Li, Guanda, Jun Shintake, and Mitsuhiro Hayashibe. "Deep Reinforcement Learning Framework for Underwater Locomotion of Soft Robot." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021.
• Centurelli, Andrea, et al. "Closed-loop Dynamic Control of a Soft Manipulator using Deep Reinforcement Learning." IEEE Robotics and Automation Letters 7.2 (2022): 4741-4748.

Tricks

Ensemble

• [Esemble] Du, Xinqi, et al. "A Contrastive-Enhanced Ensemble Framework for Efficient Multi-Agent Reinforcement Learning." Expert Systems with Applications (2024): 123158.
• [Esemble] Lan, Qingfeng, et al. "Maxmin q-learning: Controlling the estimation bias of q-learning." arXiv preprint arXiv:2002.06487 (2020).
• [Esemble] Chen, Xinyue, et al. "Randomized ensembled double q-learning: Learning fast without a model." arXiv preprint arXiv:2101.05982 (2021).
• [Esemble] Hiraoka, Takuya, et al. "Dropout Q-Functions for Doubly Efficient Reinforcement Learning." arXiv preprint arXiv:2110.02034 (2021).

Curriculum Learning

• Portelas, Rémy, et al. "Automatic curriculum learning for deep rl: A short survey." arXiv preprint arXiv:2003.04664 (2020).
• Chen, Dong, et al. "Deep multi-agent reinforcement learning for highway on-ramp merging in mixed traffic." arXiv preprint arXiv:2105.05701 (2021).
• Liu, Xuan, et al. "Learning to locomote with artificial neural-network and cpg-based control in a soft snake robot." 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.
• Zeng, Yilei, et al. "Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment." arXiv preprint arXiv:2208.02932 (2022).

Contrastive Learning

• Gao, Tianyu, Xingcheng Yao, and Danqi Chen. "Simcse: Simple contrastive learning of sentence embeddings." arXiv preprint arXiv:2104.08821 (2021).
• Du, Xinqi, et al. "A Contrastive-Enhanced Ensemble Framework for Efficient Multi-Agent Reinforcement Learning." Expert Systems with Applications (2024): 123158.

Auxiliary Task

• Zhou,Shumin, et al. ""Auxiliary Task-based Deep Reinforcement Learning for Quantum Control"", arXiv preprint arXiv:2302.14312 (2023).
• Devlin, Sam, et al. "Potential-based difference rewards for multiagent reinforcement learning." Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. 2014.

Impressive Works

Multi-agent Systems

• Kortvelesy, Ryan, Steven Morad, and Amanda Prorok. "Permutation-Invariant Set Autoencoders with Fixed-Size Embeddings for Multi-Agent Learning." arXiv preprint arXiv:2302.12826 (2023).
• Chen, Wubing. "Learning Multi-intersection Traffic Signal Control via Coevolutionary Multi-Agent Reinforcement Learning." (2023).

Smart Agriculture

• Tao, Ran, et al. "Optimizing crop management with reinforcement learning and imitation learning." arXiv preprint arXiv:2209.09991 (2022).
• Steininger, Daniel, et al. "The CropAndWeed Dataset: A Multi-Modal Learning Approach for Efficient Crop and Weed Manipulation." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.

Reinforcement Learning

• Das, Lokesh Chandra, and Myounggyu Won. "Saint-acc: Safety-aware intelligent adaptive cruise control for autonomous vehicles using deep reinforcement learning." International Conference on Machine Learning. PMLR, 2021.
• Li, Yuecheng, Hongwen He, Amir Khajepour, Yong Chen, Weiwei Huo, and Hao Wang. "Deep reinforcement learning for intelligent energy management systems of hybrid-electric powertrains: Recent advances, open issues, and prospects." IEEE Transactions on Transportation Electrification (2024).