Diffusion Models for Sequential Decision-Making: A Survey

This repository contains a collection of resources and papers on Diffusion Models for Sequential Decision-Making.

:rocket: Please check out our survey paper Diffusion Models for Reinforcement Learning: A Survey

Table of Contents

• Diffusion Models for Sequential Decision-Making: A Survey
  • Table of Contents
  • Papers
    • Offline Reinforcement Learning
    • Online Reinforcement Learning
    • Imitation Learning
    • Trajectory Generation
    • Data Augmentation
  • Citation

Papers

Offline Reinforcement Learning

• Planning with Diffusion for Flexible Behavior Synthesis, ICML 2022. [paper] [code]

• Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning, ICLR 2023. [paper] [code]

• Offline Reinforcement Learning via High-fidelity Generative Behavior Modeling, ICLR 2023. [paper] [code]

• Is Conditional Generative Modeling all you need for Decision-Making?, ICLR 2023. [paper] [code]

• AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners, ICML 2023. [paper] [code]

• Metadiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL, ICML 2023. [paper]

• Hierarchical Diffusion for Offline Decision Making, ICML 2023. [paper] [code]

• Contrastive Energy Prediction for Exact Energy-guided Diffusion Sampling in Offline Reinforcement Learning, ICML 2023. [paper] [code]

• Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks, arXiv 2023. [paper] [code]

• IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies, arXiv 2023. [paper] [code]

• Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning, NeurIPS 2023. [paper] [code]

• EDGI: Equivariant Diffusion for Planning with Embodied Agents, NeurIPS 2023. [paper]

• Extracting Reward Functions from Diffusion Models, NeurIPS 2023. [paper]

• Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?, NeurIPS 2023. [paper]

• Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvement, NeurIPS 2023. [paper] [code]

• Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Plans, NeurIPS 2023. [paper] [code]

• SafeDiffuser: Safe Planning with Diffusion Probabilistic Models, arXiv 2023. [paper]

• Efficient Diffusion Policies for Offline Reinforcement Learning, arXiv 2023. [paper] [code]

• MADiff: Offline Multi-agent Learning with Diffusion Models, arXiv 2023. [paper] [code]

• Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning, arXiv 2023. [paper]

• Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching, CoRL 2023. [paper] [code]

• Value function estimation using conditional diffusion models for control, arXiv 2023. [paper]

• Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning, arXiv 2023. [paper]

• Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning, arXiv 2023. [paper]

• Diffusion Policies as Multi-Agent Reinforcement Learning Strategies, ICANN 2023. [paper]

• DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning, arXiv 2023. [paper] [code]

• Score Regularized Policy Optimization through Diffusion Behavior, ICLR 2024. [paper] [code]

• Adaptive Online Replanning with Diffusion Models, arXiv 2023. [paper]

• AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model, arXiv 2023. [paper] [code]

• SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution, CVPR 2024. [paper] [website]

• Learning a Diffusion Model Policy from Rewards vis Q-score Matching, arXiv 2023. [paper]

• Simple Hierarchical Planning with Diffusion, ICLR 2024. [paper]

• Reasoning with Latent Diffusion in Offline Reinforcement Learning, ICLR 2024. [paper]

• Efficient Planning with Latent Diffusion, ICLR 2024. [paper]

• Contrastive Diffuser: Planning Towards High Return States via Contrastive Learning, arXiv 2024. [paper]

• DMBP: Diffusion model-based predictor for robust offline reinforcement learning against state observation perturbations, ICLR 2024. [paper] [code]

• Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning, arXiv 2024. [paper] [code]

• Diffusion World Model, arXiv 2024. [paper]

• Diffusion World Models, OpenReview 2024. [paper]

• Policy-Guided Diffusion, arXiv 2024. [paper] [code]

Online Reinforcement Learning

• Policy Representation via Diffusion Probability Model for Reinforcement Learning, arXiv 2023. [paper]

• Boosting Continuous Control with Consistency Policy, arXiv 2023. [paper]

• Diffusion Reward: Learning Rewards via Conditional Video Diffusion, arXiv 2023. [paper] [website] [code]

• ATraDiff: Accelerating Online Reinforcement Learning with Imaginary Trajectories, OpenReview 2024. [paper]

Imitation Learning

• Imitating Human Behaviour with Diffusion Models, ICLR 2023. [paper] [code]

• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion, RSS 2023. [paper] [code]

• Goal-Conditioned Imitation Learning using Score-based Diffusion Policies, RSS 2023. [paper] [code]

• To the Noise and Back: Diffusion for Shared Autonomy, RSS 2023. [paper] [code]

• DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics, RAL 2023. [paper]

• Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition, CoRL 2023. [paper] [code]

• XSkill: Cross Embodiment Skill Discovery, CoRL 2023. [paper]

• ChainedDiffuser: Unifying Trajectory Diffusion and Keypose Prediction for Robotic Manipulation, CoRL 2023. [paper] [code]

• PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play, CoRL 2023. [paper]

• Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models, CoRL 2023. [paper] [[c