google-deepmind/ddgc

google-deepmind/ddgc

Language: Python

License: Apache-2.0

Stars: 2

Forks: 0

Open issues: 0

Created: 2026-04-15T16:59:42Z

Pushed: 2026-04-24T06:20:24Z

Default branch: main

Fork: no

Archived: no

README:

DDGC: Dense and Diverse Goal Coverage

Code for training and evaluating RL agents on multi-goal, non-goal-conditioned, continuing environments built on Brax.

Environments

Four environments with sparse binary rewards (+1 at goal, 0 otherwise) and no goal information in observations:

| Environment | Obs Dim | Action Dim | Goals | Threshold | |---|---|---|---|---| | ant | 27 | 8 | 10 | 0.2 m | | reacher | 6 | 2 | 10 | 0.01 m | | pusher | 20 | 7 | 5 | 0.03 m | | half_cheetah | 17 | 6 | 6 | 0.15 m |

Agents

| Agent | Directory | Description | |---|---|---| | SAC | agents/sac/ | Soft Actor-Critic with automatic entropy tuning | | SAC+PC | agents/sac_pc/ | SAC + SimHash pseudo-count intrinsic reward | | SMM | agents/smm/ | State Marginal Matching via fictitious play | | DDGC | agents/ddgc/ | RND exploration → iterative policy mixture via Fitted AC | | GFlowNet | agents/gflownet/ | Trajectory balance + critic Q-value maximization |

Installation

pip install jax jaxlib brax flax optax orbax-checkpoint absl-py numpy

Usage

Each agent has a run.sh script. Example:

# Train DDGC agent on Ant environment
bash agents/ddgc/run.sh --env_name=ant --total_env_steps=5000000 --seed=0

# Train SAC agent on Reacher environment
bash agents/sac/run.sh --env_name=reacher --total_env_steps=1000000 --seed=0

All agents accept these common flags:

| Flag | Default | Description | |---|---|---| | --env_name | ant | Environment: ant, reacher, pusher, half_cheetah | | --seed | 0 | Random seed | | --total_env_steps | 5000000 | Total environment steps | | --num_envs | 64 | Parallel environment count | | --episode_length | 500 | Episode length | | --checkpoint_dir | '' | Directory for checkpoints |

Citing this work

@article{ddgc2026,
title={DDGC: Dense and Diverse Goal Coverage},
author={Sagalpreet Singh and Rishi Saket and Aravindan Raghuveer},
year={2026},
}

License and disclaimer

Copyright 2026 Google LLC

All software is licensed under the Apache License, Version 2.0 (Apache 2.0); you may not use this file except in compliance with the Apache 2.0 license. You may obtain a copy of the Apache 2.0 license at: https://www.apache.org/licenses/LICENSE-2.0

All other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY). You may obtain a copy of the CC-BY license at: https://creativecommons.org/licenses/by/4.0/legalcode

Unless required by applicable law or agreed to in writing, all software and materials distributed here under the Apache 2.0 or CC-BY licenses are distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the licenses for the specific language governing permissions and limitations under those licenses.

This is not an official Google product.