* Update RTD config
* Switch to ruff for sorting imports
* Evaluate falsy to truthy with not rather than `is False`
* Add `features_extractor` argument to maskable policy
* Add set_options for AsyncEval
* Doc fixes
* Add support for Gym 0.24
* Fixes for gym 0.24
* Fix for new reset signature
* Add tmp SB3 branch
* Fixes for gym 0.26
* Remove unused import
* Fix dependency
* Type annotations fixes
* Reformat
* Reformat with black 23
* Move to gymnasium
* Patch env if needed
* Fix types
* Fix CI
* Fixes for gymnasium
* Fix wrapper annotations
* Update version
* Fix type check
* Update QRDQN type hints and bug fix with multi envs
* Fix TQC type hints
* Fix TRPO type hints
* Additional fixes
* Update SB3 version
* Update issue templates and CI
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
* Issue forms and pyproject.toml
* [ci skip] Fix typos
* Fix isort config
* Use secret link to download atari roms
* Fix for mypy and update config
* Upgrade SB3 and fix warnings
* Fix doc build
* Update Makefile
* Lint first
* Add wrappers
* Add maskable distributions
* Add mypy configuration
* Add maskable base datastructures
* Add ppo_mask package
* Fix circular dependency and remove test code that slipped in
* Automatically mask vecenv if env is masked
* Fix debugging change that slipped in
* Workaround for subclassing RolloutBufferSamples
* Duplicate lots of policy code in order to swap out the distributions used
* Fix pytype error
* Maintain py 3.6 compatibility
* Fix isort lint errors
* Use pyproject.toml to configure black line length
* Blacken
* Remove mypy.ini
* Fully replace RolloutBufferSamples
* Drop support for continuous distributions, remove SDE-related code
* Eliminate MaskableAlgorithm and MaskableOnPolicyAlgorithm
* Fix formatting
* Override superclass methods as needed, fix circular import, improve naming
* Fix codestyle
* Eliminate VecActionMasker, replace with utils
* Fix codestyle
* Support masking for MultiDiscrete action spaces
* Fix codestyle
* Don't require the env to provide the mask already flattened
* Consistent naming, prefer 'Maskable' to 'Masked'
* Register policy
* Link to abstract instead of pdf
* Allow distribution masking to be unapplied + improved comments and docstrings
* Don't use deprecated implicit optional typing
* Check codestyle
* Add docstring and remove misplaced TODO
* Simplify env masking API, error if API unmet. Make use_masking a learn() kwarg
* Fix codestyle
* Update various internals to be consistent with latest SB3
* Simplify MaskableRolloutBuffer reset
* Add docstring and type annotations
* Ensure old probs aren't cached
* Fix for new logger
* Add test + fixes
* Start doc
* Fix type annotation
* Remove abstract class + add test
* Fix evaluation (add support for multi envs)
* Handle merge conflicts in documentation
* Bugfix: mask updates should apply to original logits, not the last masked output
* Add test of distribution masking behavior
* Reformat
* Add MultiBinary support, remove unneeded distribution type checks
* Remove unused import
* Fix when using multiple envs
* Remove addressed TODO
* Upgrade for SB3 1.2.0
* Update docs with results + how to replicate
* Add action masker tests, move wrapper tests
* Move distributions, add more distribution tests
* Add MaskablePPO tests, simplify and rename discrete test env
* Address TODO
* Add tests for MaskableMultiCategoricalDistribution, fix distributions
* Add maskable identity envs for all supported action spaces, add tests, fix bug
* Formatting fixes
* Update doc env
* Dict support not ready
* Cleanup
Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
Antonin RAFFIN
kronion