stable-baselines3-contrib-sacd/CONTRIBUTING.md

Contributing to Stable-Baselines3 - Contrib

This contrib repository is designed for experimental implementations of various training algorithms so that others may make use of them. This includes full training algorithms, different tools (e.g. new environment wrappers, callbacks) and extending algorithms implemented in stable-baselines3.

Before opening a pull request, open an issue discussing the contribution. Once we agree that the plan looks good, go ahead and implement it.

Contributions and review focuses on following three parts:

1. Implementation quality

• Performance of the training algorithms should match what proposed authors reported.
• This is ensured by including a code that replicates an experiment from the original paper or from an established codebase (e.g. the code from authors), as well as a test to check that implementation works on program level (does not crash).

2. Documentation

• Documentation quality should match that of stable-baselines3, with each algorithm containing its own README file, changelog, in-code documentation to clarify the flow of logic and report the expected results.

3. Consistency with stable-baselines3

• To ease readibility, all contributions need to follow the code style (see below) and ideoms used in stable-baselines3.

The implementation quality is a strict requirements with little room for changes, because otherwise the implementation can do more harm than good (wrong results). Parts two and three are taken into account during review but being a repository for more experimental code, these are not very strict.

How to implement your suggestion

Implement your feature/suggestion/algorithm in following ways, using the first one that applies:

1. Environment wrapper: This can be used with any algorithm and even outside stable-baselines3. Place code for these under sb3_contrib/common/wrappers directory.
2. Custom callback. Place code under sb3_contrib/common/callbacks directory.
3. Following the style/naming of common files in the stable-baseline3. If your suggestion is a specific network architecture for feature extraction from images, place this in sb3_contrib/common/torch_layers.py, for example.
4. A new learning algorithm. This is the last resort but most applicable solution. Even if your suggestion is a (trivial) modification to an existing algorithm, create a new algorithm for it unless otherwise discussed (which inherits the base algorithm). The algorithm should use same API as stable-baselines3 algorithms (e.g. learn, load), and the code should be placed under sb3_contrib/[algorithm_name] directory.

Look over stable-baselines3 code for the general naming of variables and try to keep this style.

If algorithm you are implementing involves more complex/uncommon equations, comment each part of these calculations with references to the parts in paper.

Pull Request (PR) and review

Before proposing a PR, please open an issue, where the feature will be discussed. This prevent from duplicated PR to be proposed and also ease the code review process.

Each PR need to be reviewed and accepted by at least one of the maintainers. A PR must pass the Continuous Integration tests to be merged with the master branch.

Along with the code, PR must include the following:

1. README.md file for the feature (see template below). This is placed in the algorithm's directory.
2. Results of a replicated experiment from the original paper, which must match the results from authors unless solid arguments can be provided why they did not match.
3. The exact code to run the replicated experiment (i.e. it should produce the above results), and inside the code information about the environment used (Python version, library versions, OS, hardware information).
4. Updated tests in tests/test_run.py and tests/test_save_load.py to test that algorithms run as expected and serialize correctly. This this is not for testing e.g. training performance of a learning algorithm, and should be relatively quick to run.

README.md template:

# [Feature/Algorithm name]

* Non-abreviated name and/or one-sentence description of the method.
* Link and reference to the original publications the present the feature, or other established source(s).
* Links to any codebases that were used for reference (e.g. authors' implementations)

## Example

A minimal example on how to use the feature (full, runnable code).

## Results

A copy of results reported in the original paper and results obtained by your replicate of the experiments, as well
as an overview of the experiment setup (full details are in the code you will provide).

## Comments

Comments regarding the implementation, e.g. missing parts, uncertain parts, differences
to the original implementation.

## Changelog

Per-algorithm changelog, in format "dd/mm/yyyy username: comment". E.g:
* 05.10.2020 Miffyli: Adding missing instructions for contrib repo
* 04.10.2020 Miffyli: Initial commit

If you are not familiar with creating a Pull Request, here are some guides:

• http://stackoverflow.com/questions/14680711/how-to-do-a-github-pull-request
• https://help.github.com/articles/creating-a-pull-request/

Codestyle

We are using black codestyle (max line length of 127 characters) together with isort to sort the imports.

Please run make format to reformat your code. You can check the codestyle using make check-codestyle and make lint.

Please document each function/method and type them using the following template:

def my_function(arg1: type1, arg2: type2) -> returntype:
    """
    Short description of the function.

    :param arg1: describe what is arg1
    :param arg2: describe what is arg2
    :return: describe what is returned
    """
    ...
    return my_variable

Tests

All new features and algorithms must add tests in the tests/ folder ensuring that everything works fine (on program level). We use pytest. Also, when a bug fix is proposed, tests should be added to avoid regression.

To run tests with pytest:

make pytest

Type checking with pytype:

make type

Codestyle check with black, isort and flake8:

make check-codestyle
make lint

To run pytype, format and lint in one command:

make commit-checks

Changelog and Documentation

Please do not forget to update the changelog (CHANGELOG.rst).

Credits: this contributing guide is based on the PyTorch one.