Skip to content

Contributing to program-synthesis-benchmarks

All interest in program-synthesis-benchmarks, as a user or contributor, is greatly appreciated! This document will go into detail on how to contribute to the development of program-synthesis-benchmarks.

Before Contributing

Before reading further we ask that you read our Code of Conduct which will be enforced by the maintainers in order to ensure that development of program-synthesis-benchmarks stays focused and productive.

If you are new to contributing to open source, or GitHub, the following links may be helpful starting places:

We Use Github Flow

This means that all code and documentation changes happen through pull requests. We actively welcome your pull requests. We highly recommend the following workflow.

  1. Fork the repo and create your branch from main.
  2. If you've added code that should be tested, add tests.
  3. If you've changed APIs, update the documentation.
  4. Ensure the test suite passes.
  5. Create the pull request.

Any contributions you make will be under the MIT Software License

In short, when you submit code changes, your submissions are understood to be under the same MIT License that covers the project. Feel free to contact the maintainers if that's a concern.

How to contribute a ...

Bug Report

We use GitHub issues to track public bugs. Report a bug by opening a new issue.

Great Bug Reports tend to have at least the following:

  • A quick summary and/or background
  • The steps to reproduce.
  • When possible, minimal code that reproduces the bug.
  • A description of what you expected versus what actually happens.

Feature Request

We like to hear all feature requests and discussion around the direction of the project. The best place to discuss future features is the project's discussion page under the Ideas & Requests category.

Bug fix, new feature, documentation improvement, or other change.

We welcome contribution to the codebase via pull requests. In most cases, it is beneficial to discuss your change with the community via a GitHub issue or discussion before working on a pull request. Once you decide to work on a pull request, please follow the workflow outlined in the above sections.

Once you open the pull request, it will be tested with by CI and reviewed by other contributors (including at least one project maintainer). After all iterations of review are finished, one of the project maintainers will merge your pull request.

Development Environment

This project uses Poetry to create and maintain python virual environments. Contributors should follow the following step to setup or update their virtual environment before starting any contribution that requires a code change. The steps are as follows:

  1. Install Poetry if you haven't already.
  2. If your contribution includes a change to pyproject.toml, run poetry lock to pick up changes.
  3. Run poetry install to setup the environment.

The project's virtual environment will have all dependencies (and dev dependcies) require to run the code and tests in the project.

Running Tests

After setting up the environment (see above) you can run the test suite using pytest. To ensure you are using the correct environment, it is easiest to run pytest using poetry.

poetry run pytest