Skip to content

Getting Started for Developers

Project Structure

File / Directory Purpose
.github CI/CD workflow definitions and a PR template
binned_cdf Project import modules
docs Documentation directory (better write docs there instead of readme.md)
tests Python module unit- & integration tests
.pre-commit-config.yaml git hook definitions comsumed by pre-commit
license.md The license in its long form
mkdocs.yml Documentation config consumed by mkdocs
pyproject.toml Project information, dependencies and task runner configurations
readme.md General project overview, displayed when visiting GitHub repository
uv.lock Contains the locked dependencies to exactly reproduce installations.

Dependency Management & Packaging

To keep the dependencies of different projects from interfering with each other, it is highly recommended to create an isolated python environment for every project. We use uv to address this issue. By running uv sync inside the project directory, a new virtual environment is created automatically into which all your dependencies are installed (from the uv.lock file). This is different from running pip install . in an isolated virtual environment as this might use different dependency versions. Afterwards you can run any command within the virtual environment by simply calling

uv run <command>

Testing

Executing

uv run pytest --cov

will run pytest, compute the test coverage and fail if below the minimum coverage defined by the tool.coverage.report.fail_under threshold in the pyproject.toml file.

Documentation

The code documentation is based on mkdocs which converts markdown files into a nicely-rendered web-page. In particular, we use the mkdocs-material package which offers more than just theming. To generate documentation for different versions, mike is used as a plugin within mkdocs.

To build and develop docs on a local server, run

uv run mkdocs serve

To deploy the docs to the gh-pages remote branch, call

uv run mike deploy --push --update-aliases <version> <alias>

where <alias> may be any name alias for your version such as latest, stable or whatever.

The final documentation is located at:

https://famura.github.io/binned-cdf/<alias>

Git Hooks

We use pre-commit to run git hooks helping you to develop high-quality code. The hooks are configured in the .pre-commit-config.yaml file and executed before commit.

For instance, ruff & ruff-format fix the code base in-place to adhere to reasonable coding standards. mypymypy & ruff lint the code for correctness. These tools are configured via pyproject.toml and .pre-commit-config.yaml files.

Installation

After you cloned this project and plan to develop in it, don't forget to install these hooks via

uv run pre-commit install
Available pre-commit hooks
minimum_pre_commit_version: "3.6.0"
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v6.0.0
    hooks:
      - id: check-added-large-files
      - id: check-ast
      - id: check-case-conflict
      - id: check-merge-conflict
      - id: check-shebang-scripts-are-executable
      - id: check-symlinks
      - id: check-toml
      - id: check-yaml
        args: ["--unsafe"]
      - id: end-of-file-fixer
      # - id: no-commit-to-branch # default: main, master
      - id: trailing-whitespace

  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.33.3
    hooks:
      - id: check-dependabot
      - id: check-github-actions
      - id: check-github-workflows

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.13.0
    hooks:
      - id: ruff-format
      - id: ruff
        args: ["--unsafe-fixes"]

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.18.1
    hooks:
      - id: mypy
        args: ["--explicit-package-bases"]

GitHub Actions

There are basic CI and CD pipelines, executed as GitHub Actions workflow when pushing changes or opening PR's.

Available workflows
name: Continuous Integration

on:
    pull_request:
        types: [opened, ready_for_review, reopened, synchronize]
    workflow_call:
    workflow_dispatch:

concurrency:
    group: ${{ github.job }}/${{ github.workflow }}/${{ github.head_ref || github.ref }}
    cancel-in-progress: true

jobs:
    ci-default-python-version:
        name: Default Python Version
        runs-on: ubuntu-latest
        permissions:
            contents: write
            pull-requests: write
        timeout-minutes: 120

        steps:
            - name: Check out repository
              uses: actions/checkout@v6

            - name: Set up uv
              uses: astral-sh/setup-uv@v7

            - name: Lint & test
              uses: ./.github/actions/lint-test

            - name: Add coverage comment
              if: github.event_name == 'pull_request'
              uses: MishaKav/pytest-coverage-comment@v1
              with:
                  coverage-path-prefix: "binned_cdf/"
                  junitxml-path: pytest.xml
                  pytest-xml-coverage-path: coverage.xml

            - name: Build & deploy temporary docs
              if: github.event_name == 'pull_request' && github.event.pull_request.draft == false
              uses: ./.github/actions/publish-docs
              with:
                  alias: pr-${{ github.event.number }}
                  version: next-pr-${{ github.event.number }}

            - name: Add docs comment
              if: github.event_name == 'pull_request' && github.event.pull_request.draft == false
              uses: marocchino/sticky-pull-request-comment@v2.9.4
              with:
                  header: docs-comment
                  message: |
                      :books: Created [temporary docs](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}).

                      Useful URLs:

                      - [Coverage](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}/exported/coverage)
                      - [Tests](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}/exported/tests)

    ci-other-python-versions:
        if: github.event.pull_request.draft == false
        name: Other Python Versions
        runs-on: ubuntu-latest
        strategy:
            matrix:
                python-version: [3.13]
        timeout-minutes: 120

        steps:
            - name: Check out repository
              uses: actions/checkout@v6

            - name: Set up uv
              uses: astral-sh/setup-uv@v7

            - name: Set Python version to ${{ matrix.python-version }}
              run: uv python pin ${{ matrix.python-version }}

            - name: Lint & test
              uses: ./.github/actions/lint-test
name: Continuous Deployment

on:
    push:
        branches: [main]
    workflow_dispatch:
        inputs:
            bumped-version-part:
                description: "The version part to bump."
                type: choice
                options:
                    - major
                    - minor
                    - patch
                default: patch
                required: true

concurrency:
    group: ${{ github.workflow }}/${{ github.head_ref || github.ref }}
    cancel-in-progress: true

jobs:
    bump-version:
        name: Bump Version
        runs-on: ubuntu-latest
        container: docker:git
        permissions:
            contents: write
        timeout-minutes: 10
        steps:
            - name: Install prerequisites
              run: apk add nodejs

            - name: Check out repository
              uses: actions/checkout@v6

            - name: Bump version and push tag
              id: version
              uses: mathieudutour/github-tag-action@v6.2
              with:
                  default_bump: ${{ github.event.inputs.bumped-version-part || 'patch' }}
                  github_token: ${{ secrets.GITHUB_TOKEN }}

            - name: Add version info
              run: echo "Bumped ${VERSION_PART} version part from ${OLD_TAG} to ${NEW_TAG}." >> $GITHUB_STEP_SUMMARY
              env:
                  VERSION_PART: ${{ steps.version.outputs.release_type }}
                  OLD_TAG: ${{ steps.version.outputs.previous_tag }}
                  NEW_TAG: ${{ steps.version.outputs.new_tag }}

    ci:
        name: CI
        needs: bump-version
        uses: ./.github/workflows/ci.yaml
        secrets: inherit
        permissions:
            contents: write
            pull-requests: write

    deploy:
        name: Deploy Docs
        needs: ci
        if: github.repository == 'famura/binned-cdf'
        runs-on: ubuntu-latest
        timeout-minutes: 30
        permissions:
            contents: write
        steps:
            - name: Check out repository
              uses: actions/checkout@v6
              with:
                  fetch-depth: 0

            - name: Set up uv
              uses: astral-sh/setup-uv@v7

            - name: Lint & test
              uses: ./.github/actions/lint-test

            - name: Build & publish docs
              uses: ./.github/actions/publish-docs

    publish-pypi:
        name: Publish to PyPI
        needs: deploy
        runs-on: ubuntu-latest
        environment: release
        timeout-minutes: 15
        permissions:
            id-token: write
        steps:
            - name: Check out repository
              uses: actions/checkout@v6
              with:
                  fetch-depth: 0 # it is necessary for versioningit to get full git history

            - name: Set up uv
              uses: astral-sh/setup-uv@v7

            - name: Build package
              run: uv build

            - name: Publish to PyPI
              uses: pypa/gh-action-pypi-publish@release/v1