Getting Started for Developers¶
Project Structure¶
| File / Directory | Purpose |
|---|---|
.github | CI/CD workflow definitions and a PR template |
binned_cdf | Project import modules |
docs | Documentation directory (better write docs there instead of readme.md) |
tests | Python module unit- & integration tests |
.pre-commit-config.yaml | git hook definitions comsumed by pre-commit |
license.md | The license in its long form |
mkdocs.yml | Documentation config consumed by mkdocs |
pyproject.toml | Project information, dependencies and task runner configurations |
readme.md | General project overview, displayed when visiting GitHub repository |
uv.lock | Contains the locked dependencies to exactly reproduce installations. |
Dependency Management & Packaging¶
To keep the dependencies of different projects from interfering with each other, it is highly recommended to create an isolated python environment for every project. We use uv to address this issue. By running uv sync inside the project directory, a new virtual environment is created automatically into which all your dependencies are installed (from the uv.lock file). This is different from running pip install . in an isolated virtual environment as this might use different dependency versions. Afterwards you can run any command within the virtual environment by simply calling
Testing¶
Executing
will run pytest, compute the test coverage and fail if below the minimum coverage defined by the tool.coverage.report.fail_under threshold in the pyproject.toml file.
Documentation¶
The code documentation is based on mkdocs which converts markdown files into a nicely-rendered web-page. In particular, we use the mkdocs-material package which offers more than just theming. To generate documentation for different versions, mike is used as a plugin within mkdocs.
To build and develop docs on a local server, run
To deploy the docs to the gh-pages remote branch, call
where <alias> may be any name alias for your version such as latest, stable or whatever.
The final documentation is located at:
Git Hooks¶
We use pre-commit to run git hooks helping you to develop high-quality code. The hooks are configured in the .pre-commit-config.yaml file and executed before commit.
For instance, ruff & ruff-format fix the code base in-place to adhere to reasonable coding standards. mypymypy & ruff lint the code for correctness. These tools are configured via pyproject.toml and .pre-commit-config.yaml files.
Installation
After you cloned this project and plan to develop in it, don't forget to install these hooks via
Available pre-commit hooks
minimum_pre_commit_version: "3.6.0"
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-added-large-files
- id: check-ast
- id: check-case-conflict
- id: check-merge-conflict
- id: check-shebang-scripts-are-executable
- id: check-symlinks
- id: check-toml
- id: check-yaml
args: ["--unsafe"]
- id: end-of-file-fixer
# - id: no-commit-to-branch # default: main, master
- id: trailing-whitespace
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.33.3
hooks:
- id: check-dependabot
- id: check-github-actions
- id: check-github-workflows
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.13.0
hooks:
- id: ruff-format
- id: ruff
args: ["--unsafe-fixes"]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.18.1
hooks:
- id: mypy
args: ["--explicit-package-bases"]
GitHub Actions¶
There are basic CI and CD pipelines, executed as GitHub Actions workflow when pushing changes or opening PR's.
Available workflows
name: Continuous Integration
on:
pull_request:
types: [opened, ready_for_review, reopened, synchronize]
workflow_call:
workflow_dispatch:
concurrency:
group: ${{ github.job }}/${{ github.workflow }}/${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
ci-default-python-version:
name: Default Python Version
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
timeout-minutes: 120
steps:
- name: Check out repository
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@v7
- name: Lint & test
uses: ./.github/actions/lint-test
- name: Add coverage comment
if: github.event_name == 'pull_request'
uses: MishaKav/pytest-coverage-comment@v1
with:
coverage-path-prefix: "binned_cdf/"
junitxml-path: pytest.xml
pytest-xml-coverage-path: coverage.xml
- name: Build & deploy temporary docs
if: github.event_name == 'pull_request' && github.event.pull_request.draft == false
uses: ./.github/actions/publish-docs
with:
alias: pr-${{ github.event.number }}
version: next-pr-${{ github.event.number }}
- name: Add docs comment
if: github.event_name == 'pull_request' && github.event.pull_request.draft == false
uses: marocchino/sticky-pull-request-comment@v2.9.4
with:
header: docs-comment
message: |
:books: Created [temporary docs](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}).
Useful URLs:
- [Coverage](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}/exported/coverage)
- [Tests](https://famura.github.io/binned-cdf/pr-${{ github.event.number }}/exported/tests)
ci-other-python-versions:
if: github.event.pull_request.draft == false
name: Other Python Versions
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.13]
timeout-minutes: 120
steps:
- name: Check out repository
uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@v7
- name: Set Python version to ${{ matrix.python-version }}
run: uv python pin ${{ matrix.python-version }}
- name: Lint & test
uses: ./.github/actions/lint-test
name: Continuous Deployment
on:
push:
branches: [main]
workflow_dispatch:
inputs:
bumped-version-part:
description: "The version part to bump."
type: choice
options:
- major
- minor
- patch
default: patch
required: true
concurrency:
group: ${{ github.workflow }}/${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
bump-version:
name: Bump Version
runs-on: ubuntu-latest
container: docker:git
permissions:
contents: write
timeout-minutes: 10
steps:
- name: Install prerequisites
run: apk add nodejs
- name: Check out repository
uses: actions/checkout@v6
- name: Bump version and push tag
id: version
uses: mathieudutour/github-tag-action@v6.2
with:
default_bump: ${{ github.event.inputs.bumped-version-part || 'patch' }}
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Add version info
run: echo "Bumped ${VERSION_PART} version part from ${OLD_TAG} to ${NEW_TAG}." >> $GITHUB_STEP_SUMMARY
env:
VERSION_PART: ${{ steps.version.outputs.release_type }}
OLD_TAG: ${{ steps.version.outputs.previous_tag }}
NEW_TAG: ${{ steps.version.outputs.new_tag }}
ci:
name: CI
needs: bump-version
uses: ./.github/workflows/ci.yaml
secrets: inherit
permissions:
contents: write
pull-requests: write
deploy:
name: Deploy Docs
needs: ci
if: github.repository == 'famura/binned-cdf'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
steps:
- name: Check out repository
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Set up uv
uses: astral-sh/setup-uv@v7
- name: Lint & test
uses: ./.github/actions/lint-test
- name: Build & publish docs
uses: ./.github/actions/publish-docs
publish-pypi:
name: Publish to PyPI
needs: deploy
runs-on: ubuntu-latest
environment: release
timeout-minutes: 15
permissions:
id-token: write
steps:
- name: Check out repository
uses: actions/checkout@v6
with:
fetch-depth: 0 # it is necessary for versioningit to get full git history
- name: Set up uv
uses: astral-sh/setup-uv@v7
- name: Build package
run: uv build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1