Extending GaNDLF

Environment

Before starting to work on the code-level on GaNDLF, please follow the instructions to install GaNDLF from sources. Once that's done, please verify the installation using the following command:

# continue from previous shell
(venv_gandlf) $> 
# you should be in the "GaNDLF" git repo
(venv_gandlf) $> python ./gandlf_verifyInstall

Submodule flowcharts

• The following flowcharts are intended to provide a high-level overview of the different submodules in GaNDLF.
• Navigate to the README.md file in each submodule folder for details.

Overall Architecture

• Command-line parsing: gandlf_run
• Parameters from training configuration get passed as a dict via the config manager
• Training Manager:
  • Handles k-fold training
  • Main entry point from CLI
• Training Function:
  • Performs actual training
• Inference Manager:
  • Handles inference functionality
  • Main entry point from CLI
• Inference Function:
  • Performs actual inference

Dependency Management

To update/change/add a dependency in setup, please ensure at least the following conditions are met:

• The package is being actively maintained.
• The new dependency is being testing against the minimum python version supported by GaNDLF (see the python_requires variable in setup).
• It does not clash with any existing dependencies.

Adding Models

• For details, please see README for GANDLF.models submodule.
• Update Tests

Adding Augmentation Transformations

• Update or add dependency in setup, if appropriate.
• Add transformation to global_augs_dict, defined in GANDLF/data/augmentation/__init__.py
• Ensure probability is used as input; probability is not used for any preprocessing operations
• For details, please see README for GANDLF.data.augmentation submodule.
• Update Tests

Adding Preprocessing functionality

• Update or add dependency in setup, if appropriate; see section on Dependency Management for details.
• All transforms should be defined by inheriting from torchio.transforms.intensity_transform.IntensityTransform. For example, please see the threshold/clip functionality in the GANDLF/data/preprocessing/threshold_and_clip.py file.
• Define each option in the configuration file under the correct key (again, see threshold/clip as examples)
• Add transformation to global_preprocessing_dict, defined in GANDLF/data/preprocessing/__init__.py
• For details, please see README for GANDLF.data.preprocessing submodule.
• Update Tests

Adding Training Functionality

• Update Training Function
• Update Training Manager, if any training API has changed
• Update Tests

Adding Inference Functionality

• Update Inference Function
• Update Inference Manager, if any inference API has changed
• Update Tests

Update parameters

For any new feature, please ensure the corresponding option in the sample configuration is added, so that others can review/use/extend it as needed.

Update Tests

Once you have made changes to functionality, it is imperative that the unit tests be updated to cover the new code. Please see the full testing suite for details and examples.

Run Tests

Prerequisites

There are two types of tests: unit tests for GaNDLF code, which tests the functionality, and integration tests for deploying and running mlcubes. Some additional steps are required for running tests:

1. Ensure that the install optional dependencies [ref] have been installed.
2. Tests are using sample data, which gets downloaded and prepared automatically when you run unit tests. Prepared data is stored at ${GaNDLF_root_dir}/testing/data/ folder. However, you may want to download & explore data by yourself.

Unit tests

Once you have the virtual environment set up, tests can be run using the following command:

# continue from previous shell
(venv_gandlf) $> pytest --device cuda # can be cuda or cpu, defaults to cpu

Any failures will be reported in the file ${GaNDLF_HOME}/testing/failures.log.

Integration tests

All integration tests are combined to one shell script:

# it's assumed you are in `GaNDLF/` repo root directory
cd testing/
./test_deploy.sh

Code coverage

The code coverage for the unit tests can be obtained by the following command:

bash
# continue from previous shell
(venv_gandlf) $> coverage run -m pytest --device cuda; coverage report -m