Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .copier-answers.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Changes here will be overwritten by Copier
# DO NOT MANUALLY MODIFY THIS FILE
# !!!!! DO NOT MANUALLY MODIFY THIS FILE !!!!!
_commit: latest
_src_path: https://github.com/calliope-project/data-module-template
_src_path: https://github.com/modelblocks-org/data-module-template
author_email: s.pfenninger@tudelft.nl
author_family_name: Pfenninger-Lee
author_given_name: Stefan
github_org: calliope-project
github_org: modelblocks-org
license: Apache-2.0
module_description: ''
module_description: This module performs geospatial analyses to determine the available
land area for specific technologies, on a by-pixel basis and aggregated to given
geographic boundaries.
module_long_name: Area potentials
module_short_name: module_area_potentials
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A clear and concise description of what you expected to happen.
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. Linux Fedora 41, Windows 10...]
- OS: [e.g. Linux Fedora 43, Windows 11...]
- Version [e.g. v0.1.1]

**Additional context**
Expand Down
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: General information
url: https://www.modelblocks.org/
about: Please consult our website for general information on the Modelblocks methodology.
11 changes: 8 additions & 3 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ Fixes #

## Reviewer checklist

* [ ] `INTERFACE.yaml` is up-to-date with all relevant user resources and results.
* [ ] The integration example is up-to-date with a minimal use-case of the module.
* [ ] Module documentation is up-to-date.
* [ ] There are no `pip` dependencies in the module's environment files (`workflow/envs/`).
* [ ] All rules use `pathvars` (e.g., `<results>`) in their inputs and outputs.
* [ ] The integration test-suite is successful, including:
* [ ] `pre-commit.ci` tests pass.
* [ ] tests pass for all relevant OS configurations (linux, osx, windows).
* [ ] Module documentation is up-to-date, including:
* [ ] `INTERFACE.yaml` mentions all relevant `pathvars` and `wildcards`.
* [ ] `README.md` describes how to use the module and has the necessary citations.
6 changes: 5 additions & 1 deletion .github/workflows/check-version.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
# Check for changes in the upstream template. If changes are found, an issue is created
name: Template check.
on:
workflow_dispatch:
schedule:
- cron: '0 0 1 * *' # Runs at 00:00 UTC on the 1st day of every month

jobs:
copier-update:
uses: calliope-project/data-module-template/.github/workflows/template-check-version.yml@latest
permissions:
contents: read
issues: write
uses: modelblocks-org/data-module-template/.github/workflows/template-check-version.yml@latest
40 changes: 36 additions & 4 deletions .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,41 @@
name: CI
# !!!!! DO NOT MODIFY UNLESS ABSOLUTELY NECESSARY !!!!!
# This workflow helps standardise integration tests across data modules
name: Pull Request CI tests for Modelblocks data modules
on:
pull_request:
branches:
- "main"

jobs:
ci-tests:
uses: calliope-project/data-module-template/.github/workflows/template-pr-ci.yml@latest
build:
name: Build
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.12"]
steps:
- uses: actions/checkout@v4
- name: Setup pixi
uses: prefix-dev/setup-pixi@v0.8.3
- name: Run integration tests
id: tests
run: pixi run test-integration
continue-on-error: true
- name: Run snakemake linting
id: linting
run: |
pixi run snakemake --lint --snakefile workflow/Snakefile
pixi run snakemake --lint --snakefile tests/integration/Snakefile
continue-on-error: true
- name: Save integration logs
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: integration-test-logs-${{ matrix.os }}
path: tests/integration/resources/module/logs
if-no-files-found: ignore
retention-days: 30
- name: Fail if integration or linting failed
if: ${{ steps.tests.outcome == 'failure' || steps.linting.outcome == 'failure' }}
run: exit 1
4 changes: 3 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ on:

jobs:
release-workflow:
uses: calliope-project/data-module-template/.github/workflows/template-release.yml@latest
permissions:
contents: write
uses: modelblocks-org/data-module-template/.github/workflows/template-release.yml@latest
5 changes: 3 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ repos:
# Repo quality
- id: check-added-large-files
args: [--enforce-all]
exclude: ^pixi\.lock$
- id: forbid-submodules
- id: check-case-conflict
- id: check-illegal-windows-names
Expand All @@ -32,7 +33,7 @@ repos:

# Snakemake file formatting
- repo: https://github.com/snakemake/snakefmt
rev: v0.10.2
rev: v0.11.4
hooks:
- id: snakefmt

Expand All @@ -44,4 +45,4 @@ repos:
files: .*\.(py|smk|md)$|^Snakefile$

ci: # https://pre-commit.ci/
autoupdate_schedule: monthly
autoupdate_schedule: quarterly
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# https://citation-file-format.github.io/
cff-version: 1.2.0
message: If you use this software or data produced by it, please cite it using the metadata from this file.
title: "clio - module_area_potentials: Area potentials"
title: "Modelblocks - module_area_potentials: Area potentials"
repository: "https://github.com/calliope-project/module_area_potentials"
license: Apache-2.0
authors:
Expand Down
37 changes: 25 additions & 12 deletions INTERFACE.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
# Module Input-Output structure for automated doc. generation
resources:
user:
"shapes/{shape}.parquet": "Region geometries in parquet format. These should conform to the schema defined in https://github.com/calliope-project/module_geo_boundaries/blob/main/workflow/internal/shape.schema.yaml"
"wdpa.gdb": "WDPA protected areas database from https://www.protectedplanet.net/ in GeoDB format (choose 'File Geodatabase' when downloading)."
results:
"{shape}/{subunit}/area_potential_{tech}.tif": "Area potential GeoTIFF raster for the specified technology and subunit."
"{shape}/{subunit}/area_potential_{tech}.png": "Area potential GeoTIFF raster for the specified technology and subunit."
"{shape}/area_potential_{tech}.tif": "Area potential GeoTIFF raster for the specified technology across all subunits."
"{shape}/area_potential_{tech}.png": "Area potential PNG for the specified technology across all subunits."
"{shape}/area_potential_report.csv": "CSV summary report of area potential for all techs, across all subunits."
"{shape}/area_potential_report.html": "HTML summary report of area potential for all techs, across all subunits."
pathvars:
snakemake_defaults:
logs:
default: "<logs>"
description: location of rule log files.
resources:
default: "<resources>"
description: "location of module resource files."
results:
default: "<results>"
description: "location of module results."
user_resources:
shapes:
default: "<resources>/user/shapes/{shape}.parquet"
description: >
Region geometries in parquet format.
These should conform to the schema defined in https://github.com/calliope-project/module_geo_boundaries/blob/main/workflow/internal/shape.schema.yaml.
wdpa:
default: "<resources>/user/wdpa.gdb"
description: >
WDPA protected areas database from https://www.protectedplanet.net/ in GeoDB format (choose 'File Geodatabase' when downloading).
results:
area_potential:
default: "<results>/{shape}/area_potential_{tech}.tif"
description: "Area potential GeoTIFF raster for the specified technology across all subunits."
wildcards:
shape: "Name of the shape to be processed, e.g., 'world', 'europe', 'MEX'."
subunit: "Name of a subunit from within the shape, e.g 'IRL' for Ireland within 'europe'."
tech: "Name of the technology, e.g., 'pv_rooftop' or 'wind_offshore'. Available technologies are defined in the module configuration."
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2025 AUTHORS
Copyright 2026 AUTHORS

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
121 changes: 109 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,108 @@
# Area potentials

This module performs geospatial analyses to determine the available land area for specific technologies, on a by-pixel basis and aggregated to given geographic boundaries.

<!-- Place an attractive image of module outputs here -->
<p align="center">
<img src="./figures/area_potential_wind_onshore.png" width="70%">
</p>

A modular `snakemake` workflow built for [`clio`](https://clio.readthedocs.io/) data modules.
## About
<!-- Please do not modify this templated section -->

## Using this module
This is a modular `snakemake` workflow created as part of the [Modelblocks project](https://www.modelblocks.org/). It can be imported directly into any `snakemake` workflow.

This module can be imported directly into any `snakemake` workflow.
Please consult the integration example in `tests/integration/Snakefile` for more information.
For more information, please consult the Modelblocks [documentation](https://modelblocks.readthedocs.io/en/latest/),
the [integration example](./tests/integration/Snakefile),
and the `snakemake` [documentation](https://snakemake.readthedocs.io/en/stable/snakefiles/modularization.html).

## Overview
<!-- Please describe the processing stages of this module here -->

Data processing steps:

<p align="center">
<img src="./figures/rulegraph.png" width="66%">
</p>


* Geospatial input data (vector and raster) are acquired. This is automatic for most data, but the following data need to be manually supplied:
* Geographic boundaries in the parquet format
* WDPA protected areas database from https://www.protectedplanet.net/ in GeoDB format (choose 'File Geodatabase' when downloading).
* For the extent of the provided boundaries, the input data are reprojected and rasterised to the resolution of the land cover data (GlobCover), and merged into a single dataset for further processing.
* Based on the supplied configuration, the land-use analysis is done for each defined technology.
* Results can be reported on a per-geography and per-technology basis, as pixel surface area values (TIFF files), images for reporting purposes (PNG files), and also as a summary report with per-region capacities (CSV and HTML files)

See below for the [data sources](#references).

## Configuration
<!-- Please describe how to configure this module below -->

Please consult the configuration [README](./config/README.md) and the [configuration example](./config/config.yaml) for a general overview on the configuration options of this module.

In the configuration, you can define any number of `techs`, and for each of them, specify the `initial_area`, `continuous_layers`, and `binary_layers`.

By example, here is a `pv_rooftop` tech. We use the `settlement_area`, which is the settlement area in m² in each pixel, as the initial area from which the further analysis proceeds. In `continuous_layers`, we use the `settlement_share`, which is the share (0-1) of area covered by settlement, and exclude pixels with less than 0.01 settlement share while assuming that of those pixels not excluded by that, 0.8 (80%) of the settled area can be used for rooftop PV. Finally, in the `binary_layers`, we include all land use types except `NOT_SUITABLE` (since the main selection is done via the settlement_share). This means that, for example, `FOREST` pixels with a `settlement_area` > 0 can be included.

```yaml
pv_rooftop:
initial_area: settlement_area
continuous_layers:
settlement_share:
min: 0.01
max: 1
share: 0.8
binary_layers:
regions_maritime: 0
regions_land: 1
protected: 0
landcover_FARM: 1
landcover_FOREST: 1
landcover_URBAN: 1
landcover_OTHER: 1
landcover_NOT_SUITABLE: 0
landcover_WATER: 0
```

Here is a `wind_offshore` example. We start with the `pixel_area`, the total surface area in m² for each pixel. We include pixels with a slope up to and including 20 degrees, and exclude pixels with a settlement share above 0.01. Furthermore, we include only land areas (`regions_land: 1` and `regions_maritime: 0`) and completely exclude some areas like protected areas or urban areas (`protected: 0`, `landcover_URBAN: 0`), while including only a fraction of other areas (e.g. if a pixel is considered farmland, only 20% of its surface is available: `landcover_FARM: 0.2`).

```yaml
wind_onshore:
initial_area: pixel_area
continuous_layers:
slope:
min: 0
max: 20
settlement_share:
min: 0
max: 0.01
binary_layers:
regions_maritime: 0
regions_land: 1
protected: 0
landcover_FARM: 0.2
landcover_FOREST: 0.05
landcover_URBAN: 0
landcover_OTHER: 0.3
landcover_NOT_SUITABLE: 0
landcover_WATER: 0

```


## Input / output structure
<!-- Please describe input / output file placement below -->

Please consult the [interface file](./INTERFACE.yaml) for more information.

## Development
<!-- Please do not modify this templated section -->

We use [`pixi`](https://pixi.sh/) as our package manager for development.
Once installed, run the following to clone this repo and install all dependencies.
Once installed, run the following to clone this repository and install all dependencies.

```shell
git clone git@github.com:calliope-project/module_area_potentials.git
git clone git@github.com:modelblocks-org/module_area_potentials.git
cd module_area_potentials
pixi install --all
```
Expand All @@ -26,16 +113,26 @@ For testing, simply run:
pixi run test-integration
```

To view the documentation locally, use:

```shell
pixi run serve-docs
```

To test a minimal example of a workflow using this module:

```shell
pixi shell # activate this project's environment
cd tests/integration/ # navigate to the integration example
snakemake --use-conda --cores 2 # run the workflow!
```

## References
<!-- Please provide thorough referencing below -->

This module is based on the following research and datasets:

* [GEDTM30](https://github.com/openlandmap/GEDTM30) for slope
* License: Creative Commons Attribution 4.0 International
* [GlobCover land cover data](https://due.esrin.esa.int/page_globcover.php)
* License: "You may use the GlobCover land cover map for educational and/or scientific purposes, without any fee on the condition that you credit ESA and the Université Catholique de Louvain as the source of the GlobCover products."
* [GEBCO (General Bathymetric Chart of the Oceans)](https://www.gebco.net/data-products/gridded-bathymetry-data) 15 arc-second data
* License: "The GEBCO Grid is placed in the public domain and may be used free of charge. [...] Users must: Acknowledge the source of The GEBCO Grid. A suitable form of attribution is given in the documentation that accompanies The GEBCO Grid."
* [GHSL (Global Human Settlement Layer)](https://human-settlement.emergency.copernicus.eu/download.php) built-up surface data (R2023, GHS-BUILT-S, 100m resolution)
* License: "The GHSL has been produced by the EC JRC as open and free data. Reuse is authorised, provided the source is acknowledged."
* [WDPA (World Database on Protected Areas)](https://www.protectedplanet.net/)
* License: Non-commercial allowed. Citation: "UNEP-WCMC and IUCN (2025), Protected Planet: The World Database on Protected Areas (WDPA) and World Database on Other Effective Area-based Conservation Measures (WD-OECM) [Online], June 2025, Cambridge, UK: UNEP-WCMC and IUCN. Available at: www.protectedplanet.net."
15 changes: 7 additions & 8 deletions config/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# Configuration
We recommend consulting the following before using this module:
- `config/config.yaml`: a generic example configuration of this module.
- `workflow/internal/config.schema.yaml`: a schematic overview of all the configuration options of this module.
- `INTERFACE.yaml`: lists module input and output files, and their default locations.
- `tests/integration/Snakefile`: an example of how to call this module from another workflow.

This workflow is part of the [clio project](https://clio.readthedocs.io/en/latest/).
Please consult our documentation for more details.
Other useful resources are:

- `INTERFACE.yaml`: user input files (placed in `resources/user`) and module output files (placed in `results`).
- `workflow/internal/config.schema.yaml`: general configuration options.
- `tests/integration/`: a simple example of how to use this module.
This data module is part of the [Modelblocks](https://www.modelblocks.org/) project.
Please consult the [Modelblocks documentation](https://modelblocks.readthedocs.io/) for more details.
Loading
Loading