Skip to content

Commit 6443a73

Browse files
wlangerawlangera
authored
Merge pull request #4 from b-cubed-eu/add-spatial-data
Add spatial data
2 parents 84fef6c + 940c7e1 commit 6443a73

7 files changed

Lines changed: 276 additions & 46 deletions

File tree

‎.gitignore‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@
1616
*.shx
1717
*.xls*
1818
*_files
19-
*.csv
2019
*.gz
21-
*.zip
20+
*.geojson
2221
docs
2322
libs
2423
output
2524
renv/library
25+
data/raw

‎README.Rmd‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,22 @@ This repository contains scripts to create the b3data [frictionless](https://doc
4646

4747
This code is developed in context of **T5.5** of the [B-Cubed project](https://b-cubed.eu/).
4848

49+
## Order of execution
50+
51+
Follow the steps below to run the scripts in a logical order.
52+
53+
**1.** `create_b3data_package.Rmd`
54+
55+
- creates the data package
56+
- adds tabular resources
57+
- writes data package
58+
59+
**2.** `add_spatial_resources.Rmd`
60+
61+
- loads data package
62+
- adds spatial resources
63+
- writes data package
64+
4965
## 📦 The `b3data` data package
5066

5167
- **Name**: `b3data`

‎README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,22 @@ on [Zenodo](https://zenodo.org/). <!-- description: end -->
3333
This code is developed in context of **T5.5** of the [B-Cubed
3434
project](https://b-cubed.eu/).
3535

36+
## Order of execution
37+
38+
Follow the steps below to run the scripts in a logical order.
39+
40+
**1.** `create_b3data_package.Rmd`
41+
42+
- creates the data package
43+
- adds tabular resources
44+
- writes data package
45+
46+
**2.** `add_spatial_resources.Rmd`
47+
48+
- loads data package
49+
- adds spatial resources
50+
- writes data package
51+
3652
## 📦 The `b3data` data package
3753

3854
- **Name**: `b3data`

‎data/b3data_package/datapackage.json‎

Lines changed: 48 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,16 @@
1010
"mediatype": "text/csv",
1111
"encoding": "utf-8",
1212
"title": "Occurrence cube for birds in Belgium (MGRS 10 km)",
13-
"description": "Occurrence cube for birds in Belgium between 2000 en 2024. The taxonomical resolution is 'species' and the temporal resolution is 'year' Spatial aggregation is done using the MGRS grid at 10 km scale.",
14-
"sources": ["GBIF Occurrence Download", "https://doi.org/10.15468/dl.y3wpwk"],
15-
"licenses": [" CC BY-NC 4.0", "https://creativecommons.org/licenses/by-nc/4.0/", "Creative Commons Attribution-NonCommercial 4.0 International"],
13+
"description": "Occurrence cube for birds in Belgium between 2000 en 2024. The taxonomical resolution is 'species' and the temporal resolution is 'year' Spatial aggregation is done using the MGRS grid at 10 km scale. Only grid cells that fall within the 10 km MGRS reference grid for mainland Belgium (see b3data: `mgrs10_refgrid_belgium.geojson`) are included.",
14+
"sources": {
15+
"title": "GBIF Occurrence Download",
16+
"path": "https://doi.org/10.15468/dl.y3wpwk"
17+
},
18+
"licenses": {
19+
"name": "CC BY-NC 4.0",
20+
"path": "https://creativecommons.org/licenses/by-nc/4.0/",
21+
"title": "Creative Commons Attribution-NonCommercial 4.0 International"
22+
},
1623
"schema": {
1724
"fields": [
1825
{
@@ -49,27 +56,47 @@
4956
}
5057
]
5158
}
59+
},
60+
{
61+
"name": "mgrs10_refgrid_belgium",
62+
"path": "mgrs10_refgrid_belgium.geojson",
63+
"profile": "spatial-data-resource",
64+
"format": "geojson",
65+
"title": "MGRS 10 Km reference grid Belgium",
66+
"description": "MGRS 10 Km reference grid for the mainland of Belgium.",
67+
"licenses": {
68+
"name": "CC0 1.0",
69+
"path": "https://creativecommons.org/publicdomain/zero/1.0/",
70+
"title": "Creative Commons Zero v1.0 Universal"
71+
}
5272
}
5373
],
5474
"title": "b3data: Data resources for the b3verse",
5575
"description": "This data package contains data resources to be used across the b3verse (https://docs.b-cubed.eu/guides/b3verse/). This includes example datasets (occurrence cubes) as well as spatial resources like reference grids or raster data.",
56-
"keywords1": "data cubes",
57-
"licenses.name": " CC BY-NC 4.0",
76+
"keywords": ["data cubes", "b3verse", "frictionless", "biodiversity"],
77+
"licenses": {
78+
"name": "CC BY-NC 4.0",
79+
"path": "https://creativecommons.org/licenses/by-nc/4.0/",
80+
"title": "Creative Commons Attribution-NonCommercial 4.0 International"
81+
},
5882
"version": "0.1.0",
59-
"sources.title": "b3data-scripts",
60-
"contributors.title": "Ward Langeraert",
61-
"contributors.path": "https://orcid.org/0000-0002-5900-8109",
62-
"contributors.email": "ward.langeraert@inbo.be",
63-
"contributors.role": "author",
64-
"contributors.organization": "Research Institute for Nature and Forest (INBO)",
65-
"contributors.title.1": "Toon Van Daele",
66-
"contributors.path.1": "https://orcid.org/0000-0002-1362-853X",
67-
"contributors.role.1": "contributor",
68-
"contributors.organization.1": "Research Institute for Nature and Forest (INBO)",
69-
"sources.path": "https://github.com/b-cubed-eu/b3data-scripts",
70-
"licenses.path": "https://creativecommons.org/licenses/by-nc/4.0/",
71-
"licenses.title": "Creative Commons Attribution-NonCommercial 4.0 International",
72-
"keywords2": "b3verse",
73-
"keywords3": "frictionless",
74-
"keywords4": "biodiversity"
83+
"sources": {
84+
"title": "b3data-scripts",
85+
"path": "https://github.com/b-cubed-eu/b3data-scripts"
86+
},
87+
"contributors": [
88+
{
89+
"title": "Ward Langeraert",
90+
"path": "https://orcid.org/0000-0002-5900-8109",
91+
"email": "ward.langeraert@inbo.be",
92+
"role": "author",
93+
"organization": "Research Institute for Nature and Forest (INBO)"
94+
},
95+
{
96+
"title": "Toon Van Daele",
97+
"path": "https://orcid.org/0000-0002-1362-853X",
98+
"role": "contributor",
99+
"organization": "Research Institute for Nature and Forest (INBO)"
100+
}
101+
]
75102
}

‎source/R/add_manual_resource.R‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
add_manual_resource <- function(package, new_resource, replace = TRUE) {
2+
if (replace) {
3+
# Filter out any resource with same name or path
4+
package$resources <- Filter(function(res) {
5+
!(res$name == new_resource$name && res$path == new_resource$path)
6+
}, package$resources)
7+
} else {
8+
# If not overwriting, check for duplicates and warn
9+
conflict <- any(vapply(package$resources, function(res) {
10+
res$name == new_resource$name && res$path == new_resource$path
11+
}, logical(1)))
12+
13+
if (conflict) {
14+
warning(paste("Resource with same name and path already exists and",
15+
"`replace = FALSE`. Skipping."))
16+
return(package)
17+
}
18+
}
19+
20+
# Append new resource
21+
package$resources <- append(package$resources, list(new_resource))
22+
return(package)
23+
}

‎source/add_spatial_resources.Rmd‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: "Add spatial resources to b3data data package"
3+
author: "Ward Langeraert"
4+
date: "`r Sys.Date()`"
5+
output:
6+
html_document:
7+
code_folding: hide
8+
toc: true
9+
toc_float: true
10+
toc_collapsed: true
11+
editor_options:
12+
chunk_output_type: console
13+
---
14+
15+
```{r setup, include=FALSE}
16+
knitr::opts_chunk$set(echo = TRUE)
17+
```
18+
19+
```{r, warning=FALSE, message=FALSE}
20+
# Load packages
21+
library(tidyverse) # Data wrangling and visualisation
22+
library(frictionless) # Create frictionless data package
23+
library(sf) # Spatial objects
24+
25+
# Source functions
26+
source(here::here("source", "R", "add_manual_resource.R"))
27+
28+
# Data path and create directory if necessary
29+
data_path <- here::here("data", "raw")
30+
dir.create(data_path, showWarnings = FALSE, recursive = TRUE)
31+
32+
package_path <- here::here("data", "b3data_package")
33+
dir.create(package_path, showWarnings = FALSE, recursive = TRUE)
34+
```
35+
36+
# Goal
37+
38+
Add spatial data resources to b3data frictionless data package.
39+
40+
# Methods
41+
42+
This report focuses on adding spatial resources (e.g. GeoJSON, raster) to the **b3data** frictionless data package.
43+
44+
## Source Data
45+
46+
Spatial data need to be saved locally in the data package directory and then added manually to the descriptor file using `append()`.
47+
We provide the reference grids for the occurrence cubes in the data package following this naming convention:
48+
49+
```
50+
resource_name = "<grid><resolution>refgrid<region>"
51+
```
52+
53+
Example:
54+
`mgrs10_refgrid_belgium` → MGRS 10 km reference grid Belgium
55+
56+
## Resource-level Metadata
57+
58+
Each dataset includes the following metadata (see: [Frictionless resource spec](https://docs.ropensci.org/frictionless/articles/data-resource.html#properties-implementation)):
59+
60+
- **profile**: `spatial-data-resource`
61+
- **format**: e.g. `geojson`
62+
- **title**: e.g. `"MGRS 10 km reference grid for Belgium"`
63+
- **description**: concise explanation of content
64+
- **sources**: source if applicable
65+
- **licenses**: [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)
66+
67+
# Datasets
68+
## MGRS 10 Km reference grid Belgium
69+
70+
Load the data.
71+
72+
```{r}
73+
# Read reference grid
74+
utm10_bel <- st_read(file.path(data_path, "utm10_bel.shp"))
75+
76+
# Visualise
77+
ggplot() + geom_sf(data = utm10_bel)
78+
```
79+
80+
Clean data and transform cell codes to MGRS.
81+
82+
```{r}
83+
mgrs10_refgrid_belgium <- utm10_bel %>%
84+
select(utm_tag = TAG, geometry) %>%
85+
mutate(mgrscode = ifelse(grepl("^[A-G]", utm_tag), paste0("31U", utm_tag),
86+
paste0("32U", utm_tag))) %>%
87+
select(mgrscode, geometry)
88+
```
89+
90+
Write to data package and add metadata.
91+
92+
```{r}
93+
st_write(mgrs10_refgrid_belgium,
94+
file.path(package_path, "mgrs10_refgrid_belgium.geojson"),
95+
delete_dsn = TRUE)
96+
```
97+
98+
```{r}
99+
# Read package
100+
b3data_package <- read_package(file.path(package_path, "datapackage.json"))
101+
102+
# Add resource to data package
103+
mgrs10_resource <- list(
104+
name = "mgrs10_refgrid_belgium",
105+
path = "mgrs10_refgrid_belgium.geojson",
106+
profile = "spatial-data-resource",
107+
format = "geojson",
108+
title = "MGRS 10 Km reference grid Belgium",
109+
description = "MGRS 10 Km reference grid for the mainland of Belgium.",
110+
licenses = list(
111+
name = "CC0 1.0",
112+
path = "https://creativecommons.org/publicdomain/zero/1.0/",
113+
title = "Creative Commons Zero v1.0 Universal"
114+
)
115+
)
116+
117+
b3data_package <- add_manual_resource(
118+
b3data_package, mgrs10_resource, replace = TRUE)
119+
120+
# Write package to directory
121+
write_package(
122+
package = b3data_package,
123+
directory = package_path,
124+
compress = TRUE)
125+
```
126+
127+
## EEA 100 km reference grid Europe
128+
129+
> Coming soon

0 commit comments

Comments
 (0)