Skip to content

Commit f9341de

Browse files
authored
Merge pull request #14370 from KratosMultiphysics/doc/wheel-building
[Doc] Adding building wheels doc
2 parents 72cf280 + f3bbd17 commit f9341de

3 files changed

Lines changed: 162 additions & 1 deletion

File tree

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Kratos Wheel Generation: Understanding PyProject files
2+
3+
## TLDR
4+
5+
Here is a quick summary of how to generate Kratos wheels depending on your deployment target:
6+
7+
### 1. Generating a local release from the build dir
8+
After building Kratos with CMake, simply navigate to your newly generated release folder (which will contain the `libs` sub-folder with your `.so`/`.dll` files) and run the standard python build module from there:
9+
```bash
10+
cd bin/Release/
11+
python3 -m build --wheel
12+
```
13+
14+
### 2. Generating a bundled release (Unified Wheel)
15+
To build a single "fat" wheel containing the Kratos Core alongside all compiled Applications, use the provided package orchestration script. Inside `scripts/wheels/build_wheel.py`, ensure the flag `CURRENT_CONFIG['UNIFIED_WHEEL'] = True`, then execute it from the project root:
16+
```bash
17+
python3 scripts/wheels/build_wheel.py
18+
```
19+
20+
### 3. Generating a regular release (Different wheels for Core and Apps)
21+
To build an ecosystem of separate wheels (one main wheel for the Core, and one standalone wheel per Application), ensure exactly the opposite: `CURRENT_CONFIG['UNIFIED_WHEEL'] = False` (which is typically the default). Run the orchestrator:
22+
```bash
23+
python3 scripts/wheels/build_wheel.py
24+
```
25+
26+
## Introduction
27+
28+
In Kratos Multiphysics, wheel packaging is governed by several `pyproject.toml` files using standard PEP-517 tools with `hatchling` as the build system. This document outlines the purpose and critical features of the `pyproject.toml` files found in the core and scripts directories.
29+
30+
## Context: Target Execution Path
31+
32+
> [!WARNING]
33+
> While these files are physically authored at `kratos/pyproject.toml` and `scripts/wheels/pyproject.toml` (or similar), **they are not executed from these source locations**.
34+
35+
During the CMake configuration and build step, the required `pyproject.toml` files are orchestrated and copied to build-staging directories or final release binary directories. Wheels must be built from these target staging locations, as that is where all the compiled dynamic libraries (`.so` / `.dll`) and Python wrappers securely reside at build time.
36+
37+
## 1. Core vs Unified Wheel Generators
38+
39+
Kratos defines distinct strategies to package either the minimal core or a "fat" unified package containing all applications:
40+
41+
### The Core `pyproject.toml` (`./kratos`)
42+
**Purpose:** This configuration generates a wheel corresponding exclusively to the Kratos kernel. It bundles only the core files, the MPI core wrappers, and fundamental system orchestrators.
43+
**Location in Source:** `kratos/pyproject.toml`
44+
45+
### The Applications `pyproject.toml` (`./applications/[APP_NAME]/pyproject.toml`)
46+
**Purpose:** This configuration generates a wheel corresponding to the selected application. It is ment to be used along with the core pyproject.toml file to generate application wheels that depend on the core wheel.
47+
**Location in Source:** `kratos/pyproject.toml`
48+
49+
### The Unified `pyproject.toml` (`./scripts`)
50+
**Purpose:** This project builds Kratos and **all compiled applications** from your local environment as a single bundled package.
51+
**Location in Source:** `scripts/wheels/pyproject.toml`
52+
53+
## 2. `pyproject.toml` Structure
54+
55+
Both configs share a common anatomy dictated by the `hatchling` backend, but handle certain specifics differently.
56+
57+
### Standard Metadata (`[build-system]` and `[project]`)
58+
Defines Kratos Multiphysics as the package name, the backend used (`hatch.build`), target python versions, and dependencies:
59+
```toml
60+
[build-system]
61+
requires = ["hatchling"]
62+
build-backend = "hatchling.build"
63+
64+
[project]
65+
name = "KratosMultiphysics"
66+
dependencies = ["numpy>=1.20"]
67+
dynamic = ["version"]
68+
```
69+
70+
### The Kratos Specific Section `[kratos]` (Core only)
71+
Because `kratos/pyproject.toml` handles the core natively alongside application builds, it contains a `[kratos]` table explicitly defining the C++ libraries needed for the kernel build.
72+
73+
```toml
74+
[kratos]
75+
libs = [
76+
"Kratos.*",
77+
"KratosMPI.*",
78+
"KratosCore.*",
79+
# ...
80+
]
81+
```
82+
> [!NOTE]
83+
> This section is parsed by the Kratos wrapping orchestration script `build_wheel.py` to filter precisely which binaries belong to the core package versus applications and is not part of a standard pyproject.toml file. The unified pyproject omits this because it intentionally scoops up *everything* in the binary tree.
84+
85+
### Build Hooks `[tool.hatch.build.targets.wheel.hooks.custom]`
86+
Kratos wheels are strictly platform-specific and are not purely python code. These hooks execute a local `hatch_build.py` script which forces the final wheel to be recognized as system-specific.
87+
The hook script typically sets:
88+
- `build_data["infer_tag"] = True`
89+
- `build_data["pure_pyton"] = False`
90+
91+
This ensures the wheel filenames correspond exactly to the current OS architecture (e.g., `linux_x86_64` or `win_amd64`) instead of `any`. It also injects the `KRATOS_VERSION` environment variable.
92+
93+
## 3. `.py`, `.dll`, and `.so` Files
94+
95+
### Python source and Hinting files inclusion
96+
Python files native to the framework are injected explicitly via standard `include` directives.
97+
```toml
98+
[tool.hatch.build.targets.wheel]
99+
include = [
100+
"KratosMultiphysics/**/*.py",
101+
"KratosMultiphysics/**/*.pyi"
102+
]
103+
```
104+
105+
### Including Compiled C++ Artifacts (`.dll` / `.so`)
106+
Packaging `.dll` (Windows), `.so` (Linux) and `.dylib` (Mac) files side-by-side with Python code is structurally the most critical operation of Kratos wheels. Kratos relies on `hatchling`'s `force-include` mapping technique mapping:
107+
108+
```toml
109+
[tool.hatch.build.targets.wheel.force-include]
110+
libs = "KratosMultiphysics/.libs"
111+
```
112+
113+
**How it works:**
114+
1. A staging folder named `libs` is set up containing all required `.so` or `.dll` binaries.
115+
2. The key string (`libs`) in the TOML corresponds to this physical layout.
116+
3. The assigned value (`"KratosMultiphysics/.libs"`) forces `hatchling` to place the files into that destination directory structure inside the final `.whl` archive.
117+
4. Hence, all collected compiled `.so`/`.dll`/`.dylib` packages are pushed into the hidden `.libs` directory, correctly satisfying dynamic links at runtime.
118+
119+
---
120+
121+
## 4. The `build_wheel.py` Script
122+
123+
While it is possible to build wheels manually by fulfilling the `force-include` prerequisites, Kratos provides a robust automation script to handle this: `scripts/wheels/build_wheel.py`.
124+
125+
### Purpose
126+
The `build_wheel.py` script is the main driver for generating Kratos Python Wheels. It orchestrates the CMake build of Kratos Python interfaces and coordinates the PEP-517 wheel packaging across multiple operating systems and Python versions.
127+
128+
### Execution Flow & Usage
129+
130+
**1. Managing the Build Target (Unified vs Separate Packages)**
131+
The script contains a configuration rule `CURRENT_CONFIG['UNIFIED_WHEEL']`:
132+
- **If `False` (Packages Target):** The script sets up the build for the "Core" using `kratos/pyproject.toml`. It then dynamically detects all compiled directories inside `applications/*Application*` and generates a standalone `.whl` extension package for *each application*.
133+
- **If `True` (Unified Target):** The script executes a single package generation utilizing `scripts/wheels/pyproject.toml`. This creates one massive package containing both the core and all application libraries.
134+
135+
**2. Staging Environment (`setupWheelDir`)**
136+
It dynamically generates an isolated `WHEEL_ROOT` staging directory. For each iteration, the script copies:
137+
- The compiled Python wrappers (`.py` files).
138+
- The raw binary libraries (`.dll` / `.so` mapped to a folder called `libs`).
139+
- The relevant `pyproject.toml`.
140+
- The `README.md` and `hatch_build.py` hook.
141+
142+
**3. Binary Filtering Using `[kratos]` Config**
143+
Because applications and core share a master binary release folder (`bin/Release/libs`), the script performs intelligent filtering.
144+
It uses Python's `toml.load` to read the target `pyproject.toml`. If the file contains a `[kratos] libs = [...]` list (like `kratos/pyproject.toml`), the orchestration script aggressively iterates over the physics binaries within the staging `libs` directory and **deletes any binaries that do not match the declared patterns**.
145+
This strictly ensures that the Core wheel gets only Core binaries, and Application wheels get only their respective subsets.
146+
147+
**4. Executing Hatchling & Wheel Output**
148+
After staging and filtering, the orchestrator triggers the Python build module natively inside the temporary root:
149+
```python
150+
subprocess.run([python_interpreter, "-m", "build", "--wheel", ...])
151+
```
152+
Resulting `.whl` files are exported to the target `--outdir` (often `WHEEL_OUT`), and the staging configuration gets safely destroyed via `cleanupWheelDir`.
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"side_bar_name": "kratos_release",
3+
"landing_page": "Wheels/Building-Wheels.md",
4+
"additional_menu_options": {
5+
"product": "Release",
6+
"title": "sidebar"
7+
}
8+
}

docs/pages/Kratos/menu_info.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
"title": "Source Documentation",
1616
"url": "https://kratos-docs.onrender.com"
1717
},
18-
"Workshops"
18+
"Workshops",
19+
"Release"
1920
]
2021
}

0 commit comments

Comments
 (0)