Skip to content

Add Dask documentation#529

Open
mohalkh5 wants to merge 8 commits intoResearchComputing:mainfrom
mohalkh5:dask
Open

Add Dask documentation#529
mohalkh5 wants to merge 8 commits intoResearchComputing:mainfrom
mohalkh5:dask

Conversation

@mohalkh5
Copy link
Copy Markdown
Contributor

This PR adds new documentation for using Dask on Alpine, motivated by Issue 433. The goal of this addition is to provide a clear, end-to-end guide covering:

  • Core Dask concepts
  • Setting up and using a distributed cluster
  • Accessing the Dask dashboard in Open OnDemand
  • Practical examples for common workflows

Copy link
Copy Markdown
Contributor

@SchneiderCode SchneiderCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great overview of utilizing Dask. Most of my comments below focus either on accessibility or personal preferences around readability/organization.

Comment thread docs/programming/dask.md
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md
Comment thread docs/programming/dask.md Outdated
```
This will generate a random array, and it will automatically create the tasks, and from there the sums will be parallelised. This is similar to what you would see in MPI, but much easier to implement.

![](./dask_images/dask-array-output2.png)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessibility: Alt Text

Check if Dask provides alt-text for the visualization.

Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md Outdated
Comment thread docs/programming/dask.md
```

```{figure} ./dask_images/dask-plugin-2.png
:alt: A screenshot of a Jupyter Notebook displaying Python code for Dask. The code imports libraries, sets up a LocalCluster with specific parameters, and shows cluster details such as the number of workers, total threads, and status.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:alt: A screenshot of a Jupyter Notebook displaying Python code for Dask. The code imports libraries, sets up a LocalCluster with specific parameters, and shows cluster details such as the number of workers, total threads, and status.
:alt: A screenshot of a Jupyter Notebook displaying Python code for Dask that imports libraries, sets up a LocalCluster with specific parameters, and shows cluster details such as the number of workers, total threads, and status. Described under the heading Setting up a Local Cluster.

Comment thread docs/programming/dask.md
```

```{figure} ./dask_images/dask-plugin-3.png
:alt: Screenshot of a Jupyter notebook interface displaying a code segment initializing a Dask client. The section labeled 'Client' shows connection details, including the connection method, cluster type, and a URL for dashboard access.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:alt: Screenshot of a Jupyter notebook interface displaying a code segment initializing a Dask client. The section labeled 'Client' shows connection details, including the connection method, cluster type, and a URL for dashboard access.
:alt: Screenshot of a Jupyter notebook interface displaying a code segment initializing a Dask client and the resulting connection details, including the connection method, cluster type, and a URL for dashboard access. Described under the heading Dask Dashboard.

Comment thread docs/programming/dask.md
After launching JupyterLab, open the Dask tab from the left sidebar. The Dask icon appears alongside other JupyterLab tools such as the file browser and notebook panel.

```{figure} ./dask_images/dask-plugin-1.png
:alt: Screenshot of a JupyterLab interface showing the Dask extension icon located in the left sidebar alongside notebook and file browser icons and displaying a code snippet for starting a local cluster on the right.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:alt: Screenshot of a JupyterLab interface showing the Dask extension icon located in the left sidebar alongside notebook and file browser icons and displaying a code snippet for starting a local cluster on the right.
:alt: Screenshot of a JupyterLab interface showing the Dask extension icon located in the left sidebar alongside notebook and file browser icons and displaying a code snippet for starting a local cluster on the right. Described under the heading Step 4: Use the Dask Extension.

Comment thread docs/programming/dask.md
```

```{figure} ./dask_images/dask-plugin-4.png
:alt: Screenshot showing a JupyterLab session and instructions for modifying the Dask dashboard URL by appending /proxy/8787/status for Open OnDemand access.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:alt: Screenshot showing a JupyterLab session and instructions for modifying the Dask dashboard URL by appending /proxy/8787/status for Open OnDemand access.
:alt: Screenshot showing a JupyterLab session and instructions for modifying the Dask dashboard URL by appending /proxy/8787/status for Open OnDemand access. Described under heading Step 5: Connecting to the Dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants