Skip to content

Commit 4c77468

Browse files
committed
docs(batch): mark Azure Blob upload complete — all final products in sd-tpos2023-full-v01
1 parent 31f4e18 commit 4c77468

1 file changed

Lines changed: 23 additions & 6 deletions

File tree

scripts/batch_processing/PROCESSING_REPORT.md

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -336,13 +336,30 @@ scp -r oceanstream@20.223.137.44:/mnt/data/output/heatmaps/ ./heatmaps/
336336
scp -r oceanstream@20.223.137.44:/mnt/data/output/campaign_echograms/ ./echograms/
337337
```
338338

339-
### Upload to Azure Blob (not yet done)
339+
### Upload to Azure Blob (completed 11 Apr 2026)
340+
341+
All final products uploaded to container `sd-tpos2023-full-v01` on storage account `ne1osvmdevtest`.
342+
343+
| Product | Blob prefix | Files | Size |
344+
|---------|-------------|-------|------|
345+
| Combined per-day zarrs | `2023-XX-XX/*--combined--*.zarr/` | 221,102 | ~82 GB |
346+
| Campaign MVBS | `campaign_mvbs_combined_38kHz.zarr/` | 2,681 | 9.5 GB |
347+
| Campaign echograms | `campaign_echograms/` | 12 | 593 MB |
348+
| Per-day echograms | `perday_echograms/` | 1,610 | 3.3 GB |
349+
| PMTiles + GeoJSON | `tiles/` | 2 | 2 MB |
350+
| NASC biomass | `nasc_biomass/` | 1 | 1.5 MB |
351+
| NASC heatmaps | `heatmaps/` | 7 | 656 KB |
352+
353+
Per-pulse-mode intermediates (`*--short_pulse--*`, `*--long_pulse--*`) were **not** uploaded.
340354

341355
```bash
342-
# From the VM:
343-
azcopy sync "/mnt/data/output/sd-tpos2023-full-v01" \
344-
"https://ne1osvmdevtest.blob.core.windows.net/sd-tpos2023-full-v01" \
345-
--recursive
356+
# Read a combined zarr from Azure (Python)
357+
import xarray as xr
358+
ds = xr.open_zarr(
359+
"az://sd-tpos2023-full-v01/2023-07-15/2023-07-15--combined--mvbs.zarr",
360+
storage_options={"account_name": "ne1osvmdevtest"}
361+
)
362+
print(ds)
346363
```
347364

348365
---
@@ -384,7 +401,7 @@ azcopy sync "/mnt/data/output/sd-tpos2023-full-v01" \
384401

385402
| Item | Priority | Notes |
386403
|------|----------|-------|
387-
| Upload to Azure Blob | High | `azcopy sync` to `sd-tpos2023-full-v01` container |
404+
| ~~Upload to Azure Blob~~ | ~~High~~ | ✅ All final products in `sd-tpos2023-full-v01` container |
388405
| 200 kHz campaign MVBS | Medium | Debug channel matching for short_pulse 200 kHz |
389406
| ~~Per-day echograms~~ | ~~Low~~ | ✅ 1,610 PNGs via `run_combine_daily.py` |
390407
| Fix 5 failed NASC zarrs | Low | Edge cases: NaN depth, zero distance, corrupt zarr |

0 commit comments

Comments
 (0)