Skip to content
This repository was archived by the owner on Apr 28, 2025. It is now read-only.

Commit f95456d

Browse files
MelissaGrahamMelissaGraham
committed
use queryDatasets and w_2021_25
1 parent 1e063be commit f95456d

1 file changed

Lines changed: 40 additions & 27 deletions

File tree

‎04_Intro_to_Butler.ipynb‎

Lines changed: 40 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@
66
"source": [
77
"<img align=\"left\" src = https://project.lsst.org/sites/default/files/Rubin-O-Logo_0.png width=250> \n",
88
"<b>Introduction to the LSST data Butler</b> <br>\n",
9-
"Last verified to run on <b>TBD</b> with LSST Science Pipelines release <b>TBD</b> <br>\n",
9+
"Last verified to run on <b>Jun 17 2021</b> with LSST Science Pipelines release <b>w_2021_25</b> <br>\n",
1010
"Contact author: Alex Drlica-Wagner <br>\n",
11-
"Credit: Originally developed by Alex Drlica-Wagner in the context of the LSST Stack Club <br>\n",
11+
"Credit: Originally developed by Alex Drlica-Wagner in the context of the LSST Stack Club. <br>\n",
1212
"Target audience: All DP0 delegates. <br>\n",
1313
"Container Size: medium <br>\n",
14-
"Questions welcome at <a href=\"https://community.lsst.org/c/support/dp0\">community.lsst.org/c/support/dp0</a> <br>\n",
15-
"Find DP0 documentation and resources at <a href=\"https://dp0-1.lsst.io\">dp0-1.lsst.io</a> <br>"
14+
"Questions welcome at <a href=\"https://community.lsst.org/c/support/dp0\">community.lsst.org/c/support/dp0</a>. <br>\n",
15+
"Find DP0 documentation and resources at <a href=\"https://dp0-1.lsst.io\">dp0-1.lsst.io</a>. <br>"
1616
]
1717
},
1818
{
@@ -437,7 +437,10 @@
437437
"metadata": {},
438438
"source": [
439439
"To query for dimension records or datasets that overlap an arbitrary time range, we can use the `bind` argument to pass times through to `where`.\n",
440-
"Use this, along with [astropy.time](https://docs.astropy.org/en/stable/time/index.html), to look for visits within one minute of this one on either side."
440+
"Using `bind` to define an alias for a variable saves us from having to string-format the times into the `where` expression.\n",
441+
"Note that a `dafButler.Timespan` will accept a `begin` or `end` value that is equal to `None` if it is unbounded on that side.\n",
442+
"\n",
443+
"Use `bind` and `where`, along with [astropy.time](https://docs.astropy.org/en/stable/time/index.html), to look for visits within one minute of this one on either side."
441444
]
442445
},
443446
{
@@ -446,22 +449,32 @@
446449
"metadata": {},
447450
"outputs": [],
448451
"source": [
449-
"import astropy.time\n",
450-
"minute = astropy.time.TimeDelta(60, format=\"sec\")\n",
451-
"timespan = dafButler.Timespan(record.timespan.begin - minute, record.timespan.end + minute)\n",
452+
"# import astropy.time\n",
453+
"# minute = astropy.time.TimeDelta(60, format=\"sec\")\n",
454+
"# timespan = dafButler.Timespan(record.timespan.begin - minute, record.timespan.end + minute)\n",
452455
"\n",
453-
"for visit in registry.queryDimensionRecords(\"visit\", where=\"visit.timespan OVERLAPS my_timespan\", bind={\"my_timespan\": timespan}):\n",
454-
" print(visit.id, visit.timespan, visit.physical_filter)"
456+
"# for visit in registry.queryDimensionRecords(\"visit\", where=\"visit.timespan OVERLAPS my_timespan\", \n",
457+
"# bind={\"my_timespan\": timespan}):\n",
458+
"# print(visit.id, visit.timespan, visit.physical_filter)"
455459
]
456460
},
457461
{
458-
"cell_type": "markdown",
462+
"cell_type": "code",
463+
"execution_count": null,
459464
"metadata": {},
465+
"outputs": [],
460466
"source": [
461-
"Using `bind` to define an alias for a variable saves us from having to string-format the times into the `where` expression.\n",
462-
"Unfortunately, there is a bug in `queryDatasets` that prevents `bind` from working there (fixed in `w_2021_25`).\n",
467+
"import astropy.time\n",
468+
"minute = astropy.time.TimeDelta(60, format=\"sec\")\n",
469+
"timespan = dafButler.Timespan(record.timespan.begin - minute, record.timespan.end + minute)\n",
463470
"\n",
464-
"Note that a `dafButler.Timespan` will accept a `begin` or `end` value that is equal to `None` if it is unbounded on that side."
471+
"datasetRefs = registry.queryDatasets(\"calexp\", where=\"visit.timespan OVERLAPS my_timespan\",\n",
472+
" bind={\"my_timespan\": timespan})\n",
473+
"\n",
474+
"for i, ref in enumerate(datasetRefs):\n",
475+
" print(ref)\n",
476+
" if i > 6:\n",
477+
" break"
465478
]
466479
},
467480
{
@@ -470,7 +483,10 @@
470483
"source": [
471484
"##### Spatial queries\n",
472485
"\n",
473-
"Arbitrary spatial queries are not supported, but we do have set of dimensions that correspond to different levels of the HTM (hierarchical triangular mesh) pixelization of the sky ([HTM primer](http://www.skyserver.org/htm/)).\n",
486+
"Arbitrary spatial queries are not supported at this time, such as the \"POINT() IN (REGION)\" example found in this [Butler queries](https://pipelines.lsst.io/v/weekly/modules/lsst.daf.butler/queries.html) documentation.\n",
487+
"In other words, at this time it is only possible to do queries involving regions that are already \"in\" the data repository, either because they are HTM pixel regions or because they are tract/patch/visit/visit+detector regions.\n",
488+
"\n",
489+
"Thus, for this example we use the set of dimensions that correspond to different levels of the HTM (hierarchical triangular mesh) pixelization of the sky ([HTM primer](http://www.skyserver.org/htm/)).\n",
474490
"The process is to transform a region or point into one or more HTM identifiers (HTM IDs), and then create a query using the HTM ID as the spatial data ID.\n",
475491
"The `lsst.sphgeom` library supports region objects and HTM pixelization in the LSST Science Pipelines.\n",
476492
"\n",
@@ -511,28 +527,25 @@
511527
"metadata": {},
512528
"outputs": [],
513529
"source": [
514-
"visits = registry.queryDimensionRecords(\"visit\", htm20=htm_id,\n",
515-
" where=\"visit.timespan OVERLAPS my_timespan\",\n",
516-
" bind={\"my_timespan\": timespan})\n",
517-
"exposures = registry.queryDimensionRecords(\"exposure\", htm20=htm_id,\n",
518-
" where=\"visit.timespan OVERLAPS my_timespan\",\n",
519-
" bind={\"my_timespan\": timespan})\n",
520-
"detectors = registry.queryDimensionRecords(\"detector\", htm20=htm_id,\n",
521-
" where=\"visit.timespan OVERLAPS my_timespan\",\n",
522-
" bind={\"my_timespan\": timespan})\n",
530+
"datasetRefs = registry.queryDatasets(\"calexp\", htm20=htm_id,\n",
531+
" where=\"visit.timespan OVERLAPS my_timespan\",\n",
532+
" bind={\"my_timespan\": timespan})\n",
523533
"\n",
524-
"for visit, exposure, detector in zip(visits, exposures, detectors):\n",
525-
" print(visit.id, visit.timespan, visit.physical_filter, exposure.tracking_ra, exposure.tracking_dec, detector.id)"
534+
"for i, ref in enumerate(datasetRefs):\n",
535+
" print(ref)\n",
536+
" if i > 6:\n",
537+
" break"
526538
]
527539
},
528540
{
529541
"cell_type": "markdown",
530542
"metadata": {},
531543
"source": [
532544
"Thus, with the above query, we have uniquely identified the visit and detector for our desired temporal and spatial constraints.\n",
545+
"\n",
533546
"Note that if a smaller HTM level is used (like 7), which is a larger sky pixel (~2200 arcseconds), the above query will return many more visits and detectors which overlap with that larger region. Try it and see!\n",
534547
"\n",
535-
"Note that queries using the HTM ID can also be used to, e.g., find the set of all `src` catalog data products that overlap this point in i."
548+
"Note that queries using the HTM ID can also be used to, e.g., find the set of all i-band `src` catalog data products that overlap this point."
536549
]
537550
},
538551
{

0 commit comments

Comments
 (0)