Renamed RBF and TV benchmarks

Author: Markus28

docs/metrics/gaussian_tv_mmd.md

@@ -11,14 +11,14 @@
 
 ## Summary Benchmark
 
-::: polygraph.metrics.MMD2CollectionGaussianTV
+::: polygraph.metrics.GaussianTVMMD2Benchmark
     options:
         show_root_heading: true
         show_full_path: true
         show_source: false
         heading_level: 3
 
-::: polygraph.metrics.MMD2IntervalCollectionGaussianTV
+::: polygraph.metrics.GaussianTVMMD2BenchmarkInterval
     options:
         show_root_heading: true
         show_full_path: true

docs/metrics/rbf_mmd.md

@@ -11,14 +11,14 @@
 
 ## Summary Benchmark
 
-::: polygraph.metrics.MMD2CollectionRBF
+::: polygraph.metrics.RBFMMD2Benchmark
     options:
         show_root_heading: true
         show_full_path: true
         show_source: false
         heading_level: 3
 
-::: polygraph.metrics.MMD2IntervalCollectionRBF
+::: polygraph.metrics.RBFMMD2BenchmarkInterval
     options:
         show_root_heading: true
         show_full_path: true

docs/tutorials/basic_usage.md

@@ -30,22 +30,22 @@ print(planar_nx[0])         # (Networkx) Graph with 64 nodes and 177 edges
 When evaluating graph generative models, we want to compare a set of *generated* graphs to a set of *reference* graphs (typically the test set).
 In `polygraph`, we provide various different metrics to quantify how similar these two sets of graphs are.
 We usually pass collections of NetworkX graphs to metrics.
-Below, we demonstrate how a set of these metrics, combined in the [`MMD2CollectionGaussianTV`][polygraph.metrics.MMD2CollectionGaussianTV] benchmark may be computed:
+Below, we demonstrate how a set of these metrics, combined in the [`GaussianTVMMD2Benchmark`][polygraph.metrics.GaussianTVMMD2Benchmark] benchmark may be computed:
 
 ```python
-from polygraph.metrics import MMD2CollectionGaussianTV
+from polygraph.metrics import GaussianTVMMD2Benchmark
 
 reference = planar.to_nx()
 generated = sbm.to_nx()
 
-benchmark = MMD2CollectionGaussianTV(reference)
+benchmark = GaussianTVMMD2Benchmark(reference)
 print(benchmark.compute(generated))          # Dictionary of different metrics
 ```
 
 We discuss available metrics [in the next tutorial](metrics_overview.md).
 
 All metrics are evaluated in a similar fashion, as defined by the common [interface](../api_reference/metrics/interface.md):
 
-- We first initialize a metric object via `benchmark = MMD2CollectionGaussianTV(reference)`. This fits the metric to the `reference` set, caching data that is required in later computations
+- We first initialize a metric object via `benchmark = GaussianTVMMD2Benchmark(reference)`. This fits the metric to the `reference` set, caching data that is required in later computations
 - We then compute the metric against the generated set via `benchmark.compute(generated)`
 - We may call `benchmark.compute` repeatedly with different generated sets, e.g. over the course of training

docs/tutorials/metrics_overview.md

@@ -9,16 +9,16 @@ For convenience, `polygraph` allows metrics that follow this interface to be bun
 
 ```python
 from polygraph.metrics import MetricCollection
-from polygraph.metrics import MMD2CollectionRBF, MMD2CollectionGaussianTV
+from polygraph.metrics import RBFMMD2Benchmark, GaussianTVMMD2Benchmark
 from polygraph.datasets import PlanarGraphDataset, SBMGraphDataset
 
 reference_graphs = PlanarGraphDataset("val").to_nx()
 generated_graphs = SBMGraphDataset("val").to_nx()
 
 metrics = MetricCollection(
     metrics={
-        "rbf_mmd": MMD2CollectionRBF(reference_graphs),
-        "tv_mmd": MMD2CollectionGaussianTV(reference_graphs),
+        "rbf_mmd": RBFMMD2Benchmark(reference_graphs),
+        "tv_mmd": GaussianTVMMD2Benchmark(reference_graphs),
     }
 )
 print(metrics.compute(generated_graphs))        # Dictionary of metrics
@@ -31,17 +31,17 @@ We now proceed to give a high-level overview over the different types of metrics
 [Maximum Mean Discrepancy (MMD)](../api_reference/metrics/mmd.md) is the predominant method for comparing graph distributions.
 The two distributions are embedded in a reproducing kernel Hilbert space (RKHS) and their distance is then computed in this space.
 
-In `polygraph`, we bundle the most commonly used MMD metrics in two benchmark classes: [`MMD2CollectionGaussianTV`][polygraph.metrics.MMD2CollectionGaussianTV] and [`MMD2CollectionRBF`][polygraph.metrics.MMD2CollectionRBF]. These benchmarks may be evaluated in the following fashion:
+In `polygraph`, we bundle the most commonly used MMD metrics in two benchmark classes: [`GaussianTVMMD2Benchmark`][polygraph.metrics.GaussianTVMMD2Benchmark] and [`RBFMMD2Benchmark`][polygraph.metrics.RBFMMD2Benchmark]. These benchmarks may be evaluated in the following fashion:
 
 ```python
 from polygraph.datasets import PlanarGraphDataset, SBMGraphDataset
-from polygraph.metrics import MMD2CollectionGaussianTV, MMD2IntervalCollectionGaussianTV
+from polygraph.metrics import GaussianTVMMD2Benchmark, GaussianTVMMD2BenchmarkInterval
 
 reference = PlanarGraphDataset("val").to_nx()
 generated = SBMGraphDataset("val").to_nx()
 
 # Evaluate the benchmark with point estimates
-benchmark = MMD2CollectionGaussianTV(reference)
+benchmark = GaussianTVMMD2Benchmark(reference)
 print(benchmark.compute(generated))     # {'orbit': 1.067700488335175, 'clustering': 0.32549637224264394, 'degree': 0.3375409762261701, 'spectral': 0.0830197437100697}
 ```
 

polygraph/metrics/__init__.py

@@ -1,19 +1,19 @@
 from .base import MetricCollection
 from .polygraphscore import PGS5, PGS5Interval
 from .gaussian_tv_mmd import (
-    MMD2CollectionGaussianTV,
-    MMD2IntervalCollectionGaussianTV,
+    GaussianTVMMD2Benchmark,
+    GaussianTVMMD2BenchmarkInterval,
 )
-from .rbf_mmd import MMD2CollectionRBF, MMD2IntervalCollectionRBF
+from .rbf_mmd import RBFMMD2Benchmark, RBFMMD2BenchmarkInterval
 from .vun import VUN
 
 __all__ = [
     "VUN",
     "MetricCollection",
     "PGS5",
     "PGS5Interval",
-    "MMD2CollectionGaussianTV",
-    "MMD2IntervalCollectionGaussianTV",
-    "MMD2CollectionRBF",
-    "MMD2IntervalCollectionRBF",
+    "GaussianTVMMD2Benchmark",
+    "GaussianTVMMD2BenchmarkInterval",
+    "RBFMMD2Benchmark",
+    "RBFMMD2BenchmarkInterval",
 ]

polygraph/metrics/gaussian_tv_mmd.py

@@ -17,23 +17,23 @@
 
 
 Below, we demonstrate how to evaluate all metrics in the benchmark with point estimates and with uncertainty quantification.
-Note that the parameter `subsample_size` in [`MMD2IntervalCollectionGaussianTV`][polygraph.metrics.MMD2IntervalCollectionGaussianTV]
-should match the number of generated and reference graphs in [`MMD2CollectionGaussianTV`][polygraph.metrics.MMD2CollectionGaussianTV]
+Note that the parameter `subsample_size` in [`GaussianTVMMD2BenchmarkInterval`][polygraph.metrics.GaussianTVMMD2BenchmarkInterval]
+should match the number of generated and reference graphs in [`GaussianTVMMD2Benchmark`][polygraph.metrics.GaussianTVMMD2Benchmark]
 to obtain comparable results:
 
 ```python
 from polygraph.datasets import PlanarGraphDataset, SBMGraphDataset
-from polygraph.metrics import MMD2CollectionGaussianTV, MMD2IntervalCollectionGaussianTV
+from polygraph.metrics import GaussianTVMMD2Benchmark, GaussianTVMMD2BenchmarkInterval
 
 reference = list(PlanarGraphDataset("val").to_nx())
 generated = list(SBMGraphDataset("val").to_nx())
 
 # Evaluate the benchmark with point estimates
-benchmark = MMD2CollectionGaussianTV(reference[:20])
+benchmark = GaussianTVMMD2Benchmark(reference[:20])
 print(benchmark.compute(generated[:20]))
 
 # Evaluate the benchmark with uncertainty quantification
-benchmark_with_uncertainty = MMD2IntervalCollectionGaussianTV(
+benchmark_with_uncertainty = GaussianTVMMD2BenchmarkInterval(
     reference,
     subsample_size=20,
     num_samples=100,
@@ -71,8 +71,8 @@
 
 
 __all__ = [
-    "MMD2CollectionGaussianTV",
-    "MMD2IntervalCollectionGaussianTV",
+    "GaussianTVMMD2Benchmark",
+    "GaussianTVMMD2BenchmarkInterval",
     "GaussianTVOrbitMMD2",
     "GaussianTVOrbitMMD2Interval",
     "GaussianTVClusteringMMD2",
@@ -84,7 +84,7 @@
 ]
 
 
-class MMD2CollectionGaussianTV(MetricCollection):
+class GaussianTVMMD2Benchmark(MetricCollection):
     """Collection of MMD2 metrics using the Gaussian TV kernel.
 
     This graphs combines the following graph descriptors into one benchmark:
@@ -109,10 +109,10 @@ def __init__(self, reference_graphs: Collection[nx.Graph]):
         )
 
 
-class MMD2IntervalCollectionGaussianTV(MetricCollection):
+class GaussianTVMMD2BenchmarkInterval(MetricCollection):
     """Collection of MMD2 metrics using the Gaussian TV kernel with uncertainty quantification.
 
-    This class provides the same metrics as [`MMD2CollectionGaussianTV`][polygraph.metrics.MMD2CollectionGaussianTV] but with uncertainty quantification.
+    This class provides the same metrics as [`GaussianTVMMD2Benchmark`][polygraph.metrics.GaussianTVMMD2Benchmark] but with uncertainty quantification.
 
     Args:
         reference_graphs: Collection of reference graphs to fit the metric to.

polygraph/metrics/rbf_mmd.py

@@ -13,17 +13,17 @@
 
 ```python
 from polygraph.datasets import PlanarGraphDataset, SBMGraphDataset
-from polygraph.metrics import MMD2CollectionRBF, MMD2IntervalCollectionRBF
+from polygraph.metrics import RBFMMD2Benchmark, RBFMMD2BenchmarkInterval
 
 reference = list(PlanarGraphDataset("val").to_nx())
 generated = list(SBMGraphDataset("val").to_nx())
 
 # Evaluate the benchmark with point estimates
-benchmark = MMD2CollectionRBF(reference[:20])
+benchmark = RBFMMD2Benchmark(reference[:20])
 print(benchmark.compute(generated[:20]))
 
 # Evaluate the benchmark with uncertainty quantification
-benchmark_with_uncertainty = MMD2IntervalCollectionRBF(
+benchmark_with_uncertainty = RBFMMD2BenchmarkInterval(
     reference,
     subsample_size=20,
     num_samples=100,
@@ -58,8 +58,8 @@
 from polygraph.metrics.base import MetricCollection
 
 __all__ = [
-    "MMD2CollectionRBF",
-    "MMD2IntervalCollectionRBF",
+    "RBFMMD2Benchmark",
+    "RBFMMD2BenchmarkInterval",
     "RBFOrbitMMD2",
     "RBFOrbitMMD2Interval",
     "RBFClusteringMMD2",
@@ -73,7 +73,7 @@
 ]
 
 
-class MMD2CollectionRBF(MetricCollection):
+class RBFMMD2Benchmark(MetricCollection):
     """Collection of MMD2 metrics using RBF kernels with dynamic bandwidths."""
 
     def __init__(self, reference_graphs: Collection[nx.Graph]):
@@ -88,7 +88,7 @@ def __init__(self, reference_graphs: Collection[nx.Graph]):
         )
 
 
-class MMD2IntervalCollectionRBF(MetricCollection):
+class RBFMMD2BenchmarkInterval(MetricCollection):
     """Collection of MMD2 metrics using RBF kernels with dynamic bandwidths and uncertainty quantification."""
 
     def __init__(

tests/test_mmd.py

@@ -39,10 +39,10 @@
     RBFGraphNeuralNetworkMMD2,
 )
 from polygraph.metrics import (
-    MMD2CollectionGaussianTV,
-    MMD2IntervalCollectionGaussianTV,
+    GaussianTVMMD2Benchmark,
+    GaussianTVMMD2BenchmarkInterval,
 )
-from polygraph.metrics import MMD2CollectionRBF, MMD2IntervalCollectionRBF
+from polygraph.metrics import RBFMMD2Benchmark, RBFMMD2BenchmarkInterval
 from polygraph.utils.kernels import LinearKernel
 from polygraph.utils.graph_descriptors import WeisfeilerLehmanDescriptor
 from polygraph.utils.mmd_utils import mmd_from_gram
@@ -332,15 +332,15 @@ def test_mmd_collections(datasets, variant):
             "spectral": RBFSpectralMMD2(planar),
             "gin": RBFGraphNeuralNetworkMMD2(planar),
         }
-        benchmark = MMD2CollectionRBF(planar)
+        benchmark = RBFMMD2Benchmark(planar)
     elif variant == "gaussian_tv":
         separate_metrics = {
             "orbit": GaussianTVOrbitMMD2(planar),
             "clustering": GaussianTVClusteringMMD2(planar),
             "degree": GaussianTVDegreeMMD2(planar),
             "spectral": GaussianTVSpectralMMD2(planar),
         }
-        benchmark = MMD2CollectionGaussianTV(planar)
+        benchmark = GaussianTVMMD2Benchmark(planar)
     else:
         raise ValueError(f"Invalid variant: {variant}")
 
@@ -357,9 +357,9 @@ def test_mmd_collections(datasets, variant):
     )
 
     if variant == "rbf":
-        metric = MMD2IntervalCollectionRBF(planar, subsample_size=16)
+        metric = RBFMMD2BenchmarkInterval(planar, subsample_size=16)
     elif variant == "gaussian_tv":
-        metric = MMD2IntervalCollectionGaussianTV(planar, subsample_size=16)
+        metric = GaussianTVMMD2BenchmarkInterval(planar, subsample_size=16)
     else:
         raise ValueError(f"Invalid variant: {variant}")