Keep data in memory after _save

[astrojuanlu]

When declaring a dataset in the catalog, the data will be serialized (good). However, at the moment looks like it's an either/or situation: the user needs to choose between serializing the data and passing it in memory. The problem is that users many times want to do both. Typical reasons:

• The dataset in question doesn't have a _load method implemented, so it requires hacks
• Loading is possible, but expensive, troublesome, or has side effects

This week I was asked about this twice already by different users. It has been mentioned several times but I cannot locate specific issues or Slack conversations.

[astrojuanlu]

Looks like the answer to this is just to use CachedDataset? (thanks @noklam)

https://docs.kedro.org/en/stable/api/kedro.io.CachedDataset.html

test_ds:
   type: CachedDataset
   dataset:
      type: pandas.CSVDataset
      filepath: example.csv

[rashidakanchwala]

Closing this discussion as there's a solution highlighted above.