[Feature]: Request: Need to load encrypted models. Some local offline models require encryption, please add the corresponding methods.

[qishuo1688]

Background & Description

Request: Need to load encrypted models. Some local offline models require encryption, please add the corresponding methods.

API & Usage

No response

How to implement

No response

[martindevans]

Could you include a link to the functions in llama.cpp that support loading encrypted models? If that's something llama.cpp supports then it should hopefully be fairly easy to add an overload to the c# model loading methods :)

[qishuo1688]

As far as I know, llama.cpp currently does not support loading encrypted models nor provide any related functions for encrypted models. Could LlamaSharp add such a feature additionally?

[martindevans]

LlamaSharp is purely a wrapper around llama.cpp, so we don't generally add many features on top of what llama.cpp offers.

We could probably add a general purpose method to load from a Stream, that way it would integrate better into the normal flow of c# file loading. Would that work for you?

[qishuo1688]

Perfect plan, that sounds great to me!