[Question] Model size limitations

[BohdanSol]

Can somebody define the maximum model size for llama/MLC?

I've tried a lot of models under 2Gb on llama, but when I try to use ~10Gb it fails to download.
I think there is limitation on react-native-blob-util, since I tried both downloading methods: via downloadModel and straight via react-native-blob-util.
Sometimes for >8Gb models isModelDownloaded returns true (but not consistent on different size models).

For example from https://huggingface.co/lmstudio-community/gemma-2-2b-it-GGUF/tree/main here's breakdown:

• (1.92Gb) gemma-2-2b-it-Q5_K_M.gguf -> The largest model I got successfully
• (2.15Gb) gemma-2-2b-it-Q6_K.gguf -> Model where the issue starts

[BohdanSol]

UPD: There might be Platform limitation from what I've read after submitting the ticket.
So for Android I've found this:
https://stackoverflow.com/questions/1071858/java-creating-byte-array-whose-size-is-represented-by-a-long/10787175#10787175
That is implemented here: https://github.com/RonRadtke/react-native-blob-util/blob/a2ddf875a4f52d4e1324b41bc87daf13571e1ff5/android/src/main/java/com/ReactNativeBlobUtil/ReactNativeBlobUtilFS.java#L255

Basically probably can be bypassed if rewritten to Map, but I'm not that much familiar with native Java to support more

[BohdanSol]

UPD: I've bypassed the limitations for downloading size on this fork (Android only for now): https://github.com/BohdanSol/react-native-blob-util/tree/master, and created PR in core repo: RonRadtke/react-native-blob-util#452
Successfully installed:

• (10.5Gb) gemma-2-2b-it-f32.gguf -> Pixel 7 Pro device speed is dramatically slow (I think expected)
• (2.15Gb) gemma-2-2b-it-Q8_0.gguf -> Noticeable speed change, but not critical
• (12.4Gb) unsloth/Qwen3.6-27B-GGUF/Qwen3.6-27B-Q3_K_S.gguf -> ~23-30 sec to prepare, but haven't got success generation after ~30 min of waiting (Pixel 7 Pro)

Overall on larger models (>~3Gb) I've caught few times when model.prepare() caused app crashes, not sure if production ready, but is pretty much usable.