Gemma 4 on Graphene OS

xyzalphabeta

Title says it all. Anyone tried running Gemma 4 on their GOS phone? With well managed storage scopes and it being fully local it shouldnt be a privacy risk afaik. Happy to stand corrected.

hmmm

I'm interested in this to use it as a basic digital assistant. What I'm trying to figure out is what apps do I use to interface with it that can work well on graphene OS.

bongocongo

Inference on AI Edge Gallery (made by Google) is decently fast, you need network permission + a HuggingFace account to download Gemma 4 but after that you can disable network. It's also possible to install models without network access if the model is in .litert format. Now if you use Termux you can install Ollama which potentially avoids Google, but inference is a little slower. Either way, I wouldn't expect serious intelligence right now since the models that fit on phones don't have a ton of parameters.

randomname

only a small models would run and it's going to use most of the phone ram you can just run bigger models on a pc and access it remotely, there is multiples apps like https://github.com/Vali-98/ChatterUI to use with lmstudio or ollama remote api