- Edited
000000 Is there an AI i can use on Grapheneos? Privacy friendly offline on my device?
No, unfortunately not. Even smaller models like 8B or 14B models require like 10 GB of RAM to run, and a very strong CPU. Or a similarly sized GPU. This is generally not possible to run on phones yet. For one, if the CPUs or GPUs in phones were powerful enough, you would still drain your fully charged battery by just asking 3-5 questions. And model sizes below 8B are generally worthless, too stupid for anything.
But you can run 8B and 14B models on most modern computers. To get started, download LM Studio, and then install Gemma 2 8B, Qwen 2.5 14B, Llama 3.1 8B or another top performing model in that parameter range. LM Studio does not leak your chats, it runs entirely locally on your computer, and works perfectly even if you disconnects the internet. All while being beginner friendly.
And as @yore said, you need to go all the way up to 70B models or the like to be competitive with GPT4, and now we are talking about a computer with 128 GB RAM and four serial connected 24 GB VRAM GPUs. The 8B/14B models are comparable with the original ChatGPT version, not GPT4 or other modern ones.