Which setup to use AI privately on Graphene?

Gebdax

Hello,
I'm wondering, in your opinion which is the best setup to use Ai securely and privately, i'm not at all an expert in LLM but i was asked advice about that for someone considering graphene. With pretty high threat model
Which models ? In PWA ? With orbot or VPN ?

If you have ressources i'd be happy to read it
Thanks in advance

Graphene1

Gebdax you can mask your IP and use email aliases but ultimately your still feeding the AI information when using the service.

Duck.ai and Lumo appear to be somewhat useful and likely collect less data than ChatGPT.

DeletedUser405

Gebdax

For local AI there's Google AI Edge Gallery
https://github.com/google-ai-edge/gallery

Graphene1 Duck.ai and Lumo

Both of them connect to other servers, with Duck.ai in particular just proxying chats with models and being subject to their privacy policy. With Duck.ai ChatGPT, OpenAI won't directly get your IP address, but they now store chat lofs indefinitely

Eirikr70

Bear in mind the power consumption of AI if that is relevant for you.

r134a

For fully local AI u cloud look into ChatterUI.

wonder75

A private way can be to host your own LLM with Ollama on a machine you own.
(https://ollama.com/)

There are many clients available for Android that can connect to Ollama instances.
For example Ollama App
(https://github.com/JHubi1/ollama-app)