AI on your device ?

Hat

All new PC, Mac, phone are released with AI chip on soc today. This is a good thing for privacy, because you can run AI locally more efficiently.

I know that smartphones are far less powerful than computers.

So do you use AI on GrapheneOS, and which AI app ?
Do you use AI locally on your computer already ?
For what kind of tasks do you use it ?

missing-root

There is "Image upscaler". I used it to drain my 7pro's battery before replacing the screen lol. Because a full battery is said to be more dangerous in heat.

But running an LLM, best even training it on curated datasets would be cool.

Hazelfoot

Distaste for companies data mining me to train AI models is the reason I use GrapheneOS in the first place.

zzz

This is a good thing for privacy, because you can run AI locally more efficiently.

Maybe my comment is unwelcome here, but I feel the need to point out - many AI applications are built on abuses of privacy, using training data that is impossible for the human creators of that data to meaningfully consent to being co-opted into the model.

For some, the best thing for privacy is to resist shiny new AI toys altogether, and discourage its use outside of a very narrow range of public datasets free of licensing claims or abusive data collection practices.

Here is an example of an AI application that is good or neutral for privacy and other human rights:
https://hrdag.org/tech-notes/large-language-models-IPNO.html

Here is an example that is definitely not:
https://openai.com/chatgpt

Here's a relevant article:
https://www.businessinsider.com/tech-updated-terms-to-use-customer-data-to-train-ai-2023-9

I understand that this is perhaps offtopic. Just food for thought for privacy enthusiasts contemplating their personal use of AI.

Hat

Poketpal AI, you can download small language models. You can also benchmark your device and have a look at the results of other devices.

Local AI (from Ishan Vohra) has a voice mode, but the application crashes when launched, the log looks like the application is trying to access restricted resources.

Laurent

I use chatGPT because it is a very useful tool, just like any tool.

Graphene1

MapleAI on the surface seems interesting. I have it installed as a PWA.. Might be worth looking into

NetRunner88

I use ollama on my PC and ollama app on my phone to interact with it. You can't expect to have good tk/s speed with embedded model or maybe if they are total worthless garbage.

DeletedUser127

In one of the other discussions Private AI was suggested.

https://play.google.com/store/apps/details?id=us.valkon.privateai

It tested very well for me. Just choose and download one of the free language models, revoke network and you're good to go offline.

r134a

I use ollama app for android to connect to my selfhosted server running ollama with different models. I mainly use this option since my server is more performant than my phone, i.e. faster generation of responses.

I also use ChatterUI, which can run (small) models on device. I find 3b models acceptable in terms of speed (pixel 9 pro). 8b models also work, though slower. I mainly use this as my offline mode AI, mostly as a small offline 'wikipedia'.