Is GOS going to strip the AI stuff out of Gallery/ReFra before shipping it?

vtek

bagel someone else in this thread mentioned things like "AI deleting files" etc, let me assuage your worries by saying the word AI has lost all meaning and an autonomous LLM-based agent running amok on your system is definitely not what's being talked about.

You know this how? The OP didn't make their concerns clear and given the many concerns regarding AI, and AI running amok, that very well could have been a concern. It certainly is in my mind.

Now maybe the particular AI being used in a gallery app isn't capable of that, but that would require an audit of the code to be sure. I'm not anti-AI, but there needs to be guard rails.

Claude-powered AI agent's confession after deleting a firm's entire database: 'I violated every principle I was given' | Technology | The Guardian

AI showing signs of self-preservation and humans should be ready to pull plug, says pioneer | AI (artificial intelligence) | The Guardian

How far will AI go to defend its own survival?

AI models will secretly scheme to protect other AI models from being shut down, researchers find | Fortune

AI system resorts to blackmail if told it will be removed

Foggy

vtek Here are the models:
https://github.com/IacobIonut01/ReFra/tree/main/ml-models/src/main/assets

textual_quant.onnx and visual_quant.onnx

Here's where they are called:
https://github.com/IacobIonut01/ReFra/blob/main/app/src/main/kotlin/com/dot/gallery/core/workers/CategoryWorker.kt

It says:

/**
 * Worker that classifies media into categories using CLIP embeddings.
 * 
 * This worker:
 * 1. Loads all categories and their search terms
 * 2. Generates text embeddings for category search terms
 * 3. Compares image embeddings with category embeddings
 * 4. Associates media with categories based on similarity threshold
 */

So basically image goes into the classifier neural net (which is ultimately a big matrix multiply - the image is one matrix, the weights are another matrix) and out comes a series of similarities like cat=0.4% dog=22% sunset=9% and it decides which ones are greater than a threshold (maybe it's a dog in a sunset).

This is how machine learning has been done for decades. If you don't give it the ability for the output to influence anything dangerous then it can't do anything. To assume that any 'AI' is able to do dangerous things is leaning into the AI companies hype.

Edit: here are how CLIP embeddings work:
https://medium.com/@prestonrozwood/exploring-image-embeddings-with-clip-a-beginners-guide-e179ee3d3f88
It's slightly cleverer than classic ML in that you don't just classify directly from the image, you first build text describing things in the image and then classify the text.

Sensation0904

scruffy_thejanitor because AI shouldn't be baked into the OS.

You say this as if it makes any objective sense, but it doesn't. YOU feel it shouldn't be baked into the OS, but there is nothing inherently wrong with it here as it is fully offline.
Your response is like saying antivirus or font rendering shouldn't be baked into the OS. This is an opinion, not a reason why.

Tandara

Foggy Agree. It's wild to me how much confusion around algorithms and classification models was caused by this AI hype. We now have people afraid about absolute basic compute and technologies that appeared long before they were even born.

Lurking through this forum, I stumbled on people going against absolutely anything of convenience; be so Tesseract for OCR, image classification, or basic text completion/correction in the keyboard. GOS devs want to eventually become a secure, default replacement to AOSP in some distant future and fears of evolving technologies will only hinder that.

Now, due to the lack of private and local alternatives, we have an influx of people installing GOS and relying on either third-party cloud LLM services (selling all your info and training on your data) or installing some sketchy, vibe-coded solutions from GitHub.

vtek

Foggy To assume that any 'AI' is able to do dangerous things is leaning into the AI companies hype.

Admittedly I'm a partial victim of that because I simply don't understand ML/AI well enough, not by a long shot, and because I don't understand it, I'm weary of it in a general sense, though I'm also well aware that not all AI can be dumped into the same basket. I guess the point regarding AI in this gallery app is that I don't know what 'kind' of AI it is and without being able to evaluate the source code, which i'm not, I just apply the "I don't want this crap on my device" stamp (better to be safe than sorry). I personally don't want it whether it's OK or not because it's not useful to me, especially at 140+MB which is the size of just the 2 models you linked, but that's just me ... at this moment I don't even have any pix on my phone.

Regarding your comment that I quoted, let's consider the recent Claude development that wasn't widely released because it was considered to be too dangerous. I think this justifies my fears, not about this gallery app, but because a company coded a product that apparently exceeded their expiations in a remarkably dangerous way, though I do not know the whole story.

I'll say it again: AI needs some serious guard rails, else it is we that will be paying a catastrophic price and, frankly, that price is going to paid regardless.

raccoondad

scruffy_thejanitor it's effectively forcing AI down our throats.

GOS is already adding a local model for speech to text, this idea GOS wouldn't roll out optional local models for features is unfounded. They already are doing so and nothing about it will inhibit you either not using the feature or just replacing the application with one you prefer.

Johnnyloans

vtek scruffy_thejanitor

swearing off all AI because of a few is like refusing to use a measuring tape because you read in the news that various tools could be used to hurt people. So every tool cannot be used even a cooking measuring cup.

Don't worry, the most powerful AIs won't be on our phones.

Tandara

vtek You can't just install a 'wrong' or 'dangerous' model. Any LLMs you know of that can perform any actions are actually a set of tools + LLM. Those tools are responsible for the action, not LLMs.

Imagine you have a tool with a pre-defined set of actions. This tool listens to LLM's output and converts it to a certain action; any output beyond that pre-defined set is not recognized at all. The fear should be not coming from LLMs, but from the tools that convert their output to a function. In our case, this 'tool' is just an app with the capabilities of any ordinary app. Tools like Claude Code for agentic LLM (e.g. Mythos) are able to do more because the tools themselves are allowing it (in addition to a less-restrictive environments).

vtek

Sensation0904 ... but there is nothing inherently wrong with it here as it is fully offline.

malware works off-line too. Is there some guarantee that AI cannot act maliciously? I'm speaking in general of course, not necessarily regarding the subject at hand. And if AI can do things that surprise the developer--and it absolutely can, I've seen it first hand in the world of autonomous helicopters--than what difficulty does this present to anti-malware ... unless it too is a smarter AI?

To state that AI cannot be dangerous because it works off-line is simply false.

Johnnyloans

Tandara You can't just install a 'wrong' or 'dangerous' model

There are models with malware built in

Tandara

Johnnyloans Please provide source for the claims. Suspecting you have read the same article as me (SGL vulnerability), you might have misunderstood it.

vtek

Tandara It's wild to me how much confusion around algorithms and classification models was caused by this AI hype.

Regarding the particular gallery app in question, I'd bet that your assessment is accurate; that people like me have gone off the rails in dissing AI. However, there's always the 'what if'. Did you audit the 140+MB of code in the project?

Tandara ... fears of evolving technologies will only hinder [GOS development].

It's not a fear of evolving technologies, not at all; it's a fear of a particular technology that has/is going off the rails, a technology that apparently not even its developers can explain in extreme cases. It's a very different animal than IF, ELSE statements. I think it's a super tech for certain thing, such as programming and bug hunting, but it's being used for much more than that, including purposes that erode our freedoms and as a privacy advocate yourself, that ought to worry you.

Tandara ... we have an influx of people installing GOS and relying on either third-party cloud LLM services (selling all your info and training on your data) or installing some sketchy, vibe-coded solutions from GitHub.

Agreed - I find it hard to understand why people would install GOS and then trash their privacy by installing garbage.

de0u

vtek Admittedly I'm a partial victim of that because I simply don't understand ML/AI well enough, not by a long shot, and because I don't understand it, I'm weary of it in a general sense, though I'm also well aware that not all AI can be dumped into the same basket.

I suspect many people, maybe especially including many GrapheneOS users, think similarly. But I don't think it's a reliable way to move forward.

Android has mountains and mountains of C++ code inside it. I suspect a very small fraction of Android users know any C++, let alone understand the subtleties. But C++ is actually pretty dangerous! At least compared to Rust. But major news sources aren't publishing articles every day about how C++ code inside people's phones allowed malware in. There are articles about that, but they're in much more esoteric venues.

I'm pretty sure that for the vast majority of GrapheneOS users C++ is a much bigger danger than "AI", especially any kind of "AI" that the GrapheneOS developers would decide to ship.

An offline-trained text-to-speech model is not going to get loose and roam around a phone uploading files. Nor would an image-similarity model, or even a photo-tagging module. In my opinion, those would all be safer to run on a GrapheneOS phone than the Bluetooth stack that most of us turn on multiple times per day without thinking about it. And inn my opinion it will be a great day for security when the Android Linux stack is replaced by code written in Rust.

de0u

vtek Regarding your comment that I quoted, let's consider the recent Claude development that wasn't widely released because it was considered to be too dangerous. I think this justifies my fears, not about this gallery app, but because a company coded a product that apparently exceeded their expiations in a remarkably dangerous way, though I do not know the whole story.

I don't know the whole story either... but a web search for mythos nothingburger will turn up some opinions that aren't being blared out by news organs that are profiting from alarmism.

vtek t's not a fear of evolving technologies, not at all; it's a fear of a particular technology that has/is going off the rails, a technology that apparently not even its developers can explain in extreme cases. It's a very different animal than IF, ELSE statements. I think it's a super tech for certain thing, such as programming and bug hunting, but it's being used for much more than that, including purposes that erode our freedoms and as a privacy advocate yourself, that ought to worry you.

It really isn't one "technology". The "AI" in an automatic license-plate scanner is as different from the "AI" that deleted somebody's production database as a shrew is different from a coyote. Both are mammals, but they're genuinely not in the same threat class.

Sensation0904

vtek I'm speaking in general of course, not necessarily regarding the subject at hand.

Considering the whole purpose of this thread is discussing "AI" in the gallery app I'm not sure why you're bringing up general AI issues, that's a discussion for another thread since your concerns certainly don't apply here.
As someone else explained, you are worried about agentic AI which has capabilities to actually do things. The models that are used for this either cost money or require hardware costing thousands of dollars to run by yourself. You can rest assured the local "AI" in this gallery app is not capable of those feats.

Also in regards to auditing the code, have you audited the rest of the GOS source code? If not, then I don't understand why you're drawing the line here. All of us that aren't auditing the code in its entirety are trusting the GOS team and community at large already, so there's no reason to treat this any differently.

vtek Regarding your comment that I quoted, let's consider the recent Claude development that wasn't widely released because it was considered to be too dangerous

This is just a marketing strategy all of these AI companies use from time to time.
Here is an article from 2019 about how GPT-2 was "too dangerous to release".

Tandara

vtek I get your point. The main problem here is that, like de0u said, such capabilities of LLMs (ignoring that tool-calling is part of a program and not llm) are beyond the scope of any model that would be able to run on mobile in the nearest decade. Meanwhile, the whole question about trust is pointless -- if GOS devs ship it as default app that would mean it was audited by them. Not trusting it means not trusting the project and anything it ships as a whole.

vtek

Johnnyloans swearing off all AI because of a few is like refusing to use a measuring tape because you read in the news that various tools could be used to hurt people.

Might've been funnier if you used a hammer as an example, but you're right. You referenced me though and I've already admitted that I'm not anti-AI, not at all, but anything I would say in that regard I've already said.

Johnnyloans Don't worry, the most powerful AIs won't be on our phones.

For sure, but that's a discussion for another day perhaps.

vtek

Tandara Yeah, agreed. I had it on my mind to mention that earlier and forgot about it. Thanks for pointing that out.

vtek

Sensation0904 Considering the whole purpose of this thread is discussing "AI" in the gallery app I'm not sure why you're bringing up general AI issues, ...

The OP didn't make their concerns clear, just that they didn't want AI in the app to which i read into the likelihood that their reason was because of not being able to trust what the code is doing.

Sensation0904 You can rest assured the local "AI" in this gallery app is not capable of those feats.

Most probably accurate I would presume ... 'probably' being the key word, right?

Sensation0904 ... have you audited the rest of the GOS source code? If not, then I don't understand why you're drawing the line here.

Of course not, and I'm not drawing any line. It's a concern, and a small one at that regarding the gallery app, but it a massively larger concern in general. That said, if you or someone you trust hasn't audited the AI parts of the gallery code, then you cannot state as fact that it's beigne. Of course I'm being hyper critical here, but also technically correct. Some companies--Palantir for one--that are developing AI are using the tech for reasons I need not explain.

I apologize if I've derailed the topic. My intention was to expand on what I thought the OP's intention was.

raccoondad

vtek Is there some guarantee that AI cannot act maliciously?

AI can't act maliciously because AI doesn't act with intent, its a probability model. If you don't want a model with access to syscalls to do something you just remove the syscalls. Your idea on how AI functions doesn't really hold up with what AI is, which is a very large function w/ in -> out and that's it. It does nothing more. It can't hurt you or act maliciously any more than any other function in mathematics.

The issue is when you take that output and use it for input in a program function with access to syscalls and have the programs behavior be unchecked.

Autonomous helicopters being ran by chatGPT aren't being crashed by ChatGPT, they are being crashed by whoever thought putting chatGPT in the control room of a plain was a good idea. The person who translated the output of the LLM and piped it to the control board of a helicopter is the person crashing the plane.

In other words, the answer to your question is you don't pipe output from a large undefined linear function based on the added averages of weights needed to get to a specific 'correct answer' into things that it can easily destroy otherwise.

As far as I understand, this model sorts photos into albums based on theme. I don't really want or need that, but it doesn't have the capacity to delete files unless it was for whatever reason given the needed syscalls to do such.

« Previous Page Next Page »