AI to filter all nudity at OS level?

bbarrowes

Hi, I have a goal of using AI to filter all nudity that might be displayed on my phone, whether in the form of images, or video. I have done some searching, and it seems like there are some offerings out there of AI that are able to filter out nudity from images, and also video. How feasible would it be to make it so that every image and video that the phone tries to show has the AI filtering for nudity applied to it? Does the phone have code that deliberately runs any time a specific image or video is displayed, so that such filtering could be run along side the code that displays images or video, or is that something that can happen outside of the scope of the os code? I know it is not a small ask, but I am asking how feasible. Some offerings also let you customize the filter logic, even allowing the user to have drawings on a dummy represent what the AI will filter. If Graphene OS were to not only be stripped of idiot Google, but also a safe haven as far as encountering unwanted nudity is concerned, I could see that being a huge draw to the operating system. I know I would love it.

Camera6138

bbarrowes

not an OS dev, but a programmer. this type of thing is probably possible to implement somewhat poorly right now, with a lot of effort. check back in a couple years. but I guess I would ask, where are you encountering unwanted nudity? there are probably some good-enough solutions that exist right now, but it depends on if you're:

seeing ads with nudity
intentionally browsing to sites with nudity, and want protection from your own impulses
in a discord server where people send it sometimes inGeneral
idk, something else? I don't see any unwanted nudity, so unsure what is possible.

Max-Zorin

I highly doubt if there are any initiatives for the GOS Developers to implement something like this.

Best bet is to use something like NextDNS as your DNS provider and use custom porn filtering or family safe search filters with that service. You can also create custom blocklists to block specific sites.

burn3r

Is CleanBrowsing family filter considered a safe DNS? I can't find much on it, but it does seem to work when configured on the private DNS option

[deleted]

burn3r You can read the CleanBrowsing Privacy Policy here

raccoondad

Here's my advice,

If you are blocking for a child you are giving the device to: just don't. Any advice I can give will either be ineffective or incredibly harmful to your child's privacy.
Any effective measure in stopping ALL pornography can have serious real world effects. You don't want a middle man going thought every website your child visits and sending it to a server.
Get your child a dumbphone or wait until you think they are ready and have an open and unjudgemental environment for them.

If its for you: block porn sites client wise, there's a large list of domains on github of adult websites you can block

newbie24689

Interesting question. It could be to protect a child. It could also be to protect oneself from "quietly" downloading a child-porn image that would be subsequently found during an inspection of the phone by police/border authority/ etc.
As mentioned earlier, DNS selectivity is good.
Also look for a VPN that blocks porn.; e.g. Poton VPN advertises protection against malware, trackers, and ads, but sadly not incriminating images. Maybe some other VPN does.

Perhaps there's an anti-malware that scans downloads for known porn. e.g. Bitdefender scans my downloads, sms, etc. for a broad range of bad stuff; unfortunately not porn.

Law enforcement will use AI to scan your box; ISTM an app that'll apply AI to look for unknowingly-downloaded kiddie-porn as well as bad-ware is a good idea and will soon emerge. 'Til then, and if for a child, raccoondad has the right idea.

Hb1hf

raccoondad harmful to your child's privacy

Wait, what? I hope you mean that kids need privacy from Google & Co, not from parents...

raccoondad You don't want a middle man going thought every website your child visits and sending it to a server.

Nobody said anything about sending stuff to a server. It could be done locally, theoretically.

Best thing that can be done with a kid's phone is to give them a phone where only the parents have the password to the owner profile. There, they set a private DNS (which lost the ability to be changed in secondary profiles 1 or 2 GOS releases ago) to block adult content (and possibly softcore porn sites such as Instagram) and forbid the kid's profile from installing apps. That should cover a lot.
Kids will need a messaging app, though, and that's one entrance door I'm not sure how to close.

raccoondad

newbie24689 "Perhaps there's an anti-malware that scans downloads for known porn"

Again a very bad idea. Something like that could only exist as a vulnerability waiting to happen....I really don't see why you would even need to cyberstalk a child to this capacity.

You should really be able to have open discussions with your child, especially about stuff in this nature, before you give them a cellphone. The dangers of pornography on children are both worsen and pale in comparison to the inability for a child to communicate openly with their parents.

Theres no tech or software solution to parenting, unfortunately.

At the end of the day you have to understand theres always going to be some danger, but it only will be worse if your child doesn't have the tools of supportive and non-judgemental parents.

No amount of software is going to stop your kids from finding porn from their friends, or stop a family member from harming them, or the other millions of ways a child can be harmed. But having loyal and supportive parents can, at the very least, minimize the harm. No amount of software can replace that.

Well, this moved from GrapheneOS to tech to ethics...we also kind of assumed the use case of the OP...Regardless I think on a techincal standpoint it wouldn't be possible. You would have to do something to the image rendering functions of the device and have it send the data to an AI API and have that API return a yes or no and block from there.

I only dabbled in OS development and never graphic rendering, but I have a feeling even if you somehow got that to work that it would be a very bad idea. Every photo you take gets sent to some API...just feels like you might as well plug in your phone to the NSA charging station.

Main takeaway, yes technically the operating system handles image rendering, but im not even sure if those functions could even make network related syscalls, and the risk of it outweighs the benefit.

raccoondad

Hb1hf "Wait, what? I hope you mean that kids need privacy from Google & Co, not from parents..."

I have my own opinions on ethics in child safety, this isn't the place for that discussion. I'm referring to how majority of 'parental control' applications are both vulnerable code wise and often have data leaks. You don't want your childs personal information being sold in data leak forums

"It could be dine locally, theoretically"

Show me a man who makes and trains an AI detection tool to work with minimal resources (such as a pixel phone) and I will show you a man who should of just not given a cellphone to their child.

For young children, that's my best advice. For older teens, an open discussion about things you should have already told them about will help with the problems that a SMS application can have. That's all I can say personally. Its a tricky situation but why have the situation at all? Give your kid a dumbphone until they are ready for a smartphone.

bbarrowes

Thanks everyone for your input. I am just wondering about technical possibility, and if there is code that runs any time a file of a particular format is rendered, and not how I might use it. Sometimes in programming, if the end goal is known, sometimes the end goal can be achieved in more ways than someone may have originally thought, so it is good to ask. However, in this case, the end goal is to just get rid of unwanted nudity--for me, and anyone else who might want to also filter out unwanted nudity. Unwanted nudity is everywhere. I encounter it in ads, YouTube videos, album/playlist artist on music platforms, and yeah in Discord messages, too. I just want to attack it at the source. I have seen that it is possible to have AI detect nudity in both images and videos. I am just wondering if the operating system is structured so that when a .jpeg file--for example--is presented to the system to render, if there is specific code that runs, where I could piggy-back the AI filtering logic. I am not interested in a conversation about if this should be done or not. Just if it can be done. Thanks everyone, again, for your input.

matchboxbananasynergy

At this point, this is not something that GrapheneOS could/would focus on, and it's unclear if it would be even technically possible to implement. If it was, it would be a gigantic undertaking.

I think there's been some discussion about the topic and it is unlikely that given the subject matter (meaning the fact that it's proposing something that's almost certainly impossible at this point in time) as well as the fact that it is also presented as a potential feature request which GrapheneOS cannot undertake, I'll be locking this thread, since there isn't really much more productive and on-topic discussion to be had, and the thread has already veered off in different and unrelated directions.

Thank you all for your understanding!