Hi,
I'm considering installing GoS on a pixel 10 pro phone but before trying I would like to know first what on-device AI-based feature would work with GoS:
- The AI-generative based 100x zoom on gcam
- The AI-based Add Me feature on gcam
- The offline (on-device) AI-features of Google Photos gallery for editing photos like Magic eraser
Also if possible I would like to know which other on-device AI-based features work on GrapheneOS assuming that I install all those Google applications ?
Below is a list of the AI-based features on Pixel 10 Pro and other Pixels that work on-device (don't require using cloud servers).
📸 Google Camera (gcam)
- Camera Coach: Analyzes your frame in real-time to suggest the best angles, lighting, and camera modes before you snap the photo.
- Add Me: Allows the photographer to be merged into a group shot by taking two photos and compositing them locally.
- Auto Best Take: Automatically identifies the best facial expressions from a burst of group photos and combines them into one perfect shot.
- Pro Res Zoom (100x): Uses generative AI on-device to fill in fine details and textures when using extreme digital zoom.
- Real Tone Mapping: Locally processes image data to ensure all skin tones are represented accurately and vibrantly.
🖼️ Google Photos
- Audio Magic Eraser: Locally identifies and isolates sound layers (wind, traffic, crowds) in video files so you can reduce or remove them.
- Photo Unblur: Uses the Tensor G5's NPU to sharpen and reconstruct clarity in old or blurry photos.
- Magic Eraser: The standard tool for removing photobombers and unwanted objects from the background of any photo.
- Basic Magic Editor: Handles object moving, resizing, and simple background "generative fill" without needing to upload to the cloud.
📞 Google Phone
- Call Notes & Summaries: Transcribes phone calls in real-time and provides a bulleted summary of the conversation immediately after you hang up.
- Scam Detection: Monitors the audio of incoming calls for known fraudulent speech patterns and provides a haptic/visual alert if a scam is detected.
- Voice Translate: A real-time, two-way translation feature for calls that recreates your own voice and intonation in the target language.
- Magic Cue: Proactively displays relevant information (like an upcoming appointment or a flight number) on the call screen based on the context of the conversation.
💬 Google Messages
- Magic Compose: Provides context-aware reply suggestions that match the tone of your conversation, processed entirely on-device for privacy.
- On-Device Smart Reply: Quick, one-tap responses based on the last few messages in a thread.
🎙️ Google Recorder
- Recorder Summaries: Generates high-quality summaries of long meetings, lectures, or interviews without an internet connection.
- Speaker Labels: Automatically detects and separates different voices in a transcript locally.
🖼️ Pixel Screenshots
- On-Device Search: Uses AI to index the visual and text content of every screenshot you take, allowing you to search for them using natural language (e.g., "the receipt from the coffee shop").
- Smart Actions: Suggests actions based on the screenshot, such as "Set a reminder" or "Copy tracking number."
♊ Gemini (System-Wide)
- Gemini Nano (Multimodal): The core model that allows the assistant to "read" your screen and perform tasks across apps (like finding a date in an email and adding it to your calendar).
- Circle to Search: Locally isolates objects or text on your screen for instant identification and analysis.
- Live Translate (System-Wide): Provides instant captions and translations for any audio or video playing through the device's speakers.