So, apparently a recent update gave us "modes" (instead of just do not disturb), which has sleeping mode and can set automatic grayscale in the evening hours, it's been one thing that I really missed from stock Pixel.
Weirdly enough it broke system dark mode detection for Dark Reader extension for Firefox, it seems convinced that I'm in light mode when dark mode is set by the Sleeping mode schedule.