The feature is indeed closed source and is implemented by /system_ext/app/RepairMode in the stock OS. It's a really simple app and I don't think it'd be too difficult to reimplement if someone wanted to. It creates a DSU (dynamic system update) slot with a new userdata image. This allows repair mode to use all of the stock OS partitions, except for the user data. When booting into the new DSU slot (which they named repairmode.lock), the same RepairMode app will adjust some system settings, like disabling the initial setup wizard, the factory reset option, and the OEM unlocking option. When the user exits repair mode, the DSU slot is deleted.