Some digging has revealed that this password prompt is shown when the softwareupdate binary is called, and only on Apple Silicon devices.
Calling the softwareupdate binary is something which we Mac Admins have done for years, and it’s likely folks have workflows in place calling the softwareupdate binary which is then triggering this prompt.
So, what’s the fix? Short term, don’t call the softwareupdate binary on Apple Silicon devices and raise this issue with Apple.
Longer term? Unsure. There is still documentation around Apple Silicon devices to be posted by Apple, and until we have documentation we can but guess.