Back in the day, I earned the nickname “Link Nazi” from some fellow designers. I often raised the issue in design critiques when people were inconsistent with their link and button colors.
As a general rule, I think it’s good to have an action color reserved for interactive elements. Don’t use that color for things you don’t click/tap on.
Your action color helps with affordance, or the suggestion of what the user can do with elements in your design. That said, color is not the only way to convey that something is interactive, and there may be cases where you want to be mysterious and exploratory with your users. But definitely think about affordance and be intentional about it.
So maybe I’m still a Link Nazi. But I promise not to slowly and dramatically remove my glove so I can slap you with it, even if sometimes I want to.