I have had thoughts very similar to this! My theory is that he was jealous of her being the heir to the throne of the Twilight Realm so he started a relationship with her not revealing his true intentions, then Midna found out what he was up to and cut off the relationship and that's when Ganon...