I start by organizing my games by what system its for and its numerical number, then alphabetize them unless they're part of an ongoing series (think .Hack// -- it even has a picture that forms on the spines as you complete the series). So it's 3DS games on the topmost shelves, followed by DS games, then Wii, then Wii U; next shelf is Playstation 4 and so on and so forth.
When it's time for a console upgrade and I need to put away older games that are not backwards compatible, I'm far looser with my organization. As long as they're in the properly marked box, I don't mind what order they go in.