First of all, there didn't used to be any specialized programs and now there are quite a few. I would research them carefully, as you will probably want to move near the one you think is the best for you and not try to find courses at the school you might have gone to otherwise. It's the kind of thing that if I was already in school and realized that I want to make games and found a school I thought could help me, it would be worth transferring and moving there.
To be more specific in your search, would you be more interested in programming the game or creating art or 3d models to go into the game. Once you know which one you prefer and would more likely do better at, then that will help your search. Asking to be the designer is like asking to be conductor of the orchestra without knowing how to play a single instrument. You might get a job at a game company as an entry-level artist or programmer, but unless you have a portfolio of previous game credits I doubt they would give a new hire a chance to be the sole designer of a major game project.