How long do you have to attend college to be a doctor?

It depends on the field of study. You've tagged, "physics," so I'll assume you mean how long does it take to get a PhD in physics. Typically you need to obtain a bachelors (4 years), then go to graduate school, (typically 5-7 years max). Provided you're accepted into these institutions, pass written and oral qualifying exams, and present original peer-reviewed research (your dissertation thesis), you'll have a PhD in about 9-11 years, from start to finish. Then, you go into industry, or academia as a professor. If you take the latter route, be prepared to do anywhere between 2-6 years of post doctoral research to be considered for an academic position. Then another 6 years maximum to be considered for tenure.