This is a great question and when I was choosing a college to attend I had similar thoughts.
Typically private colleges tend to be smaller so class sizes are more intimate versus public colleges that tend to have larger class sizes. Also private colleges, sometimes, have less options in terms of academic programs. For me, that was the deciding factor as I chose a public school over a private school because the degree I wanted had more class offerings at the public school. Now, this is not always the case as each private and public school are different.
In the end, my best advice would be not to focus on whether the school is private or public. Instead, I would focus on what each specific school offered me. Be that in terms of scholarships, sports, academic programs, class sizes, commute to campus, etc.