It is a wise question you raise. I believe there are at least two answers: answer one is related to how advance your career and answer two is how to find greater happiness in your career and life?
The good news is I believe you can have and to some degree you need both. Although it's possible to compartmentalize your life into work and play, generally you'll find you don't have time to indulge in one or the other independently the way you imagine. Therefore, if one of your personal goals is to travel, then getting a job that allows for and even enables traveling would be a win win personally and in your career. If you're interested in community service securing a job that has a 'greater good' policy or initiative would allow you to do good while you work or at least in association with work.
That said, it's important to understand companies pay you to accomplish some task for which they receive revenue. If you work for a plumbing firm and you install a toilet and a water heater, the relationship between the task you performed and company revenue is pretty clear. If you're a programmer and your performing bug fixes, it's a little less direct, but the thinking is better software product quality translates to improved customer satisfaction and ultimately greater sales.
Assuming the tasks you know how to perform are in demand, the more complex and specialized your competencies are, the more your company can charge for your skills and the more you will get compensated. Much of the time, that complexity and specialization is related to domain knowledge not better programming skills. For example, knowing about the retail or the banking business will allow you to communicate more effectively with your retail or banking users and allow you to create more valuable products. Therefore, if you get excited about the stock market or online stores, then developing expertise in that kind of business will make you a more valuable programmer and, presumably, increase your engagement and satisfaction at work.
Education doesn't end with college, so rather than try and cram as much training as you can in college, it's probably more important to have a long term strategy that marries your technical skills to a specific business, government or non-profit domain about which you care. In the beginning, you'll probably be pushed enough just learning to program the way your company wants you to program. However, in time, that won't seem so challenging and, assuming it would be fun for you to learn about a domain that interests you, then that blending should yield a better balance in your overall skill set.
In short, understanding yourself enough to know the things that matter most to you and which, in doing, bring you pleasure lays the foundation for integrating your CompSci skills with that lifestyle. If approached in the right way, instead of competing for your personal time, your CompSci skills will open doors to doing things you want to do and care about (e.g. travel, social causes, learning about life). So if you want to minor in a field, I would recommend that decision is based more on your curiosities or interests so you can begin to develop domain expertise to compliment your technical expertise.