Hello! That's such a great question. Sometimes when we are fresh out of high school and we are excited about what we want to become, we forget the challenges that might come with our chosen field, and I commend you for trying to remain objective.
In my career as a doctor what I have found to be most distressing is the long hours of work and also having to admit to patients that my medical knowledge can only go so far. In cases where you find a patient has cancer and it has spread to the rest of the body, and their treatment becomes more supportive than really trying to fix the primary problem, that to me has to be the most distressing.
For the most part I think the stresses are not physical but more emotional or psychological, so losing patients, having to deliver bad news or a bad prognosis.
Once you start practicing medicine you will realise that you also become your patients' counsellor, beyond giving treatment, sometimes what patients need is somebody to explain to them why their condition is not improving. Once you explain as kindly as possible, most patients with terminal or long term illnesses, actually appreciate the honesty, even though as a doctor you will feel bad afterwards because you wish you had given them better news.
At the end of the day, it's important to remember that challenges cannot be completely erased, and you must still pursue your passion and become a great doctor, because saving lives is so worth it.