Christian, this is a great question. The difficulty of an engineering education is based strongly on aptitude and attitude. If you are comfortable with higher level math classes and physics, you have the aptitude to do well in engineering school. Engineering program is more difficult and challenging that a lot of other majors. The classwork leaves you less free time than some other degrees. This means that you will need to be focused on the end goal of graduating with an engineering degree.
As Simon said, engineering can be a very rewarding career path in doing things that are exciting, challenging, and provide you will a good income.