ETA: Some of the answers provided are simply guessing and not providing information based on facts.
Where you are building is going to dictate what is required.
However if you were 100% funding your own build, there are many places across the US that do NOT require any type of license, inspection, drawing, etc., and yes, it is as concerning as it sounds.
My husband and I, both with a construction background, built our home 100% in a rural area, which required zero oversite. No inspectors. Fortunately, we had both worked in the industry for 40+ years combined and we were very confident in the final product, having only recently sold it and the buyer paying for inspections, and the home passing with flying colors.
The go-to answer with so many construction questions is always "it depends" as there are a million factors to take into consideration before being able to provide a solid answer.
Depends on where you live. Most municipalities will only grant building permits for new home construction to licensed general contractors.
There are lots of things to consider in building a home: storm water drainage, roof, mold, ability to withstand winds and fire, etc. It takes a lot of knowledge. Even if you thought you could build a good home, say, based on internet research, I can almost guarantee you that the electric company will refuse to hook up power to your house unless you had certain levels of approval from code compliance, etc. The same for water and sewer. As appealing as it is to think about totally escaping government regulation and all the bureaucracy, it's not easy to do, and, as much as I hate to admit it, those policies serve a purpose.
(This does not address those code compliance personnel who take money for the permits but never really do the inspections!)
There are some things that you can do on your own, but certain aspects require a license.
Cindi Corr’s Answer
I agree with the earlier comment. You can google/contact your town (or city or county) and building permits (for example 'Philadelphia building permits') to get more information on the specific requirements.
Deborah Bortolin, NCARB, AIA, RIBA, ARB
You need a licensed professional (i.e. architect, engineer...) to submit the drawings for permit to the local City Council. The project will be reviewed and approved if it is respecting local code requirements.
After approval, you need a licensed contractor to proceed with the construction.
If you decide to proceed with a prefab home, the company supplying the house will run the signed and seal submittal for you to the local authorities.