It was mostly for protection, whether it was from sun, cold, or living situations (loincloths are good to prevent from sitting on hot rocks). With my fair skin, I don't go topless (even in situations where I could) because I am guarenteed a sunburn, so I can sympathize with this.

As far as more elaborate clothing goes, social status is a lot of it. In many societies, only certain classes were allowed to wear specific colors (fancy dyes, such as purples and reds were often limited to just nobility or catholic church leaders by law or custom) or fabrics.

And yes, sadly, as time went on, religion began to dictate that the body was shameful and needed to be covered up.