You often don't take large and heavy cameras with you. Small and compact cameras usually have far fewer functions. So you have to choose the right ratio. A camera should be so compact that it doesn't bother you. What that means exactly depends on you.
In principle, you can assume that less compact cameras have more functions. In some cases, however, it makes sense not to buy a compact camera. Professional photographers often prefer to use slightly larger cameras, as too small displays and setting wheels and buttons can be annoying in the long run. So if you have large hands, for example, a larger model might be just right for you. You can choose your personal preference here.