Yeah that is a very interesting observation you've made! It is hard to tell a colour from RGB, HSB was made to be more artist/human-friendly for that reason in fact
In an attempt to accommodate more traditional and intuitive color mixing models, computer graphics pioneers at PARC and NYIT developed the HSV model in the mid-1970s, formally described by Alvy Ray Smith in the August 1978 issue of Computer Graphics. [...] Their model was based more upon how colors are organized and conceptualized in human vision in terms of other color-making attributes, such as hue, lightness, and chroma; as well as upon traditional color mixing methods—e.g., in painting—that involve mixing brightly colored pigments with black or white to achieve lighter, darker, or less colorful colors.
from HSL and HSV on Wikipedia