So I don't know how Charisma/other thing/aggression affects the game, may only be what you can say to people etc. Either way, I have found that by simply talking to one NPC (Not sure if it works on just any NPC but as far as I could tell, it doesn't) and spamming the charisma option can max it out in no time. The NPC is named Mistress Broga in Almuten Estates and when I spoke to her, she had the option for charisma, I did it again by accident and saw the animation for increased charisma so I checked an voila, it went up... Spammed it and was at max with no aggression and no crowny stat at all!
She's in the little house I circled and she's on the second floor.
And there you have it, max that charisma for whatever good it might do!
Have fun y'all!