Can you spot the problem in this example? Here’s a hint: adding a little after the verb in no way makes smiled a way in which words can be spoken. On the other hand, smiling is perfectly valid as an associated action; people often smile while speaking, but the smiling and the speaking are two separate things.
The easiest/simplest solution here is to change the comma after little to a period. You could also choose to leave the comma but add and said after little.
She smiled a little. “You’re a manipulator.”
She smiled a little and said, “You’re a manipulator.”