It's hard to beat Bee!
Bee - 3 letters
Four - My name is 4 letters
Olive - 5 letters
Charlie - 7 letters
Oh man, Charlie breaks the pattern! Maybe we should have a third child just so we can give them a name with six letters.
Who has the name with the least letters in your family?