Short version - Does anyone do this? Is this crazy?

Long version - Right now both kids - DD (4) and DS (turning 2), are at the same daycare, in our town. It's 5 minutes away which is awesome because both DH and I can do drop offs / pick ups with no issues. The problem is DS will soon move up to a new room, and I do NOT like the teacher he would have.

It's a center, so there is one room per age group. This teacher was his infant teacher, and after going through it I wish I had switched daycares for him way back then, because she was just so not what I wanted for him. While she never did anything actually WRONG, she was just so not loving / caring to him, and still to this day tells me in front of him how he is annoying / dramatic / hard to handle. (He's not...he's just a toddler!) She was his teacher since he was a TEENY TINY baby at 8 weeks old and the fact that she dislikes him so much and that he was with her for so long hurts my heart to think about, and thinking about him being with her again makes me crazy. I have a feeling she might be better with this older age group, but I'm torn on if I want to even give her the chance. I can definitely talk to the director and see if there's any way he could just stay in his current room, but I have a feeling at some point he'd have to move up / he should move up to be with kids his age.

What would you do? Find somewhere new for just him? Find somewhere new for both kids? DD misses the Kindergarten cut off so she has another full year of daycare left and won't start until 2018. I like the preschool teacher at our current daycare so possibly could just move him for one year but that seems crazy too!