How long do you think it takes for caffeine to leave your system? To the point where it won't transfer to your LO through breast milk?

I would love to have a cup of coffee in the am. But the 3 times I've had since delivery my LO was extremely fussy that evening. The last time I just had two sips! Could have been a coincidence.

Now that I'm back at work I can control when I drink it to when my next pumping session is.

So how long do you think it takes caffeine to leave your system?