I think our almost four-year-old mostly understands how minutes work, but he's not so good at understanding bigger units of time, like say, "one hour." And when he refers to things that happened in the past, he usually says, "yesterday"... no matter if it was indeed yesterday or actually a few days ago.
He understands what weekends are though, and loves that he gets to spend them with his family instead of having to go to school. He's getting better at the days of the week too. I need to spend more time with him on how time works... it's just a bit of an abstract thing to talk about!
Does your LO understand how time works? How long did it take them to understand it all?