Sort:  

Screenshot_20230313-102658.png
Either select the three vertical dots to get a menu...

Screenshot_20230313-102902.png
...or swipe sideways to get the remove button. Either way it is a two step process. 😁

I could be wrong but I think the swipe method can be done from either (left) or (right) side.

Of course you can also clear out your entire watch history...

Screenshot_20230313-103554.png
...by selecting these three vertical dots.