Without this patch we won't clear that bit and instead will
clear all other bits on our endpoint flag.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Felipe Balbi <balbi@ti.com>