No longer set low_latency flag as it causes this warning backtrace:

  WARNING: at kernel/mutex.c:207 __mutex_lock_slowpath+0x6c/0x288()

Fix associated locking and wakeups.
Signed-off-by: Jon Povey <jon.povey@racelogic.co.uk>
Cc: Maulik Mankad <x0082077@ti.com>
Cc: stable <stable@kernel.org>
Acked-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>