The i.MX51 datasheet says:
Chapter 56.1.2.4 I2S Mode
...
When I2S modes are entered (I2S master (01) or I2S slave (10)),
the following settings are recommended:
...
- TX Frame Rate should be 2 i.e. (STCCR[12:8] = 1)
- RX Frame Rate should be 2 i.e. (SRCCR[12:8] = 1)

Chapter 56.3.3.12 SSI Transmit and Receive Clock Control Registers (STCCR & SRCCR)
...
Bits 12-8 DC4-DC0
Frame Rate Divider Control. These bits are used to control the divide ratio
for the programmable frame rate dividers. The divide ratio works on the word
clock. In Normal mode, this ratio determines the word transfer rate.
In Network mode, this ratio sets the number of words per frame. The divide
ratio ranges from 1 to 32 in Normal mode and from 2 to 32 in Network mode.
In Normal mode, a divide ratio of 1 (DC=00000) provides continuous periodic
data word transfer. A bit-length frame sync must be used in this case.

Function fsl_ssi_hw_params() setup Normal mode for MONO output,
so with DC=0, SSI enters to continuous periodic data word transfer.
To fix this, setup DC for any I2S mode.
Patch has tested on custom board based on Digi CCMX-51 module (i.MX51).
Signed-off-by: Alexander Shiyan <shc_work@mail.ru>
Signed-off-by: Mark Brown <broonie@kernel.org>