Commit b15ff947 authored by Nicolai Stange's avatar Nicolai Stange Committed by Ben Hutchings

lib/mpi: mpi_read_raw_data(): fix nbits calculation

commit eef0df6a upstream.

The number of bits, nbits, is calculated in mpi_read_raw_data() as follows:

  nbits = nbytes * 8;

Afterwards, the number of leading zero bits of the first byte get
subtracted:

  nbits -= count_leading_zeros(buffer[0]);

However, count_leading_zeros() takes an unsigned long and thus,
the u8 gets promoted to an unsigned long.

Thus, the above doesn't subtract the number of leading zeros in the most
significant nonzero input byte from nbits, but the number of leading
zeros of the most significant nonzero input byte promoted to unsigned long,
i.e. BITS_PER_LONG - 8 too many.

Fix this by subtracting

  count_leading_zeros(...) - (BITS_PER_LONG - 8)

from nbits only.

Fixes: e1045992 ("MPILIB: Provide a function to read raw data into an
                     MPI")
Signed-off-by: default avatarNicolai Stange <nicstange@gmail.com>
Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
[bwh: Backported to 3.16: adjust context]
Signed-off-by: default avatarBen Hutchings <ben@decadent.org.uk>
parent fa1703ff
......@@ -48,7 +48,7 @@ MPI mpi_read_raw_data(const void *xbuffer, size_t nbytes)
return NULL;
}
if (nbytes > 0)
nbits -= count_leading_zeros(buffer[0]);
nbits -= count_leading_zeros(buffer[0]) - (BITS_PER_LONG - 8);
else
nbits = 0;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment