Reference siphash takes the keys as a bytes, so it makes sense to byte swap
when reifying the keys as 64-bit integers. However, Python's siphash takes host
integers in to start with.