The x32 system call ABI

all current hardware has significant benefits if the data is aligned properly (32 bit values aligned on a multiple of 4 bytes, 64 bit values aligned on a multiple of 8 bytes). As a result, in the normal case, 48 bit unaligned values are going to be slower to use.

If you have a huge array of pointers, the memory savings will outweigh this cost, but not for the normal uses.