PackedInteger encoding does not preserve order - but it could

Question

Currently, when viewed under the default unsigned lexical ordering, "packed" integers (and longs) do not sort numerically. This is because for values requiring two or more bytes, the "value" bytes are stored little endian.
For example:
<tt> 65910 -> 7a ff 00 01</tt>
<tt> 65911 -> 7a 00 01 01</tt>
However, it's possible to encode packed integers such that they still sort correctly. You can do this by modifying the existing PackedInteger algorithm as follows:
<li>Invert the high-order bit on the first byte
<li>Reverse the order of the second and subsequent bytes (if any)
<li>If the original number was negative, invert (one's complement) the bits in the second and subsequent bytes (if any)
Here's some examples showing the two encodings:
              NUMBER             ENCODING         NEW ENCODING
-9223372036854775808   8189ffffffffffff7f   018000000000000076
         -2147483648           8589ffff7f           0580000076
              -65911             86000101             06fefeff
              -65910             86ff0001             06feff00
              -65655             86000001             06feffff
              -65654               87ffff               070000
              -65400               8701ff               0700fe
              -65399               8700ff               0700ff
                -375               870001               07feff
                -374                 88ff                 0800
                -120                 8801                 08fe
                -119                   89                   09
                  -1                   ff                   7f
                   0                   00                   80
                   1                   01                   81
                 119                   77                   f7
                 120                 7801                 f801
                 374                 78ff                 f8ff
                 375               790001               f90100
                 631               790002               f90200
               65399               7900ff               f9ff00
               65400               7901ff               f9ff01
               65654               79ffff               f9ffff
               65655             7a000001             fa010000
               65910             7aff0001             fa0100ff
               65911             7a000101             fa010100
          2147483647           7b88ffff7f           fb7fffff88
9223372036854775807   7f88ffffffffffff7f   ff7fffffffffffff88So the question is.. any particular reason this wasn't done? Would this alternate encoding be a useful addition? (I'm sure it's not possible to replace the current algorithm at this point).
Thanks.

Answer 1

greybird wrote:
Thanks Archie. I appreciate this and I know you have good intentions. Unfortunately, we won't be able to incorporate any code from your svn repository. We can only incorporate submissions if the source code is posted here in our OTN forum.Not a problem... Here is the new class:
package org.dellroad.sidekar.util;
import com.sleepycat.bind.tuple.TupleInput;
import com.sleepycat.bind.tuple.TupleOutput;
import java.util.Arrays;
* An improved version of Berkeley DB's {@link com.sleepycat.util.PackedInteger} class that packs values in such
* a way that their order is preserved when compared using the default binary unsigned lexical ordering.
* @see <a href="http://forums.oracle.com/forums/thread.jspa?messageID=4126254">Berkeley DB Java Edition forum posting</a>
public final class PackedLong {
     * Maximum possible length of an encoded value.
    public static final int MAX_ENCODED_LENGTH = 9;
     * Minimum value for the first byte of a single byte encoded value. Lower values indicate a multiple byte encoded negative value.
    public static final int MIN_SINGLE_BYTE_ENCODED = 0x08;        // values 0x00 ... 0x07 prefix negative values
     * Maximum value for the first byte of a single byte encoded value. Higher values indicate a multiple byte encoded positive value.
    public static final int MAX_SINGLE_BYTE_ENCODED = 0xf7;        // values 0xf8 ... 0xff prefix positive values
     * Adjustment applied to single byte encoded values before encoding.
    public static final int ZERO_ADJUST = 127;                     // single byte value that represents zero
     * Minimum value that can be encoded as a single byte.
    public static final int MIN_SINGLE_BYTE_VALUE = MIN_SINGLE_BYTE_ENCODED - ZERO_ADJUST;          // -119
     * Maximum value that can be encoded as a singel byte.
    public static final int MAX_SINGLE_BYTE_VALUE = MAX_SINGLE_BYTE_ENCODED - ZERO_ADJUST;          // 120
     * Adjustment applied to multiple byte encoded negative values before encoding.
    public static final int NEGATIVE_ADJUST = -MIN_SINGLE_BYTE_VALUE;                               // 119
     * Adjustment applied to multiple byte encoded positive values before encoding.
    public static final int POSITIVE_ADJUST = -(MAX_SINGLE_BYTE_VALUE + 1);                         // -121
    // Cutoff values at which the encoded length changes (this field is package private for testing purposes)
    static final long[] CUTOFF_VALUES = new long[] {
        0xff00000000000000L - NEGATIVE_ADJUST,      // [ 0] requires 8 bytes
        0xffff000000000000L - NEGATIVE_ADJUST,      // [ 1] requires 7 bytes
        0xffffff0000000000L - NEGATIVE_ADJUST,      // [ 2] requires 6 bytes
        0xffffffff00000000L - NEGATIVE_ADJUST,      // [ 3] requires 5 bytes
        0xffffffffff000000L - NEGATIVE_ADJUST,      // [ 4] requires 4 bytes
        0xffffffffffff0000L - NEGATIVE_ADJUST,      // [ 5] requires 3 bytes
        0xffffffffffffff00L - NEGATIVE_ADJUST,      // [ 6] requires 2 bytes
        MIN_SINGLE_BYTE_VALUE,                      // [ 7] requires 1 byte
        MAX_SINGLE_BYTE_VALUE + 1,                  // [ 8] requires 2 bytes
        0x0000000000000100L - POSITIVE_ADJUST,      // [ 9] requires 3 bytes
        0x0000000000010000L - POSITIVE_ADJUST,      // [10] requires 4 bytes
        0x0000000001000000L - POSITIVE_ADJUST,      // [11] requires 5 bytes
        0x0000000100000000L - POSITIVE_ADJUST,      // [12] requires 6 bytes
        0x0000010000000000L - POSITIVE_ADJUST,      // [13] requires 7 bytes
        0x0001000000000000L - POSITIVE_ADJUST,      // [14] requires 8 bytes
        0x0100000000000000L - POSITIVE_ADJUST,      // [15] requires 9 bytes
    private PackedLong() {
     * Write the encoded value to the output.
     * @param output destination for the encoded value
     * @param value value to encode
    public static void write(TupleOutput output, long value) {
        output.makeSpace(MAX_ENCODED_LENGTH);
        int len = encode(value, output.getBufferBytes(), output.getBufferLength());
        output.addSize(len);
     * Read and decode a value from the input.
     * @param input input holding an encoded value
     * @return the decoded value
    public static long read(TupleInput input) {
        long value = decode(input.getBufferBytes(), input.getBufferOffset());
        input.skipFast(getReadLength(input));
        return value;
     * Determine the length (in bytes) of an encoded value without advancing the input.
     * @param input input holding an encoded value
     * @return the length of the encoded value
    public static int getReadLength(TupleInput input) {
        return getReadLength(input.getBufferBytes(), input.getBufferOffset());
     * Determine the length (in bytes) of an encoded value.
     * @param buf buffer containing encoded value
     * @param off starting offset of encoded value
     * @return the length of the encoded value
     * @throws ArrayIndexOutOfBoundsException if {@code off} is not a valid offset in {@code buf}
    public static int getReadLength(byte[] buf, int off) {
        int prefix = buf[off] & 0xff;
        if (prefix < MIN_SINGLE_BYTE_ENCODED)
            return 1 + MIN_SINGLE_BYTE_ENCODED - prefix;
        if (prefix > MAX_SINGLE_BYTE_ENCODED)
            return 1 + prefix - MAX_SINGLE_BYTE_ENCODED;
        return 1;
     * Determine the length (in bytes) of the encoded value.
     * @return the length of the encoded value, a value between one and {@link #MAX_ENCODED_LENGTH}
    public static int getWriteLength(long value) {
        int index = Arrays.binarySearch(CUTOFF_VALUES, value);
        if (index < 0)
            index = ~index - 1;
        return index < 8 ? 8 - index : index - 6;
     * Encode the given value into a new buffer.
     * @param value value to encode
     * @return byte array containing the encoded value
    public static byte[] encode(long value) {
        byte[] buf = new byte[MAX_ENCODED_LENGTH];
        int len = encode(value, buf, 0);
        if (len != MAX_ENCODED_LENGTH) {
            byte[] newbuf = new byte[len];
            System.arraycopy(buf, 0, newbuf, 0, len);
            buf = newbuf;
        return buf;
     * Encode the given value and write the encoded bytes into the given buffer.
     * @param value value to encode
     * @param buf output buffer
     * @param off starting offset into output buffer
     * @return the number of encoded bytes written
     * @throws ArrayIndexOutOfBoundsException if {@code off} is negative or the encoded value exceeds the given buffer
    public static int encode(long value, byte[] buf, int off) {
        int len = 1;
        if (value < MIN_SINGLE_BYTE_VALUE) {
            value += NEGATIVE_ADJUST;
            long mask = 0x00ffffffffffffffL;
            for (int shift = 56; shift != 0; shift -= 8) {
                if ((value | mask) != ~0L)
                    buf[off + len++] = (byte)(value >> shift);
                mask >>= 8;
            buf[off] = (byte)(MIN_SINGLE_BYTE_ENCODED - len);
        } else if (value > MAX_SINGLE_BYTE_VALUE) {
            value += POSITIVE_ADJUST;
            long mask = 0xff00000000000000L;
            for (int shift = 56; shift != 0; shift -= 8) {
                if ((value & mask) != 0L)
                    buf[off + len++] = (byte)(value >> shift);
                mask >>= 8;
            buf[off] = (byte)(MAX_SINGLE_BYTE_ENCODED + len);
        } else {
            buf[off] = (byte)(value + ZERO_ADJUST);
            return 1;
        buf[off + len++] = (byte)value;
        return len;
     * Decode a value from the given buffer.
     * @param buf buffer containing an encoded value
     * @param off starting offset of the encoded value in {@code buf}
     * @return the decoded value
     * @throws ArrayIndexOutOfBoundsException if {@code off} is negative or the encoded value is truncated
     * @see #getReadLength
    public static long decode(byte[] buf, int off) {
        int first = buf[off++] & 0xff;
        if (first < MIN_SINGLE_BYTE_ENCODED) {
            long value = ~0L;
            while (first++ < MIN_SINGLE_BYTE_ENCODED)
                value = (value << 8) | (buf[off++] & 0xff);
            return value - NEGATIVE_ADJUST;
        if (first > MAX_SINGLE_BYTE_ENCODED) {
            long value = 0L;
            while (first-- > MAX_SINGLE_BYTE_ENCODED)
                value = (value << 8) | (buf[off++] & 0xff);
            return value - POSITIVE_ADJUST;
        return (byte)(first - ZERO_ADJUST);
}and here is the unit test:
package org.dellroad.sidekar.util;
import com.sleepycat.bind.tuple.TupleInput;
import com.sleepycat.bind.tuple.TupleOutput;
import com.sleepycat.je.DatabaseEntry;
import org.dellroad.sidekar.TestSupport;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
public class PackedLongTest extends TestSupport {
    @Test(dataProvider = "encodings")
    public void testEncoding(long value, String string) {
        // Test direct encoding
        byte[] expected = ByteArrayEncoder.decode(string);
        byte[] actual = PackedLong.encode(value);
        assertEquals(actual, expected);
        // Test write()
        TupleOutput out = new TupleOutput();
        PackedLong.write(out, value);
        assertEquals(out.toByteArray(), expected);
        // Test getWriteLength()
        assertEquals(actual.length, PackedLong.getWriteLength(value));
        // Test decoding
        long value2 = PackedLong.decode(actual, 0);
        assertEquals(value2, value);
        // Test read()
        TupleInput input = JEUtil.toTupleInput(new DatabaseEntry(actual));
        assertEquals(actual.length, PackedLong.getReadLength(input));
        value2 = PackedLong.read(input);
        assertEquals(value2, value);
        // Test getReadLength()
        assertEquals(actual.length, PackedLong.getReadLength(actual, 0));
    @Test(dataProvider = "lengths")
    public void testEncodedLength(long value, int expected) {
        int actual = PackedLong.getWriteLength(value);
        assertEquals(actual, expected);
        byte[] buf = PackedLong.encode(value);
        assertEquals(buf.length, expected);
    @DataProvider(name = "encodings")
    public Object[][] genEncodings() {
        return new Object[][] {
            // Corner cases
            { 0x8000000000000000L, "008000000000000077" },
            { 0xfeffffffffffff88L, "00feffffffffffffff" },
            { 0xfeffffffffffff89L, "0100000000000000" },
            { 0xfeffffffffffffffL, "0100000000000076" },
            { 0xfffeffffffffff88L, "01feffffffffffff" },
            { 0xfffeffffffffff89L, "02000000000000" },
            { 0xfffeffffffffffffL, "02000000000076" },
            { 0xfffffeffffffff88L, "02feffffffffff" },
            { 0xfffffeffffffff89L, "030000000000" },
            { 0xfffffeffffffffffL, "030000000076" },
            { 0xfffffffeffffff88L, "03feffffffff" },
            { 0xfffffffeffffff89L, "0400000000" },
            { 0xfffffffeffffffffL, "0400000076" },
            { 0xfffffffffeffff88L, "04feffffff" },
            { 0xfffffffffeffff89L, "05000000" },
            { 0xfffffffffeffffffL, "05000076" },
            { 0xfffffffffffeff88L, "05feffff" },
            { 0xfffffffffffeff89L, "060000" },
            { 0xfffffffffffeffffL, "060076" },
            { 0xfffffffffffffe88L, "06feff" },
            { 0xfffffffffffffe89L, "0700" },
            { 0xfffffffffffffeffL, "0776" },
            { 0xffffffffffffff88L, "07ff" },
            { 0xffffffffffffff89L, "08" },
            { 0xffffffffffffffa9L, "28" },
            { 0xffffffffffffffc9L, "48" },
            { 0xffffffffffffffe9L, "68" },
            { 0xffffffffffffffffL, "7e" },
            { 0x0000000000000000L, "7f" },
            { 0x0000000000000001L, "80" },
            { 0x0000000000000071L, "f0" },
            { 0x0000000000000077L, "f6" },
            { 0x0000000000000078L, "f7" },
            { 0x0000000000000079L, "f800" },
            { 0x0000000000000178L, "f8ff" },
            { 0x0000000000000179L, "f90100" },
            { 0x0000000000010078L, "f9ffff" },
            { 0x0000000000010079L, "fa010000" },
            { 0x0000000001000078L, "faffffff" },
            { 0x0000000001000079L, "fb01000000" },
            { 0x0000000100000078L, "fbffffffff" },
            { 0x0000000100000079L, "fc0100000000" },
            { 0x0000010000000078L, "fcffffffffff" },
            { 0x0000010000000079L, "fd010000000000" },
            { 0x0001000000000078L, "fdffffffffffff" },
            { 0x0001000000000079L, "fe01000000000000" },
            { 0x0100000000000078L, "feffffffffffffff" },
            { 0x0100000000000079L, "ff0100000000000000" },
            { 0x7fffffffffffff79L, "ff7fffffffffffff00" },
            { 0x7fffffffffffffffL, "ff7fffffffffffff86" },
            // Other cases
            { 0xffffffff80000000L, "0480000077" },
            { 0xfffffffffffefe89L, "05feff00" },
            { 0xfffffffffffefe8aL, "05feff01" },
            { 0xfffffffffffeff86L, "05fefffd" },
            { 0xfffffffffffeff87L, "05fefffe" },
            { 0xfffffffffffeff88L, "05feffff" },
            { 0xfffffffffffeff89L, "060000" },
            { 0xfffffffffffeff8aL, "060001" },
            { 0xffffffffffff0086L, "0600fd" },
            { 0xffffffffffff0087L, "0600fe" },
            { 0xffffffffffff0088L, "0600ff" },
            { 0xffffffffffff0089L, "060100" },
            { 0xfffffffffffffe86L, "06fefd" },
            { 0xfffffffffffffe87L, "06fefe" },
            { 0xfffffffffffffe89L, "0700" },
            { 0xfffffffffffffe8aL, "0701" },
            { 0xffffffffffffff87L, "07fe" },
            { 0xffffffffffffff88L, "07ff" },
            { 0xffffffffffffff89L, "08" },
            { 0xffffffffffffffffL, "7e" },
            { 0x0000000000000000L, "7f" },
            { 0x0000000000000001L, "80" },
            { 0x0000000000000077L, "f6" },
            { 0x0000000000000078L, "f7" },
            { 0x0000000000000176L, "f8fd" },
            { 0x0000000000000177L, "f8fe" },
            { 0x0000000000000178L, "f8ff" },
            { 0x0000000000000277L, "f901fe" },
            { 0x000000000000ff77L, "f9fefe" },
            { 0x000000000000ff78L, "f9feff" },
            { 0x000000000000ff79L, "f9ff00" },
            { 0x000000000000ff7aL, "f9ff01" },
            { 0x0000000000010076L, "f9fffd" },
            { 0x0000000000010077L, "f9fffe" },
            { 0x0000000000010078L, "f9ffff" },
            { 0x0000000000010079L, "fa010000" },
            { 0x000000000001007aL, "fa010001" },
            { 0x0000000000010176L, "fa0100fd" },
            { 0x0000000000010177L, "fa0100fe" },
            { 0x000000007fffffffL, "fb7fffff86" },
            { 0x7fffffffffffffffL, "ff7fffffffffffff86" },
    @DataProvider(name = "lengths")
    public Object[][] genLengths() {
        return new Object[][] {
            // Check cutoff values
            {   PackedLong.CUTOFF_VALUES[ 0] - 1,      9   },
            {   PackedLong.CUTOFF_VALUES[ 0],          8   },
            {   PackedLong.CUTOFF_VALUES[ 1] - 1,      8   },
            {   PackedLong.CUTOFF_VALUES[ 1],          7   },
            {   PackedLong.CUTOFF_VALUES[ 2] - 1,      7   },
            {   PackedLong.CUTOFF_VALUES[ 2],          6   },
            {   PackedLong.CUTOFF_VALUES[ 3] - 1,      6   },
            {   PackedLong.CUTOFF_VALUES[ 3],          5   },
            {   PackedLong.CUTOFF_VALUES[ 4] - 1,      5   },
            {   PackedLong.CUTOFF_VALUES[ 4],          4   },
            {   PackedLong.CUTOFF_VALUES[ 5] - 1,      4   },
            {   PackedLong.CUTOFF_VALUES[ 5],          3   },
            {   PackedLong.CUTOFF_VALUES[ 6] - 1,      3   },
            {   PackedLong.CUTOFF_VALUES[ 6],          2   },
            {   PackedLong.CUTOFF_VALUES[ 7] - 1,      2   },
            {   PackedLong.CUTOFF_VALUES[ 7],          1   },
            {   PackedLong.CUTOFF_VALUES[ 8] - 1,      1   },
            {   PackedLong.CUTOFF_VALUES[ 8],          2   },
            {   PackedLong.CUTOFF_VALUES[ 9] - 1,      2   },
            {   PackedLong.CUTOFF_VALUES[ 9],          3   },
            {   PackedLong.CUTOFF_VALUES[10] - 1,      3   },
            {   PackedLong.CUTOFF_VALUES[10],          4   },
            {   PackedLong.CUTOFF_VALUES[11] - 1,      4   },
            {   PackedLong.CUTOFF_VALUES[11],          5   },
            {   PackedLong.CUTOFF_VALUES[12] - 1,      5   },
            {   PackedLong.CUTOFF_VALUES[12],          6   },
            {   PackedLong.CUTOFF_VALUES[13] - 1,      6   },
            {   PackedLong.CUTOFF_VALUES[13],          7   },
            {   PackedLong.CUTOFF_VALUES[14] - 1,      7   },
            {   PackedLong.CUTOFF_VALUES[14],          8   },
            {   PackedLong.CUTOFF_VALUES[15] - 1,      8   },
            {   PackedLong.CUTOFF_VALUES[15],          9   },
            // Check some other values
            { Long.MIN_VALUE,                               9   },
            { Long.MAX_VALUE,                               9   },
            { (long)Integer.MIN_VALUE,                      5   },
            { (long)Integer.MAX_VALUE,                      5   },
            { (long)Short.MIN_VALUE,                        3   },
            { (long)Short.MAX_VALUE,                        3   },
            { (long)Byte.MIN_VALUE,                         2   },
            { (long)Byte.MAX_VALUE,                         2   },
            { 0,                                            1   },
}Edited by: archie172 on Mar 3, 2010 12:40 PM
Removed copyright message.