serialization — Binary stream helper classes

This module provides classes to help share NEO data structures over the network between nodes.

The BinaryWriter and BinaryReader classes are convenience classes to help serialize and deserialize in a consistent, predictable and easy manner. They provide utility functions for easy writing and reading of Variable Length Encoded data.

Objects that want to be serializable through the reader and writer are encouraged to implement the ISerializable interface. An example using this interface can be seen next

from neo3.core import serialization, utils, Size

class SampleObject(serialization.ISerializable):

    data = bytearray([0x00] * 10)
    is_safe = True

    def serialize(self, writer: serialization.BinaryWriter) -> None:
        writer.write_var_bytes(self.data)
        writer.write_bool(self.is_safe)

    def deserialize(self, reader: serialization.BinaryReader) -> None:
        self.data = reader.read_var_bytes()
        self.is_safe = reader.read_bool()

    def __len__(self):
        return utils.get_var_size(self.data) + Size.uint8

with serialization.BinaryWriter() as bw:
    o = SampleObject()
    o.serialize(bw)
    stream = bw._stream.getvalue()
print(stream)

with serialization.BinaryReader(stream) as br:
    o = SampleObject()
    o.deserialize(br)
print(o.data, o.is_safe)

Working with objects that implement the ISerializable interface benefit from two convenience functions that can simplify the above to

o = SampleObject()
# direct serialization to bytes
print(o.to_array())

# direct deserialization from bytes
o = SampleObject.deserialize_from_bytes(stream)
print(o.data, o.is_safe)
class neo3.core.serialization.ISerializable

Bases: abc.ABC

An interface like class supporting NEO’s network serialization protocol.

abstract __len__()

Return the length of the object in number of bytes.

abstract deserialize(reader)

Deserialize the object from a binary stream.

Parameters

reader (BinaryReader) – instance.

Return type

None

classmethod deserialize_from_bytes(data)

Parse data into an object instance.

Parameters

data (Union[bytes, bytearray]) – hex escaped bytes.

Return type

~ISerializable_T

Returns

a deserialized instance of the class.

abstract serialize(writer)

Serialize the object into a binary stream.

Parameters

writer (BinaryWriter) – instance.

Return type

None

to_array()

Serialize the object into a bytearray.

Return type

bytes

class neo3.core.serialization.BinaryReader(stream)

Bases: object

Create an instance.

Parameters

stream (Union[bytes, bytearray]) – a stream to operate on.

close()

Close the internal stream to prevent resource leaking.

Note

This is done automatically when using the context manager

Return type

None

read_bool()

Read 1 byte as a boolean value from the stream.

Returns

False for b’'. True for all other values.

Return type

bool

read_byte()

Read a single byte.

Raises

ValueError – if 1 byte of data cannot be read from the stream.

Returns

a hex escaped bytearray with 1 element.

Return type

bytes

read_bytes(length, _skip_length_check=False)

Read the specified number of bytes from the stream.

Parameters

length (int) – number of bytes to read.

Raises

ValueError – if length bytes of data cannot be read from the stream.

Returns

length number of bytes.

Return type

bytes

read_int16()

Read 2 bytes as an unsigned integer value from the stream.

Return type

int

read_int16BE()

Read 2 bytes as an unsigned integer value from the stream.

Return type

int

read_int32()

Read 4 bytes as a signed integer value from the stream.

Return type

int

read_int32BE()

Read 4 bytes as a signed integer value from the stream.

Return type

int

read_int64()

Read 8 bytes as a signed integer value from the stream.

Return type

int

read_int64BE()

Read 8 bytes as a signed integer value from the stream.

Return type

int

read_serializable(obj_type)

Read and deserialize an object of obj_type from the stream.

Parameters

obj_type (Type[~ISerializable_T]) – the object class to deserialize into.

Return type

~ISerializable_T

read_serializable_list(obj_type, max=None)

Read and deserialize a list of objects of obj_type from the stream.

Expects to start with a varint list length indicator.

Parameters
  • obj_type (Type[~ISerializable_T]) – the object class to deserialize into.

  • max (Optional[int]) – read up to max objects from the stream.

Returns

list of deserialized objects.

Return type

list[ISerializable]

read_uint16()

Read 2 bytes as an unsigned integer value from the stream.

Return type

int

read_uint16BE()

Read 2 bytes as an unsigned integer value from the stream.

Return type

int

read_uint32()

Read 4 bytes as an unsigned integer value from the stream.

Return type

int

read_uint32BE()

Read 4 bytes as an unsigned integer value from the stream.

Return type

int

read_uint64()

Read 8 bytes as an unsigned integer value from the stream.

Return type

int

read_uint64BE()

Read 8 bytes as an unsigned integer value from the stream.

Return type

int

read_uint8()

Read 1 byte as an unsigned integer value from the stream.

Return type

int

read_var_bytes(max=9223372036854775807)

Read bytes that starts with a variable length indicator.

The NEO network protocol supports encoded length indicating for saving space. See: Variable Length Encoding

Note

This will try to read the amount of bytes as encoded in the stream but will read less if insufficient data is available.

Parameters

max (int) – (Optional) maximum number of bytes to read.

Return type

bytes

read_var_int(max=9223372036854775807)

Read a integer that starts with a variable length indicator.

The NEO network protocol supports encoded length indicating for saving space. See: Variable Length Encoding

Parameters

max (int) – (Optional) maximum number of bytes to read.

Raises

ValueError – if the return value exceeds the max argument.

Return type

int

read_var_string(max=9223372036854775807)

Read a UTF-8 string that starts with a variable length indicator.

The NEO network protocol supports encoded length indicating for saving space. See: Variable Length Encoding

Parameters

max (int) – (Optional) maximum number of bytes to read.

Raises

ValueError – if decoding fails or insufficient data is present in the stream.

Return type

str

class neo3.core.serialization.BinaryWriter(stream=None)

Bases: object

A convenience class for writing data to byte streams.

Context manager support is available to ensure proper cleanup of resources.

Example:

with serialization.BinaryWriter() as bw:
    bw.write_uint8(5)
    self.assertEqual(b'\x05', bw._stream.getvalue())

Create an instance.

Parameters

stream (Union[bytearray, bytes, None]) – a stream to operate on.

close()

Close the internal stream to prevent resource leaking.

Note

This is done automatically when using the context manager

Return type

None

to_array()

Get the raw bytes from the underlying stream.

Return type

bytes

write_bool(value)

Pack the value as a bool and write 1 byte to the stream.

Parameters

value (bool) – the boolean value to write.

Returns

the number of bytes written.

Return type

int

write_bytes(value)

Write a bytes type to the stream.

Parameters

value (bytes) – array of bytes to write to the stream.

Returns

the number of bytes written.

Return type

int

write_int16(value, endian='<')

Pack the value as a signed integer and write 2 bytes to the stream. :type value: int :param value: integer value to write to the stream. :type endian: str :param endian: specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_int32(value, endian='<')

Pack the value as a signed integer and write 4 bytes to the stream. :type value: int :param value: integer value to write to the stream. :type endian: str :param endian: specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_int64(value, endian='<')

Pack the value as an unsigned integer and write 8 bytes to the stream. :type value: int :param value: integer value to write to the stream. :type endian: str :param endian: specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian. :type endian: str

Returns

the number of bytes written.

Return type

int

write_serializable(obj_instance)

Serialize an object instance and write it to the stream.

Parameters

obj_instance (~ISerializable_T) – the instance to serialize.

Return type

None

write_serializable_list(objects)

Serialize a list of objects and write them to the stream.

Parameters

objects (List[~ISerializable_T]) – a list of objects.

Return type

None

write_uint16(value, endian='<')

Pack the value as an unsigned integer and write 2 bytes to the stream.

Parameters
  • value (int) – integer value to write to the stream.

  • endian (str) – specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_uint32(value, endian='<')

Pack the value as a signed integer and write 4 bytes to the stream.

Parameters
  • value (int) – integer value to write to the stream.

  • endian (str) – specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_uint64(value, endian='<')

Pack the value as an unsigned integer and write 8 bytes to the stream.

Parameters
  • value (int) – integer value to write to the stream.

  • endian (str) – specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_uint8(value)
Parameters

value – integer value to write to the stream.

Returns

the number of bytes written.

Return type

int

write_var_bytes(value, endian='<')

Write bytes into a stream with variable length prefix.

The NEO network protocol supports encoded length indicating for saving space. See Variable Length Encoding

Parameters
  • value (bytes) – bytes to write to the stream.

  • endian (str) – specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Returns

the number of bytes written.

Return type

int

write_var_int(value, endian='<')

Write an integer value in a space saving way to the stream.

The NEO network protocol supports encoded length indicating for saving space. See: Variable Length Encoding

Parameters
  • value (int) – integer value to write to the stream.

  • endian (str) – specify the endianness. (Default) Little endian (‘<’). Use ‘>’ for big endian.

Raises
Returns

the number of bytes written.

Return type

int

write_var_string(value, encoding='utf-8')

Write a string value to the stream.

The NEO network protocol supports encoded length indicating for saving space. See: Variable Length Encoding

Parameters
  • value (str) – string to write to the stream.

  • encoding (str) – string encoding format.

Return type

int