This sub-package provides the capability to compress and decompress data using the stream specification, especially the stream specification based on a double buffer.
Because the LZ4 stream format does not define a container format, the Python bindings will by default insert the compressed data size as an integer at the start of the compressed payload. However, it is possible to set the bit depth of this compressed data size.
So far, only the double-buffer based approach is implemented.
To use the lz4 stream format bindings is straightforward:
>>> from lz4.stream import LZ4StreamCompressor, LZ4StreamDecompressor
>>> import os
>>> block_size_length = 2 # LZ4 compressed block size stored on 2 bytes
>>> page_size = 8192 # LZ4 context double buffer page size
>>> origin_stream = 10 * 1024 * os.urandom(1024) # 10MiB
>>> # LZ4 stream compression of origin_stream into compressed_stream:
>>> compressed_stream = bytearray()
>>> with LZ4StreamCompressor("double_buffer", page_size, store_comp_size=block_size_length) as proc:
... offset = 0
... while offset < len(origin_stream):
... chunk = origin_stream[offset:offset + page_size]
... block = proc.compress(chunk)
... compressed_stream.extend(block)
... offset += page_size
>>> # LZ4 stream decompression of compressed_stream into decompressed_stream:
>>> decompressed_stream = bytearray()
>>> with LZ4StreamDecompressor("double_buffer", page_size, store_comp_size=block_size_length) as proc:
... offset = 0
... while offset < len(compressed_stream):
... block = proc.get_block(compressed_stream[offset:])
... chunk = proc.decompress(block)
... decompressed_stream.extend(chunk)
... offset += block_size_length + len(block)
>>> decompressed_stream == origin_stream
True
A Python wrapper for the LZ4 stream protocol.
lz4.stream.
LZ4StreamCompressor
(strategy, buffer_size, mode='default', acceleration=True, compression_level=9, return_bytearray=False, store_comp_size=4, dictionary='')¶LZ4 stream compressing context.
__enter__
()¶Enter the LZ4 stream context.
__exit__
(exc_type, exc, exc_tb)¶Exit the LZ4 stream context.
compress
(chunk)¶Stream compress given chunk
of data.
Compress the given chunk
, using the given LZ4 stream context,
returning the compressed data as a bytearray
or as a bytes
object.
Parameters: | chunk (str, bytes or buffer-compatible object) – Data to compress |
---|---|
Returns: | Compressed data. |
Return type: | bytes or bytearray |
Raises: |
|
lz4.stream.
LZ4StreamDecompressor
(strategy, buffer_size, return_bytearray=False, store_comp_size=4, dictionary='')¶LZ4 stream decompression context.
__enter__
()¶Enter the LZ4 stream context.
__exit__
(exc_type, exc, exc_tb)¶Exit the LZ4 stream context.
decompress
(chunk)¶Decompress streamed compressed data.
Decompress the given chunk
, using the given LZ4 stream context,
Raises an exception if any error occurs.
Parameters: | chunk (str, bytes or buffer-compatible object) – Data to decompress |
---|---|
Returns: | Decompressed data. |
Return type: | bytes or bytearray |
Raises: |
|
get_block
(stream)¶Return the first LZ4 compressed block from stream
.
Parameters: | stream (str, bytes or buffer-compatible object) – LZ4 compressed stream. |
---|---|
Returns: | LZ4 compressed data block. |
Return type: | bytes or bytearray |
Raises: |
|