Package org.apache.commons.compress.compressors.lz77support

Class AbstractLZ77CompressorInputStream

java.lang.Object

java.io.InputStream

org.apache.commons.compress.compressors.CompressorInputStream

org.apache.commons.compress.compressors.lz77support.AbstractLZ77CompressorInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, InputStreamStatistics

Direct Known Subclasses:

BlockLZ4CompressorInputStream, SnappyCompressorInputStream

public abstract class AbstractLZ77CompressorInputStream
extends CompressorInputStream
implements InputStreamStatistics

Encapsulates code common to LZ77 decompressors.

Assumes the stream consists of blocks of literal data and back-references (called copies) in any order. Of course the first block must be a literal block for the scheme to work - unless the prefill method has been used to provide initial data that is never returned by read but only used for back-references.

Subclasses must override the three-arg read method as the no-arg version delegates to it and the default implementation delegates to the no-arg version, leading to infinite mutual recursion and a StackOverflowError otherwise.

The contract for subclasses' read implementation is:

• keep track of the current state of the stream. Is it inside a literal block or a back-reference or in-between blocks?
• Use readOneByte() to access the underlying stream directly.
• If a new literal block starts, use startLiteral(long) to tell this class about it and read the literal data using readLiteral(byte[], int, int) until it returns 0. hasMoreDataInBlock() will return false before the next call to readLiteral(byte[], int, int) would return 0.
• If a new back-reference starts, use startBackReference(int, long) to tell this class about it and read the literal data using readBackReference(byte[], int, int) until it returns 0. hasMoreDataInBlock() will return false before the next call to readBackReference(byte[], int, int) would return 0.
• If the end of the stream has been reached, return -1 as this class' methods will never do so themselves.

readOneByte() and readLiteral(byte[], int, int) update the counter for bytes read.

Since:

1.14

Field Summary

Fields

Modifier and Type	Field	Description
private int	backReferenceOffset	Offset of the current back-reference.
private final byte[]	buf	Buffer to write decompressed bytes to for back-references, will be three times windowSize big.
private long	bytesRemaining	Number of bytes still to be read from the current literal or back-reference.
private final CountingInputStream	in	The underlying stream to read compressed data from
private final byte[]	oneByte
private int	readIndex	Index of the next byte to be read.
private int	size	uncompressed size
protected final ByteUtils.ByteSupplier	supplier	Supplier that delegates to readOneByte().
private final int	windowSize	Size of the window - must be bigger than the biggest offset expected.
private int	writeIndex	One behind the index of the last byte in the buffer that was written, i.e.

Constructor Summary

Constructors

Constructor	Description
AbstractLZ77CompressorInputStream(InputStream is, int windowSize)	Creates a new LZ77 input stream.

Method Summary

Modifier and Type	Method	Description
int	available()
void	close()
long	getCompressedCount()
int	getSize()	Get the uncompressed size of the stream
protected final boolean	hasMoreDataInBlock()	Is there still data remaining inside the current block?
void	prefill(byte[] data)	Adds some initial data to fill the window with.
int	read()
protected final int	readBackReference(byte[] b, int off, int len)	Reads data from the current back-reference.
private int	readFromBuffer(byte[] b, int off, int len)
protected final int	readLiteral(byte[] b, int off, int len)	Reads data from the current literal block.
protected final int	readOneByte()	Reads a single byte from the real input stream and ensures the data is accounted for.
private void	slideBuffer()
protected final void	startBackReference(int offset, long length)	Used by subclasses to signal the next block contains a back-reference with the given coordinates.
protected final void	startLiteral(long length)	Used by subclasses to signal the next block contains the given amount of literal data.
private void	tryToCopy(int bytesToCopy)
private void	tryToReadLiteral(int bytesToRead)

Methods inherited from class org.apache.commons.compress.compressors.CompressorInputStream

count, count, getBytesRead, getCount, getUncompressedCount, pushedBackBytes

Methods inherited from class java.io.InputStream

mark, markSupported, nullInputStream, read, read, readAllBytes, readNBytes, readNBytes, reset, skip, skipNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.commons.compress.utils.InputStreamStatistics

getUncompressedCount

Field Details

windowSize

private final int windowSize

Size of the window - must be bigger than the biggest offset expected.

buf

private final byte[] buf

Buffer to write decompressed bytes to for back-references, will be three times windowSize big.

Three times so we can slide the whole buffer a windowSize to the left once we've read twice windowSize and still have enough data inside of it to satisfy back-references.

writeIndex

private int writeIndex

One behind the index of the last byte in the buffer that was written, i.e. the next position to write to

readIndex

private int readIndex

Index of the next byte to be read.

in

private final CountingInputStream in

The underlying stream to read compressed data from

bytesRemaining

private long bytesRemaining

Number of bytes still to be read from the current literal or back-reference.

backReferenceOffset

private int backReferenceOffset

Offset of the current back-reference.

size

private int size

uncompressed size

oneByte

private final byte[] oneByte

supplier

protected final ByteUtils.ByteSupplier supplier

Supplier that delegates to readOneByte().

Constructor Details

AbstractLZ77CompressorInputStream

public AbstractLZ77CompressorInputStream(InputStream is,
 int windowSize)
                                  throws IOException

Creates a new LZ77 input stream.

Parameters:

is - An InputStream to read compressed data from

windowSize - Size of the window kept for back-references, must be bigger than the biggest offset expected.

Throws:

IOException - if reading fails

IllegalArgumentException - if windowSize is not bigger than 0

Method Details

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class InputStream

Throws:

IOException

available

public int available()

Overrides:

available in class InputStream

getSize

public int getSize()

Get the uncompressed size of the stream

Returns:

the uncompressed size

prefill

public void prefill(byte[] data)

Adds some initial data to fill the window with.

This is used if the stream has been cut into blocks and back-references of one block may refer to data of the previous block(s). One such example is the LZ4 frame format using block dependency.

Parameters:

data - the data to fill the window with.

Throws:

IllegalStateException - if the stream has already started to read data

getCompressedCount

public long getCompressedCount()

Specified by:

getCompressedCount in interface InputStreamStatistics

Returns:

the amount of raw or compressed bytes read by the stream

Since:

1.17

startLiteral

protected final void startLiteral(long length)

Used by subclasses to signal the next block contains the given amount of literal data.

Parameters:

length - the length of the block

Throws:

IllegalArgumentException - if length is negative

hasMoreDataInBlock

protected final boolean hasMoreDataInBlock()

Is there still data remaining inside the current block?

Returns:

true if there is still data remaining inside the current block.

readLiteral

protected final int readLiteral(byte[] b,
 int off,
 int len)
                         throws IOException

Reads data from the current literal block.

Parameters:

b - buffer to write data to

off - offset to start writing to

len - maximum amount of data to read

Returns:

number of bytes read, may be 0. Will never return -1 as EOF-detection is the responsibility of the subclass

Throws:

IOException - if the underlying stream throws or signals an EOF before the amount of data promised for the block have been read

NullPointerException - if b is null

IndexOutOfBoundsException - if off is negative, len is negative, or len is greater than b.length - off

tryToReadLiteral

private void tryToReadLiteral(int bytesToRead)
                       throws IOException

Throws:

IOException

readFromBuffer

private int readFromBuffer(byte[] b,
 int off,
 int len)

slideBuffer

private void slideBuffer()

startBackReference

protected final void startBackReference(int offset,
 long length)

Used by subclasses to signal the next block contains a back-reference with the given coordinates.

Parameters:

offset - the offset of the back-reference

length - the length of the back-reference

Throws:

IllegalArgumentException - if offset not bigger than 0 or bigger than the number of bytes available for back-references or if length is negative

readBackReference

protected final int readBackReference(byte[] b,
 int off,
 int len)

Reads data from the current back-reference.

Parameters:

b - buffer to write data to

off - offset to start writing to

len - maximum amount of data to read

Returns:

number of bytes read, may be 0. Will never return -1 as EOF-detection is the responsibility of the subclass

Throws:

NullPointerException - if b is null

IndexOutOfBoundsException - if off is negative, len is negative, or len is greater than b.length - off

tryToCopy

private void tryToCopy(int bytesToCopy)

readOneByte

protected final int readOneByte()
                         throws IOException

Reads a single byte from the real input stream and ensures the data is accounted for.

Returns:

the byte read as value between 0 and 255 or -1 if EOF has been reached.

Throws:

IOException - if the underlying stream throws