Package com.google.common.io

Class ByteStreams

  • @GwtIncompatible
public final class ByteStreams
extends java.lang.Object
    Provides utility methods for working with byte arrays and I/O streams.
    Since:
    1.0

    • Field Summary

      Fields 
      Modifier and Type Field Description
      private static int BUFFER_SIZE  
      private static int MAX_ARRAY_LEN
      Max array length on JVM.
      private static java.io.OutputStream NULL_OUTPUT_STREAM  
      private static int TO_BYTE_ARRAY_DEQUE_SIZE
      Large enough to never need to expand, given the geometric progression of buffer sizes.
      private static int ZERO_COPY_CHUNK_SIZE
      There are three methods to implement FileChannel.transferTo(long, long, WritableByteChannel): Use sendfile(2) or equivalent.

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      private ByteStreams()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      private static byte[] combineBuffers​(java.util.Deque<byte[]> bufs, int totalLen)  
      static long copy​(java.io.InputStream from, java.io.OutputStream to)
      Copies all bytes from the input stream to the output stream.
      static long copy​(java.nio.channels.ReadableByteChannel from, java.nio.channels.WritableByteChannel to)
      Copies all bytes from the readable channel to the writable channel.
      (package private) static byte[] createBuffer()
      Creates a new byte array for buffering reads or writes.
      static long exhaust​(java.io.InputStream in)
      Reads and discards data from the given InputStream until the end of the stream is reached.
      static java.io.InputStream limit​(java.io.InputStream in, long limit)
      Wraps a InputStream, limiting the number of bytes which can be read.
      static ByteArrayDataInput newDataInput​(byte[] bytes)
      Returns a new ByteArrayDataInput instance to read from the bytes array from the beginning.
      static ByteArrayDataInput newDataInput​(byte[] bytes, int start)
      Returns a new ByteArrayDataInput instance to read from the bytes array, starting at the given position.
      static ByteArrayDataInput newDataInput​(java.io.ByteArrayInputStream byteArrayInputStream)
      Returns a new ByteArrayDataInput instance to read from the given ByteArrayInputStream.
      static ByteArrayDataOutput newDataOutput()
      Returns a new ByteArrayDataOutput instance with a default size.
      static ByteArrayDataOutput newDataOutput​(int size)
      Returns a new ByteArrayDataOutput instance sized to hold size bytes before resizing.
      static ByteArrayDataOutput newDataOutput​(java.io.ByteArrayOutputStream byteArrayOutputSteam)
      Returns a new ByteArrayDataOutput instance which writes to the given ByteArrayOutputStream.
      static java.io.OutputStream nullOutputStream()
      Returns an OutputStream that simply discards written bytes.
      static int read​(java.io.InputStream in, byte[] b, int off, int len)
      Reads some bytes from an input stream and stores them into the buffer array b.
      static <T> T readBytes​(java.io.InputStream input, ByteProcessor<T> processor)
      Process the bytes of the given input stream using the given processor.
      static void readFully​(java.io.InputStream in, byte[] b)
      Attempts to read enough bytes from the stream to fill the given byte array, with the same behavior as DataInput.readFully(byte[]).
      static void readFully​(java.io.InputStream in, byte[] b, int off, int len)
      Attempts to read len bytes from the stream into the given array starting at off, with the same behavior as DataInput.readFully(byte[], int, int).
      static void skipFully​(java.io.InputStream in, long n)
      Discards n bytes of data from the input stream.
      private static long skipSafely​(java.io.InputStream in, long n)
      Attempts to skip up to n bytes from the given input stream, but not more than in.available() bytes.
      (package private) static long skipUpTo​(java.io.InputStream in, long n)
      Discards up to n bytes of data from the input stream.
      static byte[] toByteArray​(java.io.InputStream in)
      Reads all bytes from an input stream into a byte array.
      (package private) static byte[] toByteArray​(java.io.InputStream in, long expectedSize)
      Reads all bytes from an input stream into a byte array.
      private static byte[] toByteArrayInternal​(java.io.InputStream in, java.util.Deque<byte[]> bufs, int totalLen)
      Returns a byte array containing the bytes from the buffers already in bufs (which have a total combined length of totalLen bytes) followed by all bytes remaining in the given input stream.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • ZERO_COPY_CHUNK_SIZE

        private static final int ZERO_COPY_CHUNK_SIZE
        There are three methods to implement FileChannel.transferTo(long, long, WritableByteChannel):
        1. Use sendfile(2) or equivalent. Requires that both the input channel and the output channel have their own file descriptors. Generally this only happens when both channels are files or sockets. This performs zero copies - the bytes never enter userspace.
        2. Use mmap(2) or equivalent. Requires that either the input channel or the output channel have file descriptors. Bytes are copied from the file into a kernel buffer, then directly into the other buffer (userspace). Note that if the file is very large, a naive implementation will effectively put the whole file in memory. On many systems with paging and virtual memory, this is not a problem - because it is mapped read-only, the kernel can always page it to disk "for free". However, on systems where killing processes happens all the time in normal conditions (i.e., android) the OS must make a tradeoff between paging memory and killing other processes - so allocating a gigantic buffer and then sequentially accessing it could result in other processes dying. This is solvable via madvise(2), but that obviously doesn't exist in java.
        3. Ordinary copy. Kernel copies bytes into a kernel buffer, from a kernel buffer into a userspace buffer (byte[] or ByteBuffer), then copies them from that buffer into the destination channel.
        This value is intended to be large enough to make the overhead of system calls negligible, without being so large that it causes problems for systems with atypical memory management if approaches 2 or 3 are used.
        See Also:
        Constant Field Values

      • MAX_ARRAY_LEN

        private static final int MAX_ARRAY_LEN
        Max array length on JVM.
        See Also:
        Constant Field Values

      • TO_BYTE_ARRAY_DEQUE_SIZE

        private static final int TO_BYTE_ARRAY_DEQUE_SIZE
        Large enough to never need to expand, given the geometric progression of buffer sizes.
        See Also:
        Constant Field Values

      • NULL_OUTPUT_STREAM

        private static final java.io.OutputStream NULL_OUTPUT_STREAM

    • Constructor Detail

      • ByteStreams

        private ByteStreams()

    • Method Detail

      • createBuffer

        static byte[] createBuffer()
        Creates a new byte array for buffering reads or writes.

      • copy

        public static long copy​(java.io.InputStream from,
                        java.io.OutputStream to)
                 throws java.io.IOException
        Copies all bytes from the input stream to the output stream. Does not close or flush either stream.
        Parameters:
        from - the input stream to read from
        to - the output stream to write to
        Returns:
        the number of bytes copied
        Throws:
        java.io.IOException - if an I/O error occurs

      • copy

        public static long copy​(java.nio.channels.ReadableByteChannel from,
                        java.nio.channels.WritableByteChannel to)
                 throws java.io.IOException
        Copies all bytes from the readable channel to the writable channel. Does not close or flush either channel.
        Parameters:
        from - the readable channel to read from
        to - the writable channel to write to
        Returns:
        the number of bytes copied
        Throws:
        java.io.IOException - if an I/O error occurs

      • toByteArrayInternal

        private static byte[] toByteArrayInternal​(java.io.InputStream in,
                                          java.util.Deque<byte[]> bufs,
                                          int totalLen)
                                   throws java.io.IOException
        Returns a byte array containing the bytes from the buffers already in bufs (which have a total combined length of totalLen bytes) followed by all bytes remaining in the given input stream.
        Throws:
        java.io.IOException

      • combineBuffers

        private static byte[] combineBuffers​(java.util.Deque<byte[]> bufs,
                                     int totalLen)

      • toByteArray

        public static byte[] toByteArray​(java.io.InputStream in)
                          throws java.io.IOException
        Reads all bytes from an input stream into a byte array. Does not close the stream.
        Parameters:
        in - the input stream to read from
        Returns:
        a byte array containing all the bytes from the stream
        Throws:
        java.io.IOException - if an I/O error occurs

      • toByteArray

        static byte[] toByteArray​(java.io.InputStream in,
                          long expectedSize)
                   throws java.io.IOException
        Reads all bytes from an input stream into a byte array. The given expected size is used to create an initial byte array, but if the actual number of bytes read from the stream differs, the correct result will be returned anyway.
        Throws:
        java.io.IOException

      • exhaust

        @Beta
public static long exhaust​(java.io.InputStream in)
                    throws java.io.IOException
        Reads and discards data from the given InputStream until the end of the stream is reached. Returns the total number of bytes read. Does not close the stream.
        Throws:
        java.io.IOException
        Since:
        20.0

      • newDataInput

        @Beta
public static ByteArrayDataInput newDataInput​(byte[] bytes,
                                              int start)
        Returns a new ByteArrayDataInput instance to read from the bytes array, starting at the given position.
        Throws:
        java.lang.IndexOutOfBoundsException - if start is negative or greater than the length of the array

      • newDataInput

        @Beta
public static ByteArrayDataInput newDataInput​(java.io.ByteArrayInputStream byteArrayInputStream)
        Returns a new ByteArrayDataInput instance to read from the given ByteArrayInputStream. The given input stream is not reset before being read from by the returned ByteArrayDataInput.
        Since:
        17.0

      • newDataOutput

        @Beta
public static ByteArrayDataOutput newDataOutput​(int size)
        Returns a new ByteArrayDataOutput instance sized to hold size bytes before resizing.
        Throws:
        java.lang.IllegalArgumentException - if size is negative

      • newDataOutput

        @Beta
public static ByteArrayDataOutput newDataOutput​(java.io.ByteArrayOutputStream byteArrayOutputSteam)
        Returns a new ByteArrayDataOutput instance which writes to the given ByteArrayOutputStream. The given output stream is not reset before being written to by the returned ByteArrayDataOutput and new data will be appended to any existing content.

        Note that if the given output stream was not empty or is modified after the ByteArrayDataOutput is created, the contract for ByteArrayDataOutput.toByteArray() will not be honored (the bytes returned in the byte array may not be exactly what was written via calls to ByteArrayDataOutput).

        Since:
        17.0

      • nullOutputStream

        @Beta
public static java.io.OutputStream nullOutputStream()
        Returns an OutputStream that simply discards written bytes.
        Since:
        14.0 (since 1.0 as com.google.common.io.NullOutputStream)

      • limit

        @Beta
public static java.io.InputStream limit​(java.io.InputStream in,
                                        long limit)
        Wraps a InputStream, limiting the number of bytes which can be read.
        Parameters:
        in - the input stream to be wrapped
        limit - the maximum number of bytes to be read
        Returns:
        a length-limited InputStream
        Since:
        14.0 (since 1.0 as com.google.common.io.LimitInputStream)

      • readFully

        @Beta
public static void readFully​(java.io.InputStream in,
                             byte[] b)
                      throws java.io.IOException
        Attempts to read enough bytes from the stream to fill the given byte array, with the same behavior as DataInput.readFully(byte[]). Does not close the stream.
        Parameters:
        in - the input stream to read from.
        b - the buffer into which the data is read.
        Throws:
        java.io.EOFException - if this stream reaches the end before reading all the bytes.
        java.io.IOException - if an I/O error occurs.

      • readFully

        @Beta
public static void readFully​(java.io.InputStream in,
                             byte[] b,
                             int off,
                             int len)
                      throws java.io.IOException
        Attempts to read len bytes from the stream into the given array starting at off, with the same behavior as DataInput.readFully(byte[], int, int). Does not close the stream.
        Parameters:
        in - the input stream to read from.
        b - the buffer into which the data is read.
        off - an int specifying the offset into the data.
        len - an int specifying the number of bytes to read.
        Throws:
        java.io.EOFException - if this stream reaches the end before reading all the bytes.
        java.io.IOException - if an I/O error occurs.

      • skipFully

        @Beta
public static void skipFully​(java.io.InputStream in,
                             long n)
                      throws java.io.IOException
        Discards n bytes of data from the input stream. This method will block until the full amount has been skipped. Does not close the stream.
        Parameters:
        in - the input stream to read from
        n - the number of bytes to skip
        Throws:
        java.io.EOFException - if this stream reaches the end before skipping all the bytes
        java.io.IOException - if an I/O error occurs, or the stream does not support skipping

      • skipUpTo

        static long skipUpTo​(java.io.InputStream in,
                     long n)
              throws java.io.IOException
        Discards up to n bytes of data from the input stream. This method will block until either the full amount has been skipped or until the end of the stream is reached, whichever happens first. Returns the total number of bytes skipped.
        Throws:
        java.io.IOException

      • skipSafely

        private static long skipSafely​(java.io.InputStream in,
                               long n)
                        throws java.io.IOException
        Attempts to skip up to n bytes from the given input stream, but not more than in.available() bytes. This prevents FileInputStream from skipping more bytes than actually remain in the file, something that it specifies it can do in its Javadoc despite the fact that it is violating the contract of InputStream.skip().
        Throws:
        java.io.IOException

      • readBytes

        @Beta
public static <T> T readBytes​(java.io.InputStream input,
                              ByteProcessor<T> processor)
                       throws java.io.IOException
        Process the bytes of the given input stream using the given processor.
        Parameters:
        input - the input stream to process
        processor - the object to which to pass the bytes of the stream
        Returns:
        the result of the byte processor
        Throws:
        java.io.IOException - if an I/O error occurs
        Since:
        14.0

      • read

        @Beta
public static int read​(java.io.InputStream in,
                       byte[] b,
                       int off,
                       int len)
                throws java.io.IOException
        Reads some bytes from an input stream and stores them into the buffer array b. This method blocks until len bytes of input data have been read into the array, or end of file is detected. The number of bytes read is returned, possibly zero. Does not close the stream.

        A caller can detect EOF if the number of bytes read is less than len. All subsequent calls on the same stream will return zero.

        If b is null, a NullPointerException is thrown. If off is negative, or len is negative, or off+len is greater than the length of the array b, then an IndexOutOfBoundsException is thrown. If len is zero, then no bytes are read. Otherwise, the first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len.

        Parameters:
        in - the input stream to read from
        b - the buffer into which the data is read
        off - an int specifying the offset into the data
        len - an int specifying the number of bytes to read
        Returns:
        the number of bytes read
        Throws:
        java.io.IOException - if an I/O error occurs