brief introduction JAVA new I/O (NIO)

Reference document: Getting started with new I/O (NIO)

Preface:

NIO was introduced with JDK1.4 for high speed, block oriented I/O in standard Java code. By defining classes to hold data, and by processing that data in blocks, NIO takes advantage of low-level optimizaitons compared to I/O package could not, without using native libraries.

We will cover most aspect of the NIO library, from high level conceptual stuff to under-the-hood programming detail. To learning about crucial I/O elements like buffers and channels, we will also see how standard I/O elements works in the updated library. You'll also learn about things you can only do with NIO, such as asynchronous I/O and direct buffers.

Input/output: A conceptual overview

I/O in Java programming, I/O has until recently been carried out using a stream metaphor. All I/O is viewed as the movement of single bytes, one at a time, through an object called a stream. Stream I/O is used for contacting the outside world, and it can also used internally for turning objects into bytes and then back into objects.

NIO plays the same role as original I/O, but it uses a different metaphor -- block I/O. As you can see the Java NIO Buffers, the block I/O is more efficient than stream I/O.

Why NIO?

NIO was created allow Java program to implement high-speed I/O without having to write custom native code. NIO moves the most time-consuming I/O activities(namely, filling and draining buffers) back into the OS, thus allowing great increase in speed.

Streams vs blocks

The most important distinction between the original I/O library and NIO has to do with how data is packaged and transmitted. The original I/O deals with data in streams, whereas NIO do with data in blocks.

A stream-oriented I/O system deals with data one byte at a time. An input stream produces one byte of data, and an output stream consumes one byte of data. It's very easy to create filters for stream data , also relatively simply to chain several filters together so that each one does its part in what amounts to single sophisticated processing mechanism. On the flip side, stream-oriented I/O is often rather slow.

Integrated I/O

The original I/O package and NIO have been well integrated in JDK 1.4 java.io.* has been reimplemented using NIO as its base , so it can now take advantage of some features of NIO. For example some java some of classes in the java.io.* package contain methods to read and write data in blocks, which leads to faster processing.

Channel and buffers

Channels and buffers are the central
objects in NIO, and are used for just about every I/O operation.
Channels are analogous to stream in the original IO package. All data
that goes anywhere must pass through a Channel object. A buffer is
essentially a container object. All data that is sent to a channel
must first be placed in a buffer. So any data that is read from a
channel is read into buffer.

What
is a buffer?

A buffer is an object, which is essentially an array. It's an
array of bytes, but other kinds of arrays can be used. But a buffer
is more than just an array. A buffer provides structured access to
data and also keeps track of the system's read/write processes.

In the NIO library, all data is handled with buffers. When data is
read, it's read directly into a buffer, when data is written, it's
written into a buffer. Anytime you access data in NIO, you are
pulling it out of the buffer. A buffer is essentially an array.

What 's a channel ?
A channel is an object from which you can
read data and to which you can write data.Comparing NIO with original
I/O, a channel is like a stream.

Kinds of Channel

Channels differ from stream in that they are bi-directional.
Whereas streams only go in one direction (a stream must be a subclass
of either InputStream or OutputStream), a channel can be opened for
reading, for writing, or for both.

Because they are bi-directional, channels better reflect the
reality of the underlying OS than streams do. In the UNIX model in
particular, the underlying OS channels are bi-directional.

Practice： Reading and
writing in NIO

Reading from a channel is simple: we create a buffer and then ask
a channel to read data into it. Writing data we just create a buffer,
fill it will data, and then ask a channel to write from it.

Reading from a file

For our first exercise, we'll read some data from a file. If we
are using original IO, we will simply create a FileInputStream and
read from that.

In NIO, however, things work a little differently; we first get Channel object from the FileInputStream, and then use that channel to read the data.

Any time you perform a read operation in an NIO system.you are reading from a channel, but you don’t read directly from a channel. Since the data ultimate resides in the buffer, you read from a channel to a buffer.

So reading from a file involves three steps:

getting the Channel from FileInputStream;
creating the Buffer
reading from the Channel into the buffer

Three easy steps：
Our first step is to get a channel. We get the channel from the FileInputStream.

FileInputStream fin = new FileInputStream("DirectMem.java");

FileChannel fc = fin.getChannel();

The next step is to create a buffer and read from the channel into the buffer, as shown here:

ByteBuffer buffer = ByteBuffer.allocate(1024);

fc.read(buffer);

We don't need to tell the channel how much to read into buffer. Each buffer has ssophisticated internal accounting system that keeps track of how much data has been read and how much room there is for more data. We will discuss the Buffer internals overview then.

Writing to a file

Writing is similiar to reading from a file. Here is the sample code:

    private static void readingWritingFromFile() throws IOException{

        String fileName = "DirectMem.java";

        FileInputStream fin = new FileInputStream(fileName);

        FileChannel fc = fin.getChannel();

        int bufLen = 2<<10;

        ByteBuffer buffer = ByteBuffer.allocate(bufLen);

        fc.read(buffer);

        String newFileName = "newDirectMem.txt";

        FileOutputStream fout = new FileOutputStream(newFileName);

        FileChannel writeFc = fout.getChannel();

        ByteBuffer newBuf = ByteBuffer.allocate(bufLen);

        String message = "test writing to channel from buffer";

        byte[] mesArray = message.getBytes();

        for (int i = 0; i < mesArray.length; i++) {

            newBuf.put(mesArray[i]);

        }

        newBuf.flip();

        writeFc.write(newBuf);

    }

Other reading and writing operation please see Reading and Writing with Java NIO

There some examples we combine reads data into the buffer from the input channel, fcin, and the second line writes the data to the output channel, fcout. Notice that before reading into a buffer from the input channel we call the method clear(), and before write newly read data into another channel we call the flip() method prepare the buffer.

Buffer internals

In this section, we will see 2 components of buffers in NIO: state variables and accessor methods.

State variables are key to the "internal accouting system" mentioned in the previous section. With each read/write operation, the internal states changes. By recording and tracking those changes, a buffer is can manage it's own resources.

A buffer has many states, in some cases write data directly into another channel, but often you'll want to look at the data itself. This is acopmlished using the accessor method get(), while if if you want to put raw data in a buffer, you use the accessor method put().

State variables

3 values can be used to specify the state of a buffer:

position
limit
capacity

these variables track the state of the buffer and the data is contains. We'll examine each one in detail, and also see how to fit read/write process. For the sake of the example, we'll assume that we are copying data from an input channel to an output channel.

Position

You will recall that a buffer is really just a glorified array when you read from a channel, position records how much data you have written. It specifies into where is array element the next byte. If you write to a channel, postion will record how much data read from the buffer. It refers to the next eelement you want to read.

Limit

The limit variable specifies how much data there is left to get, or how much room left to put data into.

Capacity

The capacity of a buffer sepcifies the maximum amount of data that can be stored therein. It specifies the size of underlyging array.

The limit is smaller than capacity.

We create buffer as example, let's assume that our buffer capacity is 8 bytes, and the buffer states is shown:

React operations

The limit can not be larger than the ccapacity, and in this case both values are set to 8. We show this by pointing them off the end of the array. The position is set to 0, when read data into the buffer, then the next byte read will be put into slot0; If we write from the buffer, the next byte taken from the buffer will be taken from slot 0.

React operations

When we read data into the buffer, increase the position. In this example we read 5 bytes so add 5 to pposition.

After read byte into buffer from channel, we want write bytes from buffer to channel. But before change ooperation to buffer, we should first call the flip() function. It resets position to 0, and set limit to the position old value.

React operations

You can see the posiiton set to 0 and limit set to 5(Meaning all data we read from buffer just now), Then we begin write bytes from the buffer to channel. And we read 5 bytes from the buffer, add 5 to position and now you can see position is equal to limit.

When all read and write operations finished, we call clear() method to clear all variables:

1. reset positon to 0.

2. limit set to capacity

Conclusion

If you want manipulate data in disk you should firstly read it into buffer from the channel, or if you want to store data back to disk you should write channel from the buffer.

For more detail about manipulation about buffer, please see Java NIO read/write file through FileChannel