devNotes 11-17-2016 File IO

http://argos.vu/devnotes-11-11-16-save-and-reload/

Problem and Solution: The Terrible Inefficiency of FileStream and BinaryReader

File I/O can be a major performance bottleneck for many apps. It’s all too easy to read files in a way that is massively inefficient. Classes like FileStream and BinaryReader make it really easy to write super slowcode. Today’s article explores why this happens and what can be done about it. Read on to learn more!

Say you want to read a 10 megabyte file containing a huge series of unsigned 2-byte values: ushort. How do you read this file? The typical approach would involve a FileStream and a BinaryReader like this:

// Open the file
using (var stream = new FileStream("/path/to/file", FileMode.Open))
{
	// Compute how many values the file has
	var numValues = stream.Length / sizeof(ushort);
 
	// Allocate an array to hold all those values
	var readValues = new ushort[numValues];
 
	// Open a reader to make reading those values easy
	using (var reader = new BinaryReader(stream))
	{
		// Read all the values
		for (var i = 0; i < numValues; ++i)
		{
			readValues[i] = reader.ReadUInt16();
		}
	}
}

On my test system (described later), this takes about 272 milliseconds! That’s definitely long enough that you’ll need to spread the load out over multiple frames of the game in order to keep a smooth frame rate. Maybe you’ll use another thread to handle the task. Regardless, your code is about to get a lot more complex, error-prone, and difficult to manage. Not to mention the user is going to have to wait longer for the file to get loaded.

But what if we could load the file faster? We know it’s important to minimize the number of read() calls made to the OS and FileStream does that for us. It has an internal buffer that defaults to a perfectly-reasonable 4 kilobytes in size. So when you read the first two bytes of the file it actually reads 4 kilobytes. The second read just takes two bytes out of the buffer and doesn’t make any read() calls to the OS. Great!

With this in mind we can plot the performance of different size reads from FileStream using its Readfunction:

var numBytesRead = myFileStream.Read(myBuffer, offsetIntoMyBuffer, numBytesToRead);

Here’s what happens when I run that test on the same test system as above:

The read size makes a huge difference! Just reading 4096 bytes at a time instead of 2 bytes at a time is 75x faster! But why is this the case? Isn’t FileStream just copying from its internal buffer? Isn’t that way faster than reading from the file system?

The answer lies in the CPU performance of FileStream, not the disk I/O performance. To find out why it’s so slow, let’s decompile the mscorlib.dll that comes with Unity 5.4’s version of Mono and see how FileStream is implemented. First, here’s the Read we call from our app:

public override int Read([In] [Out] byte[] array, int offset, int count)
{
	if (this.handle == MonoIO.InvalidHandle)
	{
		throw new ObjectDisposedException("Stream has been closed");
	}
	if (array == null)
	{
		throw new ArgumentNullException("array");
	}
	if (!this.CanRead)
	{
		throw new NotSupportedException("Stream does not support reading");
	}
	int num = array.Length;
	if (offset < 0)
	{
		throw new ArgumentOutOfRangeException("offset", "< 0");
	}
	if (count < 0)
	{
		throw new ArgumentOutOfRangeException("count", "< 0");
	}
	if (offset > num)
	{
		throw new ArgumentException("destination offset is beyond array size");
	}
	if (offset > num - count)
	{
		throw new ArgumentException("Reading would overrun buffer");
	}
	if (this.async)
	{
		IAsyncResult asyncResult = this.BeginRead(array, offset, count, null, null);
		return this.EndRead(asyncResult);
	}
	return this.ReadInternal(array, offset, count);
}

Whoa! 8 if statements right off the bat! Those can be quite slow because the CPU doesn’t know which code—the if or the else—is going to get executed, so it breaks its ability to predict and slows the code down. Notice that they’re mostly sanity checks, too. Is the stream open? Did you pass a null array? Can you read from this stream? Are your offset and count non-negative? The odds are extremely high that you’re not making any of these mistakes, but you’re going to pay for them 5,242,880 times over the course of a 10 megabyte file!

Next let’s look at the ReadInternal call at the end of Read:

Here we’ve got one or two more if statements for a total of 9-10 so far across three function calls (FileStream.Read, Stream.Count, FileStream.ReadInternal). Right away this function calls into another function: ReadSegment

Two more if statements for a total of 11-12 and 4 function calls. Here’s Buffer.BlockCopy:

That’s three more function calls for a total of 8, two of which are in native C code. ByteLength does some more redundant if checking to bring the count up to 19-20.

After that in ReadInternal we’re just dealing with the cases where data needs to actually be read from the file system. If we’re reading 2 bytes at a time with a 4096 byte buffer, that’s an extreme minority of calls. Feel free to look through the code with a decompiler if you want, but I’ll skip it for the purposes of this article.

Now that we’ve seen that a call to FileStream.Read results in 8 function calls and 19-20 if statements just to read two bytes, let’s look and see what additional overhead is incurred by using BinaryReader. Here are the functions involved in calling its ReadUInt16:

This brings the total up to 11 function calls and 23-24 if statements. BinaryReader didn’t add too much, but the overall process is super wasteful leading to the awful performance graph above.

So how can we optimize this? The goal must surely be to minimize calls to FileStream.Read. The BinaryReader class will call it every single time we want to read two bytes, so we can’t use that. It’s a shame since functions like ReadUInt16 are so useful for reading formatted data and not just blocks of bytes.

There’s nothing stopping us from writing our own BufferedBinaryReader though. The goal is to keep a buffer inside BufferedBinaryReader with the same size as the one in the FileStream. Then we can call Read just once to fill our buffer and only incur all those function calls and if statements every 2048 times we read a ushort with a 4 kilobyte buffer. For the other 2047 times we can just assume that the user knows what they’re doing and skip all the checks. After all, they’ll just get an IndexOutOfRangeExceptioninstead of the various exception types in FileStream and BinaryReader, so it’s hardly any less safe.

Lastly, we provide some functionality to control the buffer. This comes in the form of an explicit FillBufferfunction that uses the FileStream to refill its buffer and a NumBytesAvailable property to check how many bytes have been buffered and are ready for reading.

To use BufferedBinaryReader we write a very simple loop:

The BufferedBinaryReader.ReadUInt16 just looks like this:

The full source code for BufferedBinaryReader is in the following test script. It’s very basic as it only has the ability to read ushort values, but feel free to port the rest of the ReadX functions from mscorlib.dll if you’d like.

Now let’s put BufferedBinaryReader to the test against BinaryReader. The following test script has the test code for that as well as the performance chart above.

If you want to try out the test yourself, simply paste the above code into a TestScript.cs file in your Unity project’s Assets directory and attach it to the main camera game object in a new, empty project. Then build in non-development mode for 64-bit processors and run it windowed at 640×480 with fastest graphics. I ran it that way on this machine:

• 2.3 Ghz Intel Core i7-3615QM
• Mac OS X 10.11.5
• Apple SSD SM256E, HFS+ format
• Unity 5.4.0f3, Mac OS X Standalone, x86_64, non-development
• 640×480, Fastest, Windowed

And here are the results I got:

This shows that BufferedBinaryReader is about 11x faster than regular BinaryReader! That shouldn’t be much of a surprise since the performance test above showed similar times for 2 byte reads (272 here, 227 there) versus 4096 byte reads (24 here, 3 there). Remember the discussion of spreading the file loading out over multiple frames and maybe reading it from another thread? At only 24 milliseconds for a 10 megabyte file, that’s probably not going to be necessary if you have a more reasonable file size like 1 megabyte. Your code can be a lot simpler and the user can have their data immediately!

In conclusion, FileStream has a lot of CPU overhead for every call to its Read function regardless of the internal buffering its doing for calls to the OS’ native read function. To speed up your reads from disk, read as much as you can into your own buffer to minimize the FileStream overhead. A class like BufferedBinaryReader can make this pretty painless and massively speed up your performance.

If you’ve got any questions or tips you’d like to share about file I/O performance in Unity, please drop me a line in the comments!