Menu

vInsert.nl

What am I even on about

Golang Readers and Writers

Reader, Writer, ReaderWriter and ReaderWriterCloser are interfaces, interfaces that simply implement Read, Write, Close or any combination. They all take a simple byte slice as their only parameter. Either reading to, or reading from these buffers.

Let’s read a file and output the contents!

Simple enough, right?
1. We open a file handle.
2. Create a buffer to hold the contents (I used a 32kb buffer).
3. Read the file until the buffer is full or an EOF is encountered.
4. Print the content.
5. Close the file handle.

Now about writing, surely you could do something like the following to achieve the simple goal of writing to a file.

But there’s a catch. See how we have a predefined byte slice set up that we’re writing? Yeah, you need the entire file’s contents in memory in order to write it this way. This approach also breaks concurrency as it’s not possible to write from multiple goroutines to the same file at the same time, a much safer approach is to have one goroutine do all the write and use channels to output to that goroutine.

Let’s apply the Writer approach to do the same.

The Reader and Writer interfaces are the building blocks of all IO within Golang, lets dig into some additional helper functions, such as io.Copy:

io.Copy is fairly cool in what it does, when taking a quick look at its implementation it is revealed that it. Link, check the copyBuffer method.
1. creates a buffer of 32kb
2. reads either 32kb or until EOF from the source to the buffer
3. writes the buffer to the destination

It repeats these steps until the Copy method encounters an EOF. Thus very efficiently copying the file, without buffering all of it at once.

Interfaces are awesome! If you’re not using them in your every-day code, you’re not fully utilizing OOP. Because of interfaces io.Copy is source/destination agnostic.

Lets build a very simple TCP server that listens on a port and writes all connection input back to the connection output. net.Conn ( the connection object ) implements ReaderWriterCloser too! Thus, we can use known methods to write and read from the connection.

Go run the file, and do the following

Now type something and hit enter! If all is done correctly it should duplicate what you sent.

Checksumming and encrypting data works the same as writing a file too! Some ciphers feature a streaming interface, meaning the data will be encrypted on the fly, if the library doesn’t it’ll just transparently fill a buffer. Lets use this to generate the md5 checksum for a file.

Try changing the contents of the file and running the program again!

Compression is based around ReaderWriters too. Lets try to compress something with gzip.

You now know how to use readers and writers, go build some performant servers!

the packages ioutil, bufio and strings has some additional functions to make working with readers, writers and buffers more pleasant. Check them out too!

Leave a Reply