19th May 2014, 1 min read

Decoding over 4 billion integers per second in C

Programmers routinely work with lists of integers. We recently showed how to compress such lists of integers close to their entropy, while being able to decompress billions of integers per second.

To ensure anyone could do it, we published the FastPFOR C++ library. We also published the JavaFastPFOR Java library and there is even a corresponding Go library.

However, I wanted to provide also a low-level C library that advanced programmers could embed deep in their own software without the inconvenience of a bulky C++ research library.

So we wrote the SIMDComp library. It is a minimalist C library. On a recent PC, it will decompress integers at over 4 billion integers per second: less than one CPU cycle per integer. We use a liberal open source license so it should be suitable for all your projects.

To test it out for yourself, grab a copy, and type “make example; ./example”.