I was recently contemplating on whether I should use Java for algorithmic programming and I came to a conclusion. Java is not good. I mean it is great if you consider the number of libraries available. The performance is generally good. Yet, if you need top-notch performance, it is quite hard to get in Java. You would need to use awful things like manual memory management and parallel arrays…. This is awful and counterproductive. You can do much better in C++. But, if you don’t mind 2-3x loss in performance Java (Scala, or, say, OCaml) can be a nice choice.
Anonymoussays:
“The assumption is that most (but not all) values in your array use less than 32 bits.”
I think this statement belongs in this post, otherwise nothing makes sense.
Thanks for pointing out a shortcoming in my post. However, we do not make this assumption. Instead, we just assume that the integers have been sorted. I have edited the post accordingly.
So, yes, I would say that you can building streaming applications using this library.
anonymoussays:
Hi,
*noob here
I want to ask you for advice about compressing integers. I have integers in range 0-255 so each of them is 8-bit long. Which compresor schould I use? JavaFastPFOR is only for “in memory” compress right?
I was recently contemplating on whether I should use Java for algorithmic programming and I came to a conclusion. Java is not good. I mean it is great if you consider the number of libraries available. The performance is generally good. Yet, if you need top-notch performance, it is quite hard to get in Java. You would need to use awful things like manual memory management and parallel arrays…. This is awful and counterproductive. You can do much better in C++. But, if you don’t mind 2-3x loss in performance Java (Scala, or, say, OCaml) can be a nice choice.
“The assumption is that most (but not all) values in your array use less than 32 bits.”
I think this statement belongs in this post, otherwise nothing makes sense.
@Anonymous
Thanks for pointing out a shortcoming in my post. However, we do not make this assumption. Instead, we just assume that the integers have been sorted. I have edited the post accordingly.
@Itman
What is true is that there is a whole range of optimizations that are simply not possible in Java (by design).
Does it support streaming compression as integers keep appearing in the stream?
@Ishan
You can compress and uncompress data in blocks… See the “advanced” example in the example.java file:
https://github.com/lemire/JavaFastPFOR/blob/master/example.java
So, yes, I would say that you can building streaming applications using this library.
Hi,
*noob here
I want to ask you for advice about compressing integers. I have integers in range 0-255 so each of them is 8-bit long. Which compresor schould I use? JavaFastPFOR is only for “in memory” compress right?
The best strategy is to try various codecs and see which one works best for your data. There is no need to guess.
The library is low-level and won’t handle issues such as writing data to disk.