13th September 2023, 1 min read Transcoding Unicode strings at crazy speeds with AVX-512 2 thoughts on “Transcoding Unicode strings at crazy speeds with AVX-512” KWillets says: September 14, 2023 at 10:34 pm I think I understand what you’re doing in expand_and_identify, you’re trying every alignment of 4 bytes and compressing the ones that coincide with initials? Daniel Lemire says: September 15, 2023 at 2:39 pm I see you have been reading the code. This was the original approach for UTF-8 to UTF-16 but it has been replaced with something more efficient.
I think I understand what you’re doing in expand_and_identify, you’re trying every alignment of 4 bytes and compressing the ones that coincide with initials?
I see you have been reading the code. This was the original approach for UTF-8 to UTF-16 but it has been replaced with something more efficient.