Hi @lemire Thanks for following up on this issue.
using R curl package you can specify headers to accept gzip content and curl will automatically download and unzip for you
"Accept-Encoding": "deflate, gzip"
I had to use it in my own package to support fast downloads in regions with low internet speed (developing countries). More importantly, using curl open the door to more speed enhancements including asynchronous downloads which cuts the download time even further especially if the dwonload server allows for concurrent downloads. I am not a network guru, but I think we can combine the goodies of curl and Rppsimdjson in a new package.
Dr. Shishir S. Urdhwareshesays:
Thanks.
And thanks also @Mohammed O.E Abdallah for suggesting simdjason library
Hi @lemire Thanks for following up on this issue.
using R curl package you can specify headers to accept gzip content and curl will automatically download and unzip for you
"Accept-Encoding": "deflate, gzip"
I had to use it in my own package to support fast downloads in regions with low internet speed (developing countries). More importantly, using curl open the door to more speed enhancements including asynchronous downloads which cuts the download time even further especially if the dwonload server allows for concurrent downloads. I am not a network guru, but I think we can combine the goodies of curl and Rppsimdjson in a new package.
Thanks.
And thanks also @Mohammed O.E Abdallah for suggesting simdjason library