Daniel Lemire's blog

Parsing short hexadecimal strings efficiently

, 2 min read

It is common to represent binary data or numbers using the hexadecimal notation. Effectively, we use a base-16 representation where the first 10 digits are 0, 1, 2, 3, 5, 6, 7, 8, 9 and where the following digits are A, B, C, D, E, F, with the added complexity that we can use either lower or upper…

Science and Technology links (April 13th 2019)

, 3 min read

There is little evidence that digital screens are harmful to teenager’s mental health. If there is an effect, it is small. Cotton bags must be reused thousands of times before they match the environmental performance of plastic bags. Organic cotton bags are much worse than regular ones,…

Why are unrolled loops faster?

, 2 min read

A common optimization in software is to “unroll loops”. It is best explained with an example. Suppose that you want to compute the scalar product between two arrays: sum = 0; for (i = 0; i < length; i++) sum += x[i] * y[i]; An unrolled loop might look as follows: sum = 0; i =…

Science and Technology links (April 6th 2019)

, 3 min read

In a randomized trial where people reduced their caloric intake by 15% for two years, it was found that reducing calories slowed aging. This is well documented in animals, going all the way to worms and insects, but we now have some evidence that it applies to human being as well. Personnally I do…

Science and Technology links (March 30th 2019)

, 2 min read

As we age, we accumulate old and useless (senescent) cells. These cells should die, but they do not. Palmer et al. removed senescent cells in obese mice. They found that these mice were less diabetic and just generally healthier. That is, it appears that many of the health problems due to obesity…

Java is not a safe language

, 3 min read

The prime directive in programming is to write correct code. Some programming languages make it easy to achieve this objective. We can qualify these languages as ‘safe’. If you write in C++ without good tools, you are definitively in the ‘unsafe’ camp. The people working on the Rust…

Hasty comparison: Skylark (ARM) versus Skylake (Intel)

, 2 min read

In a previous post, I ran a benchmark on an ARM server and again on an Intel-based server. My purpose was to indicate that if one function is faster, even much faster, on one processor, you are not allowed to assume that it will also be faster on a vastly different processor. It wasn’t meant to…

Technological aging

, 3 min read

We are all familiar with biological aging. Roughly speaking, it is the loss of fitness that most animals undergo with time. At the present time, there is simply not much you can do against biological aging. You are just not going to win any gold medals in the Olympics at age 65. However, not all…

Science and Technology links (March 23rd 2019)

, 3 min read

Half of American households subscribe to “Amazon Prime”, a “club membership” for Amazon customers with monthly fees. And about half of these subscribes buy something from Amazon every week. If you are counting, this seems to imply that at least a quarter of all American households order…

The fastest conventional random number generator that can pass Big Crush?

, 3 min read

In software, we sometimes want to generate (pseudo-)random numbers. The general strategy is to have a state (e.g., a 64-bit integer) and modify it each time we want a new random number. From this state, we can derive a “random number”. How do you that you have generated something that can pass…

Don´t read your data from a straw

, 3 min read

It is common for binary data to be serialized to bytes. Data structures like indexes can be saved to disk or transmitted over the network in such a form. Many serialized data structures can be viewed as sets of ‘integer’ values. That is the case, for example, of a Roaring bitmap. We must then…

Science and Technology links (March 16th 2019)

, 4 min read

There is mounting evidence that clearing old cells (senescent cells) from old tissues has a rejuvenating effect. There are very few such cells in most cases, but they cause a disproportionate amount of problems. Anderson et al. find that this effect might even extend to our hearts… clearance…

Age and the productivity of professors

, 2 min read

Professors tend to earn more as they become older. Woolley believes that professor’s contributions take a sharp turn downward after a certain point. If this is correct and professors retire at an increasingly older age, we have a problem. What does the research says about this? We can look at…

Multiplying by the inverse is not the same as the division

, 1 min read

In school, we learn that the division is the same as multiplying the inverse (or reciprocal), that is x / y = x * (1/y). In software, this does not work as-is when working with integers since 1/y is not an integer when y is an integer (except for y = 1 or y = -1). However, computers have…

Science and Technology links (March 9th 2019)

, 2 min read

In 2007-2008, the recipient of a bone-marrow therapy (Timothy Brown) was cured of HIV. Another person was cured following a bone marrow transplant. And then yet another. The trick to it seems that some of us are naturally immune to the HIV virus. If we donate some of our bone marrow to an…

Should our kids use pencils or keyboards?

, 8 min read

In Montreal, most kids have to write on paper using a pencil. They have paper dictionaries. My kids have spent an enormous amount of time learning to write in cursive using pencils, and no time at all (in school) learning to touch type with a keyboard. I do have pens, that I use to take notes on…

Parsing JSON quickly: early comparisons in the wild

, 2 min read

JSON is arguably the standard data interchange format on the Internet. It is text-based and needs to be “parsed”: the input string needs to be transformed into a conveniently data structure. In some instances, when the data volume is large, ingesting JSON is a performance bottleneck. Last week,…

Science and Technology links (March 2nd, 2019)

, 4 min read

Shapiro in his book The New Childhood argues that we should stop worrying so much about how much time our kids spend on video games and on their phone. He calls for a pivot in how we view childhood: The procedural rhetoric of the classroom helped kids become accustomed to the organizational…

Science and Technology links (February 24th, 2019)

, 4 min read

Dooley interviews Barabasi (a famous researcher) on his book The Formula: The Universal Laws of Success. Here is an interesting quote: It is true that major breakthroughs are typically connected to young individual’s name, but that’s not because the older folks are not creative, but because…