Daniel Lemire's blog

Science and Technology links (December 22nd 2018)

, 2 min read

For equity reasons, many people advocate for double-blind peer review, meaning that the author does not know who the reviewer is, nor does the reviewer know who the author is. It is believed (despite little hard positive evidence and some contrary evidence) that this is surely benefificial to…

Fast Bounded Random Numbers on GPUs

, 2 min read

We often use random numbers in software in applications such as simulations or machine learning. Fast random number generators tend to produce integers in [0,232) or [0,264). Yet many applications require integers in a given interval, say the interval {0,1,2,3,4,5,6,7}. Thus standard libraries…

Sorting strings properly is stupidly hard

, 2 min read

Programming languages make it hard to sort arrays properly. Look at how JavaScript sorts arrays of integers: > v = [1,3,2,10] [ 1, 3, 2, 10 ] > v.sort() [ 1, 10, 2, 3 ] You need a magical incantation to get the right result: > v.sort((a,b)=>a-b) [ 1, 2, 3, 10 ] Though this bad default…

Science and Technology links (December 15th 2018)

, 6 min read

Academic excellence is not a strong predictor of career excellence. There is weak correlation between grades and job performance. Grant reviews the evidence in details in his New York Times piece.When recruiting research assistants, I look at grades as the last indicator. I find that imagination,…

Science and Technology links (December 8th 2018)

, 1 min read

The energy density of lithium-ion batteries doubled between 1995 and 2005 but only increased by about 15% between 2005 and 2015. It is estimated that there is relatively little further gains in energy density possible with lithium-ion batteries. However, our mobile devices typically consume far…

Asking the right question is more important than getting the right answer

, 4 min read

Schools train us to provide the right answers to predefined questions. Yet anyone with experience from the real world knows that, more often than not, the difficult part is to find the right question. To make a remarkable contribution, you need to start by asking the right question. I will go…

Science and Technology links (December 1st 2018)

, 1 min read

Autism affects about 1% of the population and four times as many males as females. In older highly educated people, drinking 2 cups of coffee a day is associated with a reduced mortality rate of 22%. (This does not mean that drinking coffee makes you less likely to die, but it might.) Amazon, the…

Quickly sampling from two arrays (C++ edition)

, 2 min read

Suppose that you are given two arrays. Maybe you have a list of cities from the USA and a list of cities from Europe. You want to generate a new list which mixes the two lists, taking a sample from one array (say 50%), and a sample from the other array (say 50%). So if you have 50 cities from the…

Science and Technology links (November 24th 2018)

, 4 min read

There is no association between birth order and personality traits: The results of both within- and between-family research designs revealed no consistent evidence of a link between birth order and the personality traits of extraversion, neuroticism, agreeableness, conscientiousness, and…

Science and Technology links (November 18th 2018)

, 1 min read

It seems that reducing your carbohydrate (sugar) intake might be a good way to lose weight: lowering dietary carbohydrate increased energy expenditure during weight loss maintenance. This metabolic effect may improve the success of obesity treatment, especially among those with high insulin…

Simple table size estimates and 128-bit numbers (Java Edition)

, 2 min read

Suppose that you are given a table. You know the number of rows, as well as how many distinct value each column has. For example, you know that there are two genders (in this particular table). Maybe there are 73 distinct age values. For a concrete example, take the standard Adult data set which is…

Memory-level parallelism: Intel Skylake versus Apple A12/A12X

, 3 min read

Modern processors execute instructions in parallel in many different ways: multi-core parallelism is just one of them. In particular, processor cores can have several outstanding memory access requests “in flight”. This is often described as “memory-level parallelism”. You can measure the…

Science and Technology links (November 10th, 2018)

, 2 min read

It already takes more energy to operate Bitcoin than to mine actual gold. Cryptocurrencies are responsible for millions of tons of CO2 emissions. (Source: Nature) “Half of countries have fertility rates below the replacement level, so if nothing happens the populations will decline in those…

Measuring the memory-level parallelism of a system using a small C++ program?

, 2 min read

Our processors can issue several memory requests at the same time. In a multicore processor, each core has an upper limit on the number of outstanding memory requests, which is reported to be 10 on recent Intel processors. In this sense, we would like to say that the level of memory-level…

Science and Technology links (November 3rd, 2018)

, 3 min read

Bitcoin, the cryptocurrency, could greatly accelerate climate change, should it succeed beyond its current speculative state. Crows can solve novel problems very quickly with tools they have never seen before. The new video game Red Dead Redemption 2 made $725 million in three days. Tesla, the…

Science and Technology links (October 28th, 2018)

, 4 min read

If you take kids born in the 1980s, who do you think did better, the rich kids or the poor kids? The answer might surprise you: The children from the poorest families ended up twice as well-off as their parents when they became adults. The children from the poorest families had the largest…

Is WebAssembly faster than JavaScript?

, 3 min read

Most programs running on web sites are written in JavaScript. There are still a few Java applets and other plugins hanging around, but they are considered obsolete at this point. While JavaScript is superbly fast, some people feel that we ought to do better. That’s where WebAssembly comes in. It…

Science and Technology links (October 20th, 2018)

, 4 min read

Should we stop eating meat to combat climate change? Maybe not. White and Hall worked out what happened if the US stopped using farm animals: The modeled system without animals only reduced total US greenhouse gas emissions by 2.6 percentage units. Compared with systems with animals, diets…

Validating UTF-8 bytes using only 0.45 cycles per byte (AVX edition)

, 2 min read

When receiving bytes from the network, we often assume that they are unicode strings, encoded using something called UTF-8. Sadly, not all streams of bytes are valid UTF-8. So we need to check the strings. It is probably a good idea to optimize this problem as much as possible. In earlier work, we…