Daniel Lemire's blog

Compressed bitset libraries in C and C++

, 1 min read

The bitset data structure is a clever way to represent efficiently sets of integers. It supports fast set operations such as union, difference, intersection. For better scalability, we compress bitsets. Bitsets are not always the right data structure, but when they are applicable, they work…

Science and Technology links (March 30, 2017)

, 14 min read

A famous and highly-funded researcher from Cornel, Brian Wansink, has published many studies regarding how we eat. His work has guided public policy all over the word. According to his Wikipedia entry, Barak Obama named him “Executive Director of the USDA’s Center for Nutrition Policy and…

Never reason from averages

, 3 min read

StackOverflow published its list of “top paying technologies“. Worldwide, the best-paid programmers, on average, work in Clojure and Rust (these are programming languages in case you are wondering). In the US, the top paying programmers work in Go and Scala. In the UK, it is TypeScript and…

The technology of Mass Effect Andromeda

, 4 min read

Mass Effect Andromeda is the long-awaited sequel to the popular Mass Effect video game series. It is available to a game console near you. It was given a rough time by the critics and many users. The game is set in the far future of humanity. Human beings are starting to colonize a new galaxy. I…

Science and Technology links (March 24, 2017)

, 14 min read

There are many claims that innovation is slowing down. In the XXth century, we went from horses to planes. What have we done lately? We have not cured cancer or old age. We did get the iPhone. There is that. But so what? There are many claims that Moore’s law, the observation that processors gets…

Does software performance still matter?

, 9 min read

This morning, a reader asked me about the real-world relevance of software performance: I’m quite interested in your work on improving algorithm performance using techniques related to computer architecture. However, I think that this may only be of interest to academia. Do you think that there…

Science and Technology links (March 17, 2017)

, 16 min read

We live in a world where the most powerful companies in the world have super smart people working on trying to emulate human intelligence in machines. Yann LeCun, a Frenchman who directs Facebook’s artificial intelligence research group, tells us that, soon, computers could acquire what he calls…

Stable Priority Queues?

, 2 min read

A priority queue is a data structure that holds a set of elements and can return quickly the smallest (or alternatively the largest) element. It is usually implemented using a binary heap. So you “add” elements to the priority queue, and then you can “poll” them out. Suppose however that…

Science and Technology links (March 10, 2017)

, 2 min read

In Mnemonic Training Reshapes Brain Networks to Support Superior Memory (published in Neuron, March 2017), we learned that 6 weeks of mnemonic training at a rate of 30 minutes a day lead to a large scale brain network re-organization making the brain of control subjects more like that of memory…

College and inequality

, 5 min read

Most college professors are squarely on the left ideologically. They believe that part of their mandate is to reduce inequality, by helping to provide college degrees to all who qualify. This always felt as strange to me. Higher education is highly subsidized, but the money goes overwhelmingly to…

The technology of Logan (2017 Wolverine movie set in 2029)

, 2 min read

Last night I went to see Logan, the latest and maybe the last Wolverine movie with Hugh Jackman. The movie is set in 2029. The year 2029 is an interesting choice, as it is the year of the story “Ghost in the shell”, made into a movie featuring Scarlett Johansson and coming out later this year.…

How many floating-point numbers are in the interval [0,1]?

, 4 min read

Most commodity processors support single-precision IEEE 754 floating-point numbers. Though they are ubiquitous, they are often misunderstood. One of my readers left a comment suggesting that picking an integer in [0,232) at random and dividing it by 232, was equivalent to picking a number at random…

Tech jobs are already largely automated

, 1 min read

Lately, I have been reading a lot about the threat to computer jobs from automation. For example, we have AI systems that can write their own code. And billionaire Mark Cuban predicts that “software will soon begin writing itself, which will ultimately eliminate those lucrative software…

Thoughts on my new laptop (Dell XPS 13 with Windows 10)

, 5 min read

I love computers. Unlike many people, who stick to one brand and one operating system, I like to use many different systems. I own several game consoles, several types of tablets and so forth. My primary laptop for many years has been an inexpensive and light MacBook Air. It is nearly perfect. I…

How fast can you count lines?

, 2 min read

Inspired by earlier work by Llogiq, I decided to look at how fast I could count the number of lines in a string. By assuming that the string relies on ASCII, UTF-8 or other ASCII superset character encoding, the bulk of the work consists in counting the linefeed (‘\n’) characters. Most…

Sorting sorted arrays in Swift

, 3 min read

Sorting arrays quickly is a classical computer science problem. It is also a common task worth optimizing. Sadly, there is no best approach, no silver bullet. Most people rely on whatever their programming language provides as a sorting function, but it is almost always suboptimal. The new Swift…

Montreal researchers “prove” that aging is the result of a genetic program

, 4 min read

Unless you are a tree, a lobster, or some other sea creature, you are probably aging… which is another way of saying that beyond adulthood, your fitness decreases while your mortality risk increases over time. Though some pretend that aging is a well-understood process… from a scientific point…

Maps and sets can have quadratic-time performance

, 5 min read

Swift is a new programming language launched by Apple slightly over two years ago. Like C and C++, it offers ahead-of-time compilation to native code but with many new modern features. It is available on Linux and macOS. Like C++, Swift comes complete with its own data structures like dictionaries…