Daniel Lemire's blog

Are universities an egalitarian force?

, 3 min read

Many people are concerned about economic inequality. Many of these people work on university campuses. My experience is that the vast majority of the university professors, including those working at prestigious schools, believe that they are working toward a more egalitarian society. I have…

Science and Technology links (November 3rd, 2017)

, 5 min read

A paper in the journal Intelligence reports that people with high intelligence are more at risk for psychological problems. As someone who works with lots of very smart people, I am not surprised. Atherosclerosis is a medical condition, common with age, that makes you more at risk for heart…

The dual-shotgun theorem of software engineering

, 5 min read

There is a long-standing problem in software engineering: how does one recognize good code? It would be great if you could point a tool at a software base and get an objective measure of how good the code is. There are tools that pretend to do this, there are textbooks that pretend to guide you…

A decade of using text-mining for citation function classification

, 3 min read

Academic work is typically filled with references to previous work. Unfortunately, most of these references have, at best, a tangential relevance. Thus you cannot trust that a paper that cites another actually “builds on it”. A more likely scenario is that the authors of the latest paper did…

Fast integer compression with Stream VByte on ARM Neon processors

, 2 min read

Stream VByte is possibly the fastest byte-oriented integer compression scheme. I presented it briefly last month when our paper came out. Our C library has been ported to Rust and Go. Our code is used by the Tantivy search engine as well as by the Trinity Information Retrieval framework. Mark…

Science and Technology links (October 27th, 2017)

, 3 min read

A well-known tech company, Snapchat, has posted some art pieces throughout the world as augmented reality artifacts. You can only see the art through their software using mobile devices. Older people frequently become frail. They lose so much muscle mass that they have difficulties moving around.…

Science and Technology links (October 20th, 2017)

, 7 min read

John Carmack is a famous game designer. In a recent interview, he invited programmers to shy away from trying to build more realistic games in virtual reality, because he expects that hardware capabilities will not keep up. Fraction of Americans who are obese: 36.5%. And it keeps on rising. Brian…

Why virtual reality (VR) might matter more than you think….

, 6 min read

I have heard it claimed that the famous novelist William Gibson uttered his famous quote, “the future is already here “it’s just not very evenly distributed”, for the first time after experiencing virtual reality, decades ago. We are fast arriving at a point where virtual-reality will be…

The Harvey-Weinstein scientific model

, 2 min read

It is widely believed that science is the process by which experts collectively decide on the truth and post it up in “peer-reviewed journals”. At that point, once you have “peer-reviewed research articles” then the truth is known. Less naïve people raise the bar somewhat. They are aware…

Bee-level intelligence

, 2 min read

How close are we to having software that can emulate human intelligence? It is hard to tell. One problem with human beings is that we have large brains, with an almost uncountable number of synapses. We have about 86 billion neurons. This does not seem far from the 4.3 billion transistors that can…

Science and Technology links (October 13th, 2017)

, 1 min read

Rodney Brooks, who commercialized robots that can vacuum your apartment, has written a great essay on artificial intelligence. It is worth reading. There is some concern that the computers necessary to control a self-driving car will use so much power that they will significantly increase the…

Post-Blade-Runner trauma: From Deep Learning to SQL and back

, 9 min read

Just after posting my review of the movie Blade Runner 2049, I went to attend the Montreal Deep Learning summit. Deep Learning is this “new” artificial-intelligence paradigm that has taken the software industry by storm. Everything, image recognition, voice recognition, and even translation,…

On Blade Runner 2049

, 2 min read

Back in 1982, an incredible movie came out, Blade Runner. It told the story of artificial human beings (replicants) that could pass as human beings, but had to be hunted down. The movie was derived from a novel by Philip Dick. It took many years for people to get Blade Runner. The esthetic of the…

Science and Technology links (October 6th, 2017)

, 6 min read

In 2011, Bollen et al. published a paper with powerful claims entitled Twitter Mood Predicts the Stock Market. The paper has generated a whole academic industry. It has been cited 3000 times, lead to the creation of workshops… and so forth. Lachanski and Pav recently tried to reproduced the…

My iPad Pro experiment

, 5 min read

Years ago, I placed a bet with Greg Linden, predicting that tablets like the iPad would replace PCs. The bet did not end well for me. My own analysis is that I lost the bet primarily because I failed to foresee the market surge of expensive and (relatively) large smartphones. Though I lost to Greg,…

Stream VByte: first independent assessment

, 1 min read

In an earlier post, I announced Stream VByte, claiming that it was very fast. Our paper was peer reviewed (Information Processing Letters) and we shared our code. Still, as Feynman said, science is the belief in the ignorance of experts. It is not because I am an expert that you should trust…