Daniel Lemire's blog

Scientific productivity tips from Hartley and Branthwaite

, 1 min read

Hartley and Branthwaite (1989) have done a questionnaire study of productive psychologists. They make the following recommendation for best productivity: Make a rough plan; Complete sections at a time; Use a word processor if possible; Revise and redraft at least twice; Spend about 2–5 hours…

Cool software design insight #1

, 1 min read

I plan to progressively discuss a few things I have learned about software design during the rest of the year. Trivial things that make a big difference in your productivity. I do not claim that any of these insights will be novel in any way. As a college professor, I do not code full time.…

Some myths about online teaching

, 2 min read

Last year, I launched an online graduate course on Information Retrieval. This summer, I am preparing an online data warehousing course, my fourth online course. It will cover topics ranging from multidimensional indexing techniques, the MDX language, what data warehouses are, Mondrian, JPivot, and…

Coverage of the cuil search engine

, 1 min read

It seems that the Cuil search engine is getting reactions from almost everyone. On my blog roll, about ten people have commented on it. Here is my verdict: Cuil.com claims to have outdone Google as far as recall goes: Cuil searches more pages on the Web than anyone else—three times as many…

Updating your model as a researcher

, 2 min read

Doing research is hard work. Most people make their life easier by following a model. This model is made of a series of recipes used to carry forward research projects. There are a few reasons why a researcher may want to update his research model: you want to do research on a new topic: some of…

Encouraging diversity in science

, 2 min read

Science follows a conservative process. It takes a long time for a fact or a law to be accepted. Several scientists must verify and reproduce the same results before acceptance is granted. So goes the theory. In practice, science is not such a clean process. Routinely, facts and theories become…

Google makes me smarter

, 2 min read

I am a bit late to the show, but I would like to comment on Carr’s Is Google Making Us Stupid? Carr’s observation is simple: Once I was a scuba diver in the sea of words. Now I zip along the surface like a guy on a Jet Ski. Here are my thoughts: Quite often, as a teenager, I would read…

We need a more negative culture

, 2 min read

There is a strong bias in science, at least in Computer Science, toward positive results. For example, showing that algorithm A is better than algorithm B, will get you published. Reporting the opposite result is likely to get your paper rejected. One justification for the value of positive results…

Do you think because you write, or write because you think?

, 2 min read

I used to believe that the pressure to publish what you did in research was inherently bad. About four years ago or so, I started to change my mind. I now believe that the more you write, the more you think about the issues, and the more ideas you have. In short, productive researchers do not write…

A small graph-theory puzzle

, 1 min read

I like to think about graph theory problems these days. Here is one: What type of graph has minimal diameter for a given number of vertices, given an upper bound on the in-degree and another upper bound on the out-degree? I will give eternal fame (among the readership of this blog) to anyone who…

I still don´t have the multiplication tables memorized

, 1 min read

I read this on slashdot: I have a PhD in math, and I still don’t have the multiplication tables memorized Now I know I am not the only one! In other news, I still deduce my age from my birth date (takes me a minute or so each time); I was identified as having a learning disability when I…

Sorting 1 terabyte in 209 seconds

, 1 min read

Yahoo! managed to sort 10 billion 100-byte elements in 209 seconds. This was done in Java using Hadoop. As a basis for comparison, on a fast and recent Mac Pro, it takes 6000 seconds to sort a 2 GB text file using Unix file utilities. Yahoo!’s problem is 500 times larger, and they solve it 30…

Backing up your Mac on an external disk

, 1 min read

A couple of weeks ago, I needed to backup my MacBook Pro to an external disk (a firewire G-Drive) because my hard drive was failing. I started shopping for a good backup solution, but none of them had the following features: support for incremental backups: if a change is made, you only backup the…

Classifying research projects by depth

, 2 min read

Everything else being equal, picking the right problems is the key factor determining your success as a researcher (no matter how you define success). In a previous post, I proposed three categories of research problems: explain a previously unexplained observation;1. perfect an existing…

Good research: invent new problems or explain mysteries

, 1 min read

It is a lot of work to grind through a research project and get an interesting paper out of it. Mostly, you have to be patient enough and work everyday at it. If you follow a sane process, it is difficult to fail entirely. Picking the right research question is very important however: it is…

Lowly tasks you should do

, 1 min read

Many of my colleagues never mark assignments. I tend to mark papers on nearly a weekly basis. Why am I doing this? Because I believe that marking assignments is the best way to identify the weaknesses in my courses and learn from my students. Many researchers never implement their ideas. They let…

The Disadvantages of an Elite Education

, 1 min read

I just read a great essay by William Deresiewicz, an associate professor of English at Yale. His message is clear: ivy-league education is flawed. Here is the killer sentence: It’s no coincidence that our current president [Bush], the apotheosis of entitled mediocrity, went to Yale. Via…

Too much stress

, 1 min read

I suffer from exhaustion. In the last few weeks, I had to resign from a few hats I wore. I resigned as union treasurer and I resigned as chair of the IT M.Sc. degree. Today, I stood up my friend Yuhong on a lunch date. Too much to do, too little time. I have reached a breaking point.

Blogs make meetings feel dull

, 1 min read

I have always hated meetings. I prefer to work alone at my desk, with the occasional email. I realized recently that blogging makes meetings feel even worse. There are many types of information you will not get through traditional channels. Peter Turney’s latest post is one such example. He…