Daniel Lemire's blog

, 5 min read

Who is keeping an eye on the tech companies?

3 thoughts on “Who is keeping an eye on the tech companies?”

  1. Can “deep learning” (whatever that means) understand music?

    Music is for me a particularly interesting – and hard – problem.

    My tastes in music have never fit within a main group. I find the streaming services worthless, as they play a lot of crap I do not want to hear. My tastes do not fall in a large group.

    Are we in the middle of a transient?

    In the last decade, the web helped me find my neighborhood in the “long tail” of music. I found musicians I liked, bought music, and even hosted house concerts. The web made this all possible.

    In the present, “big” services are discovering “big” markets in music, in a somewhat automated fashion. But so far they are hopeless in the “long tail” market.

    Eventually, someone is going to find the right clever notion, and do much better with automated discovery for the “long tail” market.

    Will there someday be a streaming music service smart enough to meet my tastes? Will smart services eventually serve the long tail?

  2. Your comments treat Spotify as an “other” to be studied, so I thought I’d say hi. I work on recommendations at Spotify. I’ve been reading and enjoying your blog for a long enough time that I no longer remember how I originally discovered it. Maybe something to do with language design or the semantic web?

    Anyway, studying and monitoring are eminently justified precautions to be collectively taken against any large systems, especially ones that by corporate nature are never wholly transparent about how they operate.

    But if it’s any oblique consolation, I can report that Spotify, at least, is fairly self-aware. Daniel Ek isn’t 23 any more, and to get Spotify to this point over the last decade he’s hired a couple thousand other people both older and younger. We are vividly aware of the tendency for automated systems to reinforce ambient biases, because we measure the performance of our features against pretty much every other variable we can quantify. So when something works better for men than women, or for Brazilians than Germans, we know and care.

    Preston, I’d be curious to hear if Spotify is better at meeting your tastes now than it was whenever you last tried it. “My tastes do not fall in a large group”, you say. But Discover Weekly, Release Radar and Daily Mix, all of which are relatively new features in Spotify, definitely do not rely on your tastes falling into any mainstream groove. I have 1496 genres on everynoise.com, and that’s hardly a complete set yet, so maybe the things you like aren’t anywhere on that map. But maybe there are. Maybe the future you’re waiting for has already started arriving.

    1. Thanks Glenn for the great comment.

      Daniel Ek isn’t 23 any more, (…)

      Just so we are clear, I was not being dismissive of Daniel Ek or of his achievements.

      This was part of the setup of my post where I argued that things move fast, faster than we often realize. So fast that I don’t think most of us can comprehend what is going on.

      Ten years ago, Daniel Ek was a kid that was certainly being dismissed by the music industry. Today, he probably can get meetings with just about everyone in the music industry. Do you think that the music industry saw it coming? I think not.

      Do you think that the music industry sees what is coming in the next ten years? Or do we assume that things are going to look the same, except maybe that Spotify will have a few more songs and a better UI?

      Anyway, studying and monitoring are eminently justified precautions to be collectively taken against any large systems, especially ones that by corporate nature are never wholly transparent about how they operate.

      Right. I think most of us who have dabbled long enough in non-trivial software take this for granted, but it is a lot less obvious to many others.

      We are vividly aware of the tendency for automated systems to reinforce ambient biases, because we measure the performance of our features against pretty much every other variable we can quantify. So when something works better for men than women, or for Brazilians than Germans, we know and care.

      This makes me very happy. I am sure that the YouTube folks would say something similar.

      As you no doubt realize, this is good but probably not sufficient. For one thing, external monitoring tends to keep people honest. For another, without hard data, the rest of the world is working from a hopelessly incomplete picture… and that’s not great in a fast changing world.