Daniel Lemire's blog

, 11 min read

Science and Technology links (July 20th 2019)

10 thoughts on “Science and Technology links (July 20th 2019)”

  1. Full finding:

    There have been marked changes in temperature extremes across the contiguous United States. The frequency of cold waves has decreased since the early 1900s, and the frequency of heat waves has increased since the mid-1960s. The Dust Bowl era of the 1930s remains the peak period for extreme heat. The number of high temperature records set in the past two decades far exceeds the number of low temperature records. (Very high confidence)

    p.s. the auto-posts you are sending to Mastodon are sending repeated code-exposed posts. You might want to have a look.

    1. Thanks. The mastodon plugin is clearly broken. 🙁

    2. As for the content… the important point here is that people who claim that we have heat waves like we have never seen before are factually wrong, at least in the US.

      What is true is that we have fewer extreme cold weather events.

      1. To the extent that they are wrong at all, they are wrong in a very narrowly defined context.

        people alive today have generally never seen such high temperatures
        most regions of the world (and even most regions in the US) have never seen such high temperatures
        the term ‘heat wave’ is defined with respect to background (average) temperatures, which are higher today than in the 1930s (ie., the dust bowl was more extreme because it was contrasted with a lower baseline temperature).

        But even more to the point, why would you focus on this one narrow area where people may be wrong, rather than on the preponderance of the evidence, which shows increasingly high temperatures (including heat waves) world wide?

        1. Journalists routinely misinform by drawing causal relationships between the current weather (“we have a heat wave”, “we have really cold weather”) and climate change which… in our lifetime is likely impossible for any one human being to observe, personally… and the human-level observations that are possible have mostly to do with fewer extreme cold events, at least in North America.

          Look at this NYT articles that appeared recently: Heat Waves in the Age of Climate Change: Longer, More Frequent and More Dangerous (find link via Google). If you read the article, they conveniently choose 1960 as a reference point. This is flat out misleading as 1960 is a global minimum in the last 100 years.

          And, of course, it is selective. People never mention that the Winter of 2013-2014 was one of the coldest on record.

          This is terrible science.

          Repeat after me: you cannot take the daily weather and is that to draw conclusions about the climate. To study the climate, you need many measures, lots of care to avoid measurement biases.

          1. Firstly, it is completely fair to say that heat waves in the era of climate change are more dangerous.

            Since (as noted by others) a heat wave is denoted by variation from the average, and the average is going up, any period that can be called a heat wave is likely to be hotter, and therefore more dangerous then hitherto.

            More numerous and longer? Well – if instead of choosing the upper 10% measure used by these climate scientists, we choose a fixed temperature above which we now call “heat wave” (a measure that non-scientists might find makes more sense), we will probably find our new “heat waves” are both more common and longer lasting.

            I find the climate paper presented is itself careful to both downplay the warming aspects (by the method chosen to define heat wave as an example), and yet slip in significant alarm throughout. It is almost as if a group of concerned scientists were asked to present their findings in such a way as to minimize a particular (and dramatic) conclusion, and yet were trying to find a way to point to that conclusion without directly saying it out loud.

            I would suggest that we take the average temperature of the entire period in question, and determine a single “heat wave” number for the entire period. Then we can measure heat waves historically with an apples to apples comparison in terms of human experience.

            1. Since (as noted by others) a heat wave is denoted by variation from the average, and the average is going up, any period that can be called a heat wave is likely to be hotter, and therefore more dangerous then hitherto.

              Heat waves are defined as series of consecutive days with temperatures above a reference threshold. The threshold is set based on a fixed reference period that does not change. You can measure both the intensity and the duration of the heat waves.

              I would suggest that we take the average temperature of the entire period in question, and determine a single “heat wave” number for the entire period. Then we can measure heat waves historically with an apples to apples comparison in terms of human experience.

              That’s what they do.

              The result is that heat waves have slightly increased in density and frequency from the 1960s, but they are both far lower than in the early part of the XXth century.

  2. I found a phrase in the report which highlights the issue, making my point and basically agreeing with the media analysis.

    There are large projected changes in the number of days exceeding key temperature thresholds throughout the contiguous United States. For instance, there are about 20–30 more days per year with a maximum over 90°F (32°C) in most areas by mid-century under RCP8.5, with increases of 40–50 days in much of the Southeast (Figure 6.9). The upper bound for projected changes is very roughly 10 days greater than the weighted multimodel mean.

  3. Maynard Handley says:

    Superficially I’m unimpressed with the recommender paper. Clearly NN training relies on huge quality datasets. I don’t know what these would be in the recommender context. The paper mentions the three training datasets used, but says nothing about their size or quality, or how one measures (and thus trains an NN) the quality of a recommendation.

    To me this doesn’t prove NNs can’t form the basis of a quality recommender so much as it shows that people are throwing NNs as problems in a silly fashion. I could believe that NNs in the hands of an entity with both the data and the will to solve the problem could do well. but the incentives have to line up…
    To the extent that eg, Netflix’ incentives seem to be “keep people on the site” rather than “delight them”, it’s not even obvious that many commercial organizations (YouTube, Facebook, Netflix, …) will optimize for good recommendations. But all of these are outside factors, they don’t (IMHO) show that NNs cannot perform this particular task.

  4. To me this doesn’t prove NNs can’t form the basis of a quality recommender so much as it shows that people are throwing NNs as problems in a silly fashion.

    That’s my understanding as well.

    To the extent that eg, Netflix’ incentives seem to be “keep people on the site” rather than “delight them”, it’s not even obvious that many commercial organizations

    My understanding of the approach that Netflix and other services take is that they are not very interested on overfitting to static datasets that may or may not be representative of actual user satisfaction in a real-world system.