Re: claim 1: Actually, Dennard scaling stopped working circa 2006 and reducing feature sizes no longer reduces power consumption. We are now well into the era of power-constrained (rather than transistor-constrained) silicon design.
It had a good 30-year run (1970s to 2000s), but it’s very much stopped now.
The main problem is subthreshold leakage. Smaller transistors have thinner gate regions (smaller gate regions switch faster, a major benefit!), and quantum tunneling depends exponentially on the barrier thickness, even in a nominally impassible “off” transistor.
Modern ICs try to strike a balance between switching current and leakage current, but it’s not easy.
(The other problem is tolerances. As feature sizes shrink, manufacturing variation becomes proportionally larger, making everything more difficult.)
I do not think I invoked Dennard scaling. High resolution processes have usually translated into more speed and reduced power usage. If you want to argue against that statement, you have to argue that from process to process, a vendor such as TSMC has not, usually, reduced the power usage (for a given processing speed).
George Spelvinsays:
If you want to argue against that statement, you have to argue that from process to process, a vendor such as TSMC has not, usually, reduced the power usage (for a given processing speed).
More specifically, the statement “Finer resolutions usually translate into lower energy usage and lower heat production.” implies a causal connection, which hasn’t been true for a long time.
Rather, contemporary process engineers have to find a separate way to reduce power consumption lest their super-small transistors be rendered useless.
The most obvious one in recent years has been FinFETs. And yes, FinFETs are normally sold in a bundle with geometries below a certain size in the same way that caches are included with processors above a certain clock speed, because it would be pointless otherwise.
But that’s not because of the finer resolution; it’s more “in spite of”.
As a crude software analogy, we all know perfectly well that the correlation between MHz and performance has been broken for some time.
In the same way, smaller transistors haven’t caused lower power for quite a few years.
As the articles linked above describe, these days power dissipation per active transistor has plateaued, and designers wishing to use more transistors have to find ways to keep the addition “dark”, i.e. non-switching. It’s straightforward to clock-gate unused cache banks, but other logic is trickier.
> As a crude software analogy, we all know perfectly well that the correlation between MHz and performance has been broken for some time.
I would still write that higher clock frequencies usually translate into higher performance.
> And that is exactly my claim.
Let us be precise. Your claim is "from process to process, a vendor such as TSMC has not, usually, reduced the power usage"? Are you sure you want to stand by that claim?
You keep citing Dennard scaling but it is not a term that I seem to have ever used on this blog. At least, a keyword search does not find anything: https://lemire.me/blog/?s=Dennard
I searched for what I might have written in Dennard scaling over the years using Google. I submit to you that it is not a term that I tend to use. Note that I have been blogging weekly for almost 20 years. I write a lot.
> But that’s not because of the finer resolution; it’s more “in spite of”.
The adoption of finer resolution processors usually translate into better energy efficiency (normalized by computational power). It is simply a long-running trend.
My post does not claim a causal link and it certainly does not invoke Dennard scaling (back in 2018 I alluded to the fact that it had run its course in a comment to another blog post). The phrase is finer resolutions usually translate into lower energy usage and lower heat production. That's not the wording I would have used if I meant to imply a causal relationship.
To disprove my statement, you have to plot power efficiency over time, process by process, and show that it is not improving. But, of course, it is improving.
Moore's law was not based on a "causal relationship" and yet it held for 30+ years.
I would say that Americans tend to be overweight. There is no causality between the fact that one is American and being overweight, but there is an association. Simply put, if you ask me who weight more, and I am only told that A is Japanese and B is American, I will vote B any day. But there are underweight Americans and obese Japaneses.
So it goes with processors… finer and finer processes tend to bring about power efficiency gains.
Maybe you thought that I was invoking Dennard scaling… but the evidence is strong that I have been aware for quite some time that it is not a factor.
We can reasonably discuss whether my implication will hold. Do you want to place a bet regarding the power efficiency of the next TSMC process? I bet that it will be at least 10% more power efficient.
Re: claim 1: Actually, Dennard scaling stopped working circa 2006 and reducing feature sizes no longer reduces power consumption. We are now well into the era of power-constrained (rather than transistor-constrained) silicon design.
It had a good 30-year run (1970s to 2000s), but it’s very much stopped now.
The main problem is subthreshold leakage. Smaller transistors have thinner gate regions (smaller gate regions switch faster, a major benefit!), and quantum tunneling depends exponentially on the barrier thickness, even in a nominally impassible “off” transistor.
Modern ICs try to strike a balance between switching current and leakage current, but it’s not easy.
(The other problem is tolerances. As feature sizes shrink, manufacturing variation becomes proportionally larger, making everything more difficult.)
I do not think I invoked Dennard scaling. High resolution processes have usually translated into more speed and reduced power usage. If you want to argue against that statement, you have to argue that from process to process, a vendor such as TSMC has not, usually, reduced the power usage (for a given processing speed).
And that is exactly my claim. This is documented all over the place, e.g. What’s Next for Moore’s Law? For Intel, III+V = 10nm QWFETs
or Power Challenges At 10nm And Below. (See also the link in the former to the coverage of the IEDM 2005 conference where the imminent demise of Dennard scaling was discussed in detail.)
More specifically, the statement “Finer resolutions usually translate into lower energy usage and lower heat production.” implies a causal connection, which hasn’t been true for a long time.
Rather, contemporary process engineers have to find a separate way to reduce power consumption lest their super-small transistors be rendered useless.
The most obvious one in recent years has been FinFETs. And yes, FinFETs are normally sold in a bundle with geometries below a certain size in the same way that caches are included with processors above a certain clock speed, because it would be pointless otherwise.
But that’s not because of the finer resolution; it’s more “in spite of”.
As a crude software analogy, we all know perfectly well that the correlation between MHz and performance has been broken for some time.
In the same way, smaller transistors haven’t caused lower power for quite a few years.
As the articles linked above describe, these days power dissipation per active transistor has plateaued, and designers wishing to use more transistors have to find ways to keep the addition “dark”, i.e. non-switching. It’s straightforward to clock-gate unused cache banks, but other logic is trickier.
> As a crude software analogy, we all know perfectly well that the correlation between MHz and performance has been broken for some time.
I would still write that higher clock frequencies usually translate into higher performance.
> And that is exactly my claim.
Let us be precise. Your claim is "from process to process, a vendor such as TSMC has not, usually, reduced the power usage"? Are you sure you want to stand by that claim?
You keep citing Dennard scaling but it is not a term that I seem to have ever used on this blog. At least, a keyword search does not find anything: https://lemire.me/blog/?s=Dennard
I have used it in comments at least once, however. In 2018, I wrote… "An idea like Dennard scaling runs its course. Then something else comes along." https://lemire.me/blog/2018/07/21/science-and-technology-links-july-21st-2018/
I searched for what I might have written in Dennard scaling over the years using Google. I submit to you that it is not a term that I tend to use. Note that I have been blogging weekly for almost 20 years. I write a lot.
> But that’s not because of the finer resolution; it’s more “in spite of”.
The adoption of finer resolution processors usually translate into better energy efficiency (normalized by computational power). It is simply a long-running trend.
My post does not claim a causal link and it certainly does not invoke Dennard scaling (back in 2018 I alluded to the fact that it had run its course in a comment to another blog post). The phrase is finer resolutions usually translate into lower energy usage and lower heat production. That's not the wording I would have used if I meant to imply a causal relationship.
To disprove my statement, you have to plot power efficiency over time, process by process, and show that it is not improving. But, of course, it is improving.
Moore's law was not based on a "causal relationship" and yet it held for 30+ years.
I would say that Americans tend to be overweight. There is no causality between the fact that one is American and being overweight, but there is an association. Simply put, if you ask me who weight more, and I am only told that A is Japanese and B is American, I will vote B any day. But there are underweight Americans and obese Japaneses.
So it goes with processors… finer and finer processes tend to bring about power efficiency gains.
Maybe you thought that I was invoking Dennard scaling… but the evidence is strong that I have been aware for quite some time that it is not a factor.
We can reasonably discuss whether my implication will hold. Do you want to place a bet regarding the power efficiency of the next TSMC process? I bet that it will be at least 10% more power efficient.