I appreciate your rabid optimism, but considering that Moores Law has ceased to be true for multiple years now I am not sure a handwave about being able to scale to infinity is a reasonable way to look at things. Plenty of things have slowed down in progress in our current age, for example airplanes.
Someone always crawls out of the woodwork to repeat this supposed "fact" which hasn't been true for the entire half-century it's been repeated. Jim Keller (designer of most of the great CPUs of the last couple decades) gave a convincing presentation several years ago about just how not-true it is: https://www.youtube.com/watch?v=oIG9ztQw2Gc Everything he says in it still applies today.
Intel struggled for a decade, and folks think that means Moore's law died. But TSMC and Samsung just kept iterating. And hopefully Intel's 18a process will see them back in the game.
During the 1990s (and for some years before and after) we got 'Dennard scaling'. The frequency of processors tended to increase exponentially, too, and featured prominently in advertising and branding.
I suspect many people conflated Dennard scaling with Moore's law and the demise of Dennard scaling is what contributes to the popular imagination that Moore's law is dead: frequencies of processors have essentially stagnated.
Yup. Since then we've seen scaling primarily in transistor count, though clock speed has increased slowly as well. Increased transistor count has led to increasingly complex and capable instruction decode, branch prediction, out of order execution, larger caches, and wider execution pipelines in attempt to increase single-threaded performance. We've also seen the rise of embarrassingly parallel architectures like GPUs which more effectively make use of additional transistors despite lower clock speeds. But Moore's been with us the whole time.
Chiplets and advanced packaging are the latest techniques improving scaling and yield keeping Moore alive. As well as continued innovation in transistor design, light sources, computational inverse lithography, and wafer scale designs like Cerebras.
Yes. Increase in transistor count is what the original Moore's law was about. But during the golden age of Dennard scaling it was easy to get confused.
Agreed. And specifically Moore's law is about transistors per constant dollar. Because even in his time, spending enough could get you scaling beyond what was readily commercially available. Even if transistor count had stagnated, there is still a massive improvement from the $4,000 386sx Dad somehow convinced Mom to greenlight in the late 80s compared to a $45 Raspberry Pi today. And that factors into the equation as well.
Of course, feature size (and thus chip size) and cost are intimately related (wafers are a relatively fixed cost). And related as well to production quantity and yield (equipment and labor costs divide across all chips produced). That the whole thing continues scaling is non-obvious, a real insight, and tantamount to a modern miracle. Thanks to the hard work and effort of many talented people.
The way I remember it, it was about the transistor count in the commercially available chip with the lowest per transistor cost. Not transistor count per constant dollar.
Wikipedia quotes it as:
> The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years.
But I'm fairly sure, if you graph how many transistors you can buy per inflation adjusted dollar, you get a very similar graph.
Yes. I think you're probably right about phrasing. And transistor count per inflation adjusted dollar is the unit most commonly used to graph it. Similar ways to say the same thing.
LoAR shows remarkably steady improvement. It's not about space or power efficiency, just ops per $1000, so transistor counts served as a very good proxy for a long time.
There's been sufficiently predictable progress that 80-100 TFLOPS in your pocket by 3035 is probably a solid bet, especially if a fully generative AI OS and platform catches on as a product. The LoAR frontier for compute in 2035 is going to be more advanced than the limits of prosumer/flagship handheld products like phones, so theres a bit of lag and variability.
I think banning or severely limiting advertising similar to cigarettes would be a good start. Stop having sports broadcasts be so intertwined with gambling, seeing odds on the screen when watching sports is gross.
Sam has claimed that they are profitable on inference. Maybe he is lying but I don't think speaking so absolutely about them losing money on that is something you can throw around so matter of fact. They lose money because they dump an enormous amount of money on R&D.
I don't think you can confidently say how it will pan out. Maybe OpenAI is only unprofitable at the 200/month tier because those users are using 20x more compute than the 20/month users. OpenAI claims that they would be profitable if they weren't spending on R&D [1], so they clearly can't be hemorrhaging money that badly on the service side if you take that statement as truthful.
"OpenAI claims that they would be profitable if they weren't spending on R&D "
Ermmm dude they are competing with Google. They have to keep reinvesting otherwise Google captures the users OAI currently has.
Free cash flows matter. Not accounting earnings. On a FCFF basis they largely in the red. Which means they have to keep raising money, at some point somebody will turn around and ask the difficult questions. This cannot go on forever.
And before someone mentions Amazon... Amazon raised enough money to sustain their reinvestment before they eventually got to the place where their EBIT(1-t) was greater than reinvestment.
Stealing a catalytic converter to sell for money cannot be equivocated to shoplifting. Plenty of shoplifters are doing it for the thrill or to obtain things that they wouldn't pay for, no one is doing that with cats, they are doing it to try and survive.
There was serious money in catalytic converter theft and an organized ring behind it raking in millions of dollars (up to $545 million) [0]. That’s not trying to survive. Since the arrest of the organizers of the ring, catalytic converter theft has fallen off significantly: without that criminal enterprise, catalytic converter theft ceased to be wildly lucrative. People who steal to survive steal essentials like food, not catalytic converters.
That’s true… BTC returned 100% per year since 2017 (thanks to ChatGPT for the math).
But I would presume a regular Joe didn’t YOLO $100k into that, either.
I guess my original point was: there is no realistic way a retail investor walked away from a $100k investment a multimillionaire after 8 years unless they took an enormous, absurd punt.
If you invested $100K in Microsoft (one of the safest and most valuable and boring company in the world at the time) in 2017 and held, you’d not be a multimillionaire, but you’d be a millionaire. People really underestimate how crazy the asset bubble has been over the last decade.
USD 100k is an absurd amount for most people to invest all at once in anything whatsoever, not to mention the risk level involved. USD 10k is a lot but more reasonable. And that nets way less than USD 100k in this situation, which isn't life changing.
As a (former) huge tesla fan/shareholder, and current model 3 enjoyer, the cybertruck really makes me upset. The millions of dollars and engineering hours devoted to that thing that could have been devoted to a new product that people actually want and use (or even a halo product like the forgotten new roadster) was and is incredibly wasteful.
The cybertruck is the full vehicle version of the mistake Musk made by pushing for the falcon-wing doors on Model X. He said that was a huge mistake, and nothing like that would ever happen again.
I would counter your charging anecdote almost entirely with my anecdote. I have never had to worry about arriving at a destination without enough to make it to another supercharger, that worked great in 2022 when I got my model 3 and works great today. I have never had so much as a single issue with a supercharger not working or being out of service and not saying it on the car screen. I don't have any rebuttals for your other points: winter range stinks, and tires wear faster than gas equivalents. EVs really need a battery innovation to add another 100 estimated miles to really push them into the mainstream imo. If we can get to the 450ish range that would help a lot.
The range thing is weird, because it’s simultaneously too little for your use case, and too much for mine. What I want is a shorter-range (like 200 km/125 miles would be perfect, it gives enough buffer by being about double the max range I’d ever need in practice) that saves on upfront cost and weight by having a smaller battery that can be my household’s secondary city car. But as-is, there’s no EV that’s cheap enough to justify buying as a cheap, low-range city car.
So instead of my household having one EV and one gas car, we’ve got two gas cars.
I dunno, like it might be the best option if I was set on an EV, but it feels like it has a bigger battery than necessary rather than just having a smaller battery, including a heat pump on the base trim, and not air-cooling the battery. (And chademo is pretty unappealing, and lack of AWD is a downer.)
All-in-all, it feels like a lot of compromises compared to our current second car. (A 2019 Impreza hatch.)
As someone from Buffalo it is (anecdotally) pretty common for Canadians to come here to fly to other parts of the US for cheaper, and for us to go to Toronto for cheaper international flights.
I just spent a few minutes on Google Flights looking at trips to various US holiday destinations like Miami, Las Vegas, Honolulu and Los Angeles. Every single destination was a lot cheaper (in some cases less than half price) when flying out Toronto compared to Buffalo. I wonder is a reflection on the collapse of demand for flights to the US.
