That's a good way of putting it. In my experience so far everyone is very excited to build LLM-enabled apps but outside of some customer service deployments (mostly at the megascale enterprise level), I haven't seen many actually successful efforts reported. Even the customer service bots always end up roundly mocked on social media once someone figures out they're talking to an LLM and tells it to pretend it's a fairy princess.
the key is to be able to traverse the abstraction hierarchy all the way from the physics of the hardware to the end-user, and that arguably is what any engineer must learn.
Joel Spolsky had a great article on leaky abstractions.
LLMs for code are leaky abstractions. They work many-a-time. But when they break, good luck fixing it.
yann-lecunn also put it well. If something works 95% of the time, and you compose it 10 times, it only works 59% of the time.
In the real world of software engineering, we cannot build on something that works 95% of the time reliably. And LLM apologists will immediately say that code written by humans has bugs too. Of course it does.
ie, things we construct by the computer are deterministic. the turing machine (and other equivalent models like the lambda calculus etc) being the canonical machine that models our computations. Arguably, all human knowledge is symbolic and determistic - even though it may model probabilistic phenomena.
> Hugely inconvenient and disrespectful of my time.
dealing with dealers and repair shops is not fun. but gas-cars are still the known-devil - masses understand their issues and are habituated to them.
evs come with unknowns which hinder fast mass adoption.
Is it true that luxury (or any) car manufacturers extend their oil-change periods so that the engine wears out earlier and their customers will replace their cars sooner. So, BMW wants you to replace your car every x miles (eg 100K miles) - whereas the mechanism can last way longer (300K miles) if maintained better.
the car maker has an inherent incentive to reduce the lifespan of the vehicle which conflicts with the customer's incentive to extend the lifespan.
this goes into the heart of what it means to "know".
All human knowledge is "symbolic". that is, knowledge is a set of abstractions (concepts) along with relations between concepts. As an example, by "knowing" addition is to understand the "algorithm" or operations involved in adding two numbers. reasoning is the act of traversing concept chains.
LLMs dont yet operate at the symbolic level, and hence, it could be argued that they dont know anything. LLM is a modern sophist excelling at language but not at reasoning.
pytorch/tensorflow etc are becoming the "OS" for inference and training.
users interact with pytorch - not with hardware libraries. so, if pytorch can abstract the hardware, users wont care.
all users will care about is dollar cost of doing their work. so expect increasing commoditization of the hardware.
further, almost everyone in the ecosystem has an incentive to commoditize the hardware (users, cloud vendors, etc). over time i see the moat eroding - as the moat does not attach directly to the user.
This is still pretty idealized. In my experience the pytorch abstraction leaks _constantly_. In particular, any interesting ML project probably pulls in at least a few dependencies with custom pytorch extensions somewhere.
> users interact with pytorch - not with hardware libraries. so, if pytorch can abstract the hardware, users wont care.
At the most basic level, yes (pretty much "hello world"). This is what I meant by "it’s interesting to watch observers/casual users claim these implementations are competitive". Take a look at a project (nearly any project) and you will see plenty of specific commits for ROCm:
Check the dates - ROCm is six years old and all of these commits are /very/ recent.
Only the most simple projects are purely PyTorch to the point where other than random curiosities I'm not sure I've seen one in years.
Check the docs and pay attention to caveats everywhere for ROCm, with tables showing feature support for ROCm with asterisks all over the place. Repeat for nearly any project (check issues and pull requests while you're at it). Do the same for CUDA and you will see just how much specific hardware and underlying software work is required.
> all users will care about is dollar cost of doing their work.
With Nvidia having 90% market share in AI and 80% market share on desktop and being supported in torch since day one those ratios are way off. For now and the foreseeable future if you're a business (time isn't free) the total cost of an actual solution from getting running, to training, to actually doing inference (especially at high production scale) very heavily favors Nvidia/CUDA. I've worked in this space for years and at least once a month since the initial releases of ROCm on Vega in 2017 I check in on AMD/ROCm and can't believe how bad it is. I've spent many thousands of dollars on AMD hardware so that I can continually evaluate it - if ROCm were anywhere close to CUDA in terms of total cost I'd be deploying it. My AMD hardware just sits there, waiting over half a decade for ROCM to be practical.
I don't have some blind fielty to Nvidia, own any stock, or care what logo is stamped on the box. I'm just trying to get stuff done.
> further, almost everyone in the ecosystem has an incentive to commoditize the hardware (users, cloud vendors, etc). over time i see the moat eroding - as the moat does not attach directly to the user.
We're very much in agreement. Your key statement is "over time" and this is what I was referring to with 'I’m really rooting for them but the reality is these CUDA “competitors” have a very very long way to go.'. It's going to be a while...
those panning for gold (app devs, startups etc) may or may not find it. remains to be seen and i remain skeptical.