It almost certainly is not. Until we know what the useful life of NVIDIA GPUs are, then it's impossible to determine whether this is profitable or not.
The depreciation schedule isn't as big a factor as you'd think.
The marginal cost of an API call is small relative to what users pay, and utilization rates at scale are pretty high. You don't need perfect certainty about GPU lifespan to see that the spread between cost-per-token and revenue-per-token leaves a lot of room.
And datacenter GPUs have been running inference workloads for years now, so companies have a good idea of rates of failure and obsolescence. They're not throwing away two-year-old chips.
> The marginal cost of an API call is small relative to what users pay, and utilization rates at scale are pretty high.
How do you know this?
> You don't need perfect certainty about GPU lifespan to see that the spread between cost-per-token and revenue-per-token leaves a lot of room.
You can't even speculate this spread without knowing even a rough idea of cost-per-token. Currently, it's total paper math on what the cost-per-token is.
> And datacenter GPUs have been running inference workloads for years now,
And inference resource intensity is a moving target. If a new model comes out that requires 2x the amount of resources now.
> They're not throwing away two-year-old chips.
Maybe, but they'll be replaced by either (a) a higher performance GPU that can deliver the same results with less energy, less physical density, and less cooling or (b) the extended support costs becomes financially untenable.
> Right now models have roughly all of the written knowledge available to mankind, minus some obscure held out private archives and so on.
Sorry for the nit, but this is a gross oversimplification. Most private archives are not obscure but obfuscated and largely are way more valuable training data then the publicly available ones.
Want to know how the DOD may technically tracks your phone? Private.
Want to know how to make Coca Cola at scale? Private.
Want to know what the schematic is for a Google TPU? Private.
> If you think otherwise, you're in a filter bubble.
Belittling. Excellent way to get your point across.
> They actively recruit on elite engineering campuses over it. It is super fucking interesting work and candidates compete for the opportunity to do it.
This seems like it should be an easy thing to verify with some sort of reference. This is exactly what the parent comment is suggesting and you still flippantly are avoiding it as "trust me bro". I actually believe you, so why don't you share some evidence then?
I don't really care in this instance, because the information I'm relaying here is obviously, verifiably correct. It's not like, a persuasion challenge for me here. I'm just relating basic facts.
If you have something to say, I think you should say it. This is vacuous. I understand why it's unpopular to relate the fact that applications for serious surveillance/espionage CNE work at NSA are competitive, and that the USG is very open about soliciting those applications. It remains a fact. Feel free to challenge me on that; we can go deeper.
What I'm not going to do is write you an apology for confronting you with that information.
> I understand why it's unpopular to relate the fact that applications for serious surveillance/espionage CNE work at NSA are competitive, and that the USG is very open about soliciting those applications.
And I'm not challenging you on that, at all. I actually believe you are correct because you typically provide very thoughtful answers from a position of authority and usually bring evidence to back your assertions. In this case, you're not. Your comments are childish and makes your position way less believable and hence why I'm pushing you.
> It remains a fact.
Just because you say it's a fact doesn't mean it is. The fact that you've done nothing but say "I know its a fact so therefore it is" doesn't help your position either.
> What I'm not going to do is write you an apology for confronting you with that information.
Yeesh. Chill out. I never asked for an apology. Your discourse is belittling and unproductive.
From a purely technical view, skills are just an automated way to introduce user and system prompt stuffing into the context right? Not to belittle this, but rather that seems like a way of reducing the need for AI wrapper apps since most AI wrappers just do systematic user and system prompt stuffing + potentially RAG + potentially MCP.
Oracle's growth and value is in SaaS apps (NetSuite) and their cloud offering, not DB licensing. The economic impact of enterprises moving off Oracle DB is massively overstated here.
It's not hard to sell $10 worth of products if you spend $20. profit is more important than revenue.
reply