Radiation would perhaps be better pictured with a light blue glow (Cherenkov radiation).
The green glow used in pop culture has its origin in the widespread use of radium paint to achieve a glow-in-the-dark effect (e.g. on watch faces) in the early 20th century. I still own a radium watch. The paint was always fluorescent green. And it did glow.
Isn’t it usually the phosphor that wears out rather than the radium decaying that stops them glowing? Would it be possible for someone with the correct safety equipment to apply a new layer of fluorescent paint?
most of the GenAI players use both PyTorch and JAX, depending on the hardware they are running on. Character, Anthro, Midjourney, etc. are dual shops (they use both). xAI only uses JAX afaik.
JAX is used by almost every large genAI player (Anthropic, Cohere, DeepMind, Midjourney, Character.ai, XAi, Apple, etc.). Its actual market share in foundation models development is something like 80%.
Are there any resources going into detail about why the big players prefer JAX? I've heard this before but have never seen explanations of why/how this happened.
It's all about cost and performance. If you can train a foundation model 2x faster with JAX on the same hardware, you are effectively slashing your training costs by 2x, which is significant for a multi-million dollar training run.
The current SOTA models (GPT4, DALL-E, Sora) were trained on GPUs. The next one (GPT5) will be, too. And the one after that. Besides, only very few people train models that need more than a few hundred H100s at a time, and PyTorch works well at that scale. And when you train large scale stuff the scaling problems are demonstrably surmountable, unlike, say, capacity problems which you will run into if you need a ton of modern TPU quota, because Google itself is pretty compute starved at the moment. Also, gone are the days when TPUs were significantly faster. GPUs have “TPUs” inside them, too, nowadays
No, I am saying, with JAX you train on G.P.U., with a G, and your training runs are >2x faster, so your training costs are 2x lower, which matters whether your training spend is $1k or $100M. You're not interested in that? That's ok, but most people are.
Have you actually tried that or are you just regurgitating Google’s marketing? I’ve seen Jax perform _slower_ than PyTorch on practical GPU workloads on the exact same machine, and not by a little, by something like 20%. I too thought I’d be getting great performance and “saving money”, but reality turned out to be a bit more complicated than that - you have to benchmark and tune.
Two Keras creators are listed as authors on this post. If they were not involved, this should be specified. I specifically talked about research and StackOverflow is not in any way representative of what's used. Do you disagree that the majority of neural net research papers now only have PyTorch implementations, not TensorFlow? Also, according to Google Trends, PyTorch is more popular: https://trends.google.com/trends/explore?geo=US&q=pytorch,te.... BTW, I would love it if TF made a strong comeback, it's always better to have two big competing frameworks and I have some issues with PyTorch, including with its performance.
7 cubic km of olivine is a staggering amount. Makes you wonder whether the effect of deploying the olivine would even offset the energy cost and CO2 emissions of extracting and moving around all that rock.
If I were to divide large tech companies between "good" and "bad", the only one I would classify as "bad" is Facebook. Most are neutral-to-good. Facebook has a huge societal and human cost and has hardly any benefits to show for it (unlike, say, fossil energy companies, which are a threat to civilization but at least serve a useful role by fulfilling our energy needs).
Facebook is a democracy-threatening, attention-wasting, extractive institution, with a deeply unethical leadership. Most tech companies are neutral -- Amazon, Microsoft, Uber etc., I assume they're doing business for profit and not social good, but they do provide value and largely play by the rules. Some other companies I would classify as "good" as they provide considerable value to society as a by-product of doing business, like Apple and Google (disclaimer: I work for Google, and I am quite happy about that).
Facebook is the only large tech company I can think of that is just plain evil. There's no other like it. It's in a league of its own.
I would easily throw Uber into the evil category. "Contractor" loophole to evade minimum wage (min. wage being already pretty pathetic), Greyball, and a general culture of knowingly ignoring laws cultivated by it's founder.
> IMO, AssertionErrors should indicate bugs in the called API. ValueErrors indicate bugs in the caller's use of the API.
Yes, I agree with this stance. `ValueError` should be used for user-provided input validation (as well as `TypeError` in some cases). But as it happens, many Python developers use `assert` statements to do input validation, and generally don't provide any error messages in their `assert` statements. I'm suggesting going with `ValueError` (and a nice message) instead.
To clarify: `tf.keras` is an implementation of the entire Keras API written from the ground-up in pure TensorFlow. The first benefit of that is a greater level of blending between non-Keras-TF workflows and TF-Keras workflows: for instance, layers from `tf.layers` and `tf.keras` are interchangeable in all use cases.
Additionally, this enables us to add TensorFlow-specific features that would be difficult to add to the multi-backend version of Keras, and to do performance optimizations that would otherwise be impossible.
Such features include support for Eager mode (dynamic graphs), support for TensorFlow Estimators, which enables distributed training and training on TPUs, and more to come.
The green glow used in pop culture has its origin in the widespread use of radium paint to achieve a glow-in-the-dark effect (e.g. on watch faces) in the early 20th century. I still own a radium watch. The paint was always fluorescent green. And it did glow.