Soldering dump a ton of heat into the cell, which has chances of destroying the cell. That's why most of the cells are spot-welded: its similar to soldering, but its much quicker and is localized only on the part of the metal that need to be melted, so the heat don't have time to reach the cell itself.
3. >> I was resounding told that the absolute error in the numbers are too small to be a problem. Frankly, I did not believe this.
> I would personally also tell that to the author. But there is a much more important reason why correct rounding would be a tremendous advantage: reproducibility.
This is also what the author want from his own experiences, but failed to realize/state explicitly: "People on different machines were seeing different patterns being generated which meant that it broke an aspect of our multiplayer game."
So yes, the reasons mentioned as a rationale for more accurate functions are in fact rationale for reproducibility across hardware and platforms. For example going from 1 ulp errors to 0.6 ulp errors would not help the author at all, but having reproducible behavior would (even with an increased worst case error).
Correctly rounded functions means the rounding error is the smallest possible, and as a consequence every implementation will always return exactly the same results: this is the main reason why people (and the author) advocates for correctly rounded implementations.
ARM have been moving away from chips with small area for a long time (see server SoC which are huge beasts), and are trying to become the standard platform for everyone trying to have custom hardware.
In this space, chiplets makes a lot of sense: you can have a compute chip with standard arm cores which is reused across your products, and add an extra chiplet with custom IPs depending on the product needs. That is for example what (as far as I'm aware) Huawei is doing: they reuse the chiplet with arm cores in different product, then add for example an IO+crypto die in the SoC in their routers/firewalls products, etc.
More than the ISA, its the memory interconnect that require standardization. At SoC level, ARM is already a de-facto standard (ACE-Lite, CHI, ...), but its only a standard for communication inside a chip, to interconnect varius IPs.
I guess this standard aim to keep being a standard interconnect even in multi-chiplets system, to create/extend the whole ecosystem around ARM partners.
In addition to the other comments, the iso C23 standard added the <stdbit.h> header to the standard library with a stdc_count_ones() function, so compiler support will become standard.
The article goes into great detail and gives several example of several CEOs borrowing for actual decades, so it passes the sniff test because it does actually happen.
The article gives examples of CEOs borrowing against their shares. The article provides no examples of CEOs rolling those loans over until their death.
Mathematicians already explored exactly what you describe: this is the difference between classical logic and intuitionistic logic:
In classical logic statements can be true in and of themselves even if there as no proof of it, but in intuitionistic logic statements are true only if there is a proof of it: the proof is the cause for the statement to be true.
In intuitionistic logic, things are not as simple as "either there is a cow in the field, or there is none" because as you said, for the knowledge of "a cow is in the field" to be true, you need a proof of it. It brings lots of nuance, for example "there isn't no cow in the field" is a weaker knowledge than "there is a cow in the field".
It is a fascinating topic. I spent a few hours on it once. I remember vaguely that the logic is very configurable and you had a lot of choices. Like you choose law of excluded middle or not I think, and things like that depending on your taste or problem. I might be wrong it was 8 years ago and I spent a couple of weeks reading about it.
Also no suprise the rabbit hole came from Haskell where those types (huh) are attracted to this more.foundational theory of computation.
The goal of this type of quantization is to move the multiplication by the fp32 rescale factor outside of the dot-product accumulation.
So the multiplications+additions are done on fp8/int8/int4/whatever (when the hardware support those operators of course) and accumulated in a fp32 or similar, and only the final accumulator is multiplied by the rescale factor in fp32.
Its worse than that: the energy gains are when comparing computations made with fp32, but for fp8 the multipliers are really tiny and the adder/shifters represent a largest part of the operators (energy-wise and area-wise) and this paper will only have small gains.
On fp8, the estimated gate count of fp8 multipliers is 296 vs. 157 with their technique, so the power gain on the multipliers will be much lower (50% would be a more reasonable estimation), but again for fp8 the additions in the dot products are a large part of the operations.
Overall, its really disingenuous to claim 80% power gain and small drop in accuracy, when the power gain is only for fp32 operations and the small drop in accuracy is only for fp8 operators. They don't analyze the accuracy drop in fp32, and don't present the power saved for fp8 dot product.
Basically no one uses FP32 at inference time. BF16/FP16 is typically considered unquantized, whereas FP8 is lightly quantized. That being said there's pretty minimal quality loss at FP8 compared to 16-bit typically; Llama 3.1 405b, for example, only benchmarks around ~1% worse when run at FP8: https://blog.vllm.ai/2024/07/23/llama31.html
Every major inference provider other than Hyperbolic Labs runs Llama 3.1 405b at FP8, FWIW (e.g. Together, Fireworks, Lepton), so to compare against FP32 is misleading to say the least. Even Hyperbolic runs it at BF16.
With serious diminishing returns. At inference time, no reason to use fp64 and should probably use fp8 or less. The accuracy loss is far less than you'd expect. AFAIK Llama 3.2 3B fp4 will outperform Llama 3.2 1B at fp32 in accuracy and speed, despite 8x precision.
I suspect that from the usage in the code, it knows that there is a module foo and a submodule subfoo with a function bar() in it, and it can look directly in the file for the definition of bar().
It would be another story if we used this opportunity to mangle the submodules names for example, but that the kind of hidden control flow that nobody want in his codebase.
Also, it is not some dark arts of import or something: it is pretty standard at this point since its one of the most sane way of breaking circular dependencies between your modules, and the feature of overloading a module __getattr__ was introduced specifically for this usecase. (I couldn't find the specific PEP that introduced it, sorry)
