@GPU_MODE @fabynou now do denormalized

@GPU_MODE @fabynou I've never seen a description of exponent and mantissa that isn't just... This. What's different except the names?

@GPU_MODE @fabynou But, but, but ... this is the same explanation as always!
He's just decided to use his own names to call what everyone else calls mantissa and exponent.

@GPU_MODE @fabynou This maybe correct, but explaining it as a window and an offset has left me completely clueless. It's far more intuitive when explained as a mantissa and exponent.

@GPU_MODE @fabynou @grok does this mean that very large floating point numbers close to the ieee limits have lesser precision ?

@GPU_MODE @fabynou this intuition model should be presented in every classes that teach floating point numbers

@GPU_MODE @fabynou That’s not really why they’re called “floats” tho.

You’re representing N significant digits and where the “radix point” is (we call it a “decimal point” in base 10. IEEE is just one way of historically representing floats, there have been literally hundreds.

@GPU_MODE @fabynou This made me feel like that video of vegetarians eating meat for the first time... and crying.

@GPU_MODE @fabynou The most intuitive explanation I ever got was an analogy between floating point numbers and ticks on a ruler. The linked explanation is a bog standard explanation. I think I first got it in assembly language programming, again in numerical analysis, & yet again in some PL course.

@GPU_MODE @fabynou This is actually great

@GPU_MODE @fabynou Very nice. I am 100% going to use this next time I explain floats. Bravo!

@GPU_MODE @fabynou Great! So for arbitrary precision, can't we just create a tree of offsets?

@GPU_MODEf@fabynou is gold exposes why FP16 denormals vanish at 2⁻¹⁴ but FP32 holds to 2⁻¹²⁶. GPU kernels lean on this: BF16 strips subnormals to save gates, while TF32 uses FP32’s mantissa with FP64’s range for matmul without overflow. Real trick? Hardware fma units fuse before rounding offset reveals the hidden guard bits. 🤯

@GPU_MODE @fabynou I've always found floating point numbers to be a bit of a black box. This explanation makes it much more intuitive. Thanks for sharing!