diff --git a/src/posts/bf16-in-golang.md b/src/posts/bf16-in-golang.md index a29ef30..1216fb6 100644 --- a/src/posts/bf16-in-golang.md +++ b/src/posts/bf16-in-golang.md @@ -11,10 +11,16 @@ In the era of large language models, low-precision floating-point numbers are no ## Introduction to BF16 -``` -+------------+-----------------+-----------------+ -| 1 sign bit | 8 exponent bits | 7 mantissa bits | -+------------+-----------------+-----------------+ +```mermaid +--- +config: + packet: + bitsPerRow: 16 +--- +packet ++1: "Sign" ++8: "Exponent" ++7: "Mantissa" ``` BF16 consists of 1 sign bit, 8 exponent bits, and 7 mantissa bits. Compared to FP32, it has the same number of exponent bits but significantly fewer mantissa bits. As a result, BF16 maintains the same dynamic range as FP32 (approximately $±3.4×10³⁸$), sacrificing some precision (around $2⁻⁷$) for a larger exponent range. The larger exponent range effectively avoids gradient overflow and underflow, making it suitable for backpropagation in deep learning. Additionally, it has strong compatibility with FP32, simplifying hardware design. BF16 is used to replace FP32 in model training and inference scenarios, reducing memory usage and bandwidth requirements while increasing the number of data processed by SIMD instructions in a single operation. diff --git a/src/zh/posts/bf16-in-golang.md b/src/zh/posts/bf16-in-golang.md index 8b3d4d7..878abbb 100644 --- a/src/zh/posts/bf16-in-golang.md +++ b/src/zh/posts/bf16-in-golang.md @@ -12,10 +12,16 @@ tag: ## BF16介绍 -``` -+---------+--------+--------+ -| 1符号位 | 8指数位 | 7尾数位 | -+---------+--------+--------+ +```mermaid +--- +config: + packet: + bitsPerRow: 16 +--- +packet ++1: "符号位" ++8: "指数位" ++7: "尾数位" ``` BF16由1位符号位、 8位指数位和7位尾数位构成。和FP32相比,指数位相同,但尾数位大幅减少。因此BF16动态范围与FP32完全一致(约$±3.4×10³⁸$),牺牲部分精度(约$2^{-7}$)换取更大的指数范围。大指数范围可有效避免梯度溢出,适合深度学习中的反向传播。同时与FP32兼容性强,硬件设计更简单。BF16用于在模型训练和推理场景替代FP32,降低内存占用和带宽需求,提升SIMD指令一次处理数据的数量。