Pinned Tweet
Any finite sequence of numbers can be represented exactly as a combination of complex oscillations, even when the original data looks nothing like a wave.
This is the mathematics behind the Discrete Fourier Transform (DFT), one of the fundamental tools of digital signal eers record sound, measure vibrations, or collect data from sensors, they obtain a sequence of samples. The DFT provides a different way to represent those same measurements, using complex exponentials at discrete frequencies.
The mathematics relies on a remarkable property called orthogonality.
For N samples, the DFT uses N orthogonal complex exponentials. Each frequency component has a corresponding complex coefficient, and the complete set of coefficients contains enough information to reconstruct every original sample.
When a complex sinusoid aligns exactly with a DFT frequency bin, its contributions reinforce at that bin and cancel at the others.
When its frequency falls between bins, the finite observation window causes its contribution to spread across multiple frequencies. This is known as spectral leakage.
These properties have enormous practical value in engineering.
In audio processing, Fourier analysis helps identify frequency components and design digital filters. In mechanical engineering, it helps detect periodic vibrations in rotating machinery. In wireless communications, OFDM systems use Fourier transforms to generate and recover signals carried by orthogonal subcarriers.
The mathematics also simplifies certain computations. Circular convolution in the sample domain becomes pointwise multiplication in the frequency domain.
The Fast Fourier Transform (FFT) makes these calculations efficient by exploiting mathematical symmetries, reducing the computational work from the quadratic cost of direct evaluation to roughly N log N for standard FFT algorithms.
There is one important distinction.
The DFT preserves all the information in the original sequence of samples, but it cannot recover information already lost during sampling.
It is an exact change of mathematical representation, not a way to recover frequencies that sampling has made indistinguishable.
This is the mathematics behind the Discrete Fourier Transform (DFT), one of the fundamental tools of digital signal eers record sound, measure vibrations, or collect data from sensors, they obtain a sequence of samples. The DFT provides a different way to represent those same measurements, using complex exponentials at discrete frequencies.
The mathematics relies on a remarkable property called orthogonality.
For N samples, the DFT uses N orthogonal complex exponentials. Each frequency component has a corresponding complex coefficient, and the complete set of coefficients contains enough information to reconstruct every original sample.
When a complex sinusoid aligns exactly with a DFT frequency bin, its contributions reinforce at that bin and cancel at the others.
When its frequency falls between bins, the finite observation window causes its contribution to spread across multiple frequencies. This is known as spectral leakage.
These properties have enormous practical value in engineering.
In audio processing, Fourier analysis helps identify frequency components and design digital filters. In mechanical engineering, it helps detect periodic vibrations in rotating machinery. In wireless communications, OFDM systems use Fourier transforms to generate and recover signals carried by orthogonal subcarriers.
The mathematics also simplifies certain computations. Circular convolution in the sample domain becomes pointwise multiplication in the frequency domain.
The Fast Fourier Transform (FFT) makes these calculations efficient by exploiting mathematical symmetries, reducing the computational work from the quadratic cost of direct evaluation to roughly N log N for standard FFT algorithms.
There is one important distinction.
The DFT preserves all the information in the original sequence of samples, but it cannot recover information already lost during sampling.
It is an exact change of mathematical representation, not a way to recover frequencies that sampling has made indistinguishable.
19
99
626
19,443
A 135K-star GitHub repo is built around one painfully obvious idea:
Your coding agent should map the codebase once, not grep it forever.
Graphify turns an entire project into a persistent, queryable knowledge graph that AI coding agents can use to understand how different parts Functions, classes, files, SQL schemas, infrastructure configurations, documentation, PDFs, images and videos can become connected nodes instead of isolated pieces of context the agent has to rediscover.
So instead of:
→ Search for authentication
→ Open twelve files
→ Follow imports and function calls
→ Lose the trail as the context fills up
The agent can ask:
> What connects authentication to the database?
> Trace the path from UserService to DatabasePool.
> Explain RateLimiter.
> Which components are central to this architecture?
Graphify returns relevant graph relationships and paths, rather than another collection of keyword matches.
For source code, the implementation is particularly interesting.
It uses tree-sitter to parse approximately 40 programming languages locally, extracting functions, classes, imports, calls, inheritance and cross-file relationships.
No embeddings, vector database or LLM calls are needed to construct the code graph.
Document and media processing works differently. PDFs, images and other non-code content can go through an optional semantic extraction step using a configured model.
Every relationship carries a confidence label:
→ EXTRACTED: Explicitly present in the source
→ INFERRED: Derived through analysis or resolution
→ AMBIGUOUS: Uncertain and flagged for review
Graphify also uses community detection to identify related subsystems and highlights highly connected nodes that may be important to the overall architecture.
But the part that makes this useful for coding agents is what happens after the graph is built.
Graphify integrates with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and more than 20 other assistants.
Depending on the platform, it installs hooks or persistent instructions that encourage the agent to query the existing graph before repeatedly searching and reading source files.
The graph can be committed to Git, updated incrementally as files change, and exposed through MCP.
There's also support for examining pull requests against the graph, including identifying PRs that touch related graph communities and may need closer review.
The local tool produces three useful artifacts: an interactive HTML graph, a Markdown architecture report and a reusable graph.json.
Long context windows help coding agents read more code.
A persistent knowledge graph gives them a way to navigate what they've already analyzed.
And as coding agents spend more time working inside large repositories, avoiding repeated discovery of the same architecture becomes increasingly useful.
Here's the GitHub Repo: github.com/Graphify-Labs/…
Your coding agent should map the codebase once, not grep it forever.
Graphify turns an entire project into a persistent, queryable knowledge graph that AI coding agents can use to understand how different parts Functions, classes, files, SQL schemas, infrastructure configurations, documentation, PDFs, images and videos can become connected nodes instead of isolated pieces of context the agent has to rediscover.
So instead of:
→ Search for authentication
→ Open twelve files
→ Follow imports and function calls
→ Lose the trail as the context fills up
The agent can ask:
> What connects authentication to the database?
> Trace the path from UserService to DatabasePool.
> Explain RateLimiter.
> Which components are central to this architecture?
Graphify returns relevant graph relationships and paths, rather than another collection of keyword matches.
For source code, the implementation is particularly interesting.
It uses tree-sitter to parse approximately 40 programming languages locally, extracting functions, classes, imports, calls, inheritance and cross-file relationships.
No embeddings, vector database or LLM calls are needed to construct the code graph.
Document and media processing works differently. PDFs, images and other non-code content can go through an optional semantic extraction step using a configured model.
Every relationship carries a confidence label:
→ EXTRACTED: Explicitly present in the source
→ INFERRED: Derived through analysis or resolution
→ AMBIGUOUS: Uncertain and flagged for review
Graphify also uses community detection to identify related subsystems and highlights highly connected nodes that may be important to the overall architecture.
But the part that makes this useful for coding agents is what happens after the graph is built.
Graphify integrates with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and more than 20 other assistants.
Depending on the platform, it installs hooks or persistent instructions that encourage the agent to query the existing graph before repeatedly searching and reading source files.
The graph can be committed to Git, updated incrementally as files change, and exposed through MCP.
There's also support for examining pull requests against the graph, including identifying PRs that touch related graph communities and may need closer review.
The local tool produces three useful artifacts: an interactive HTML graph, a Markdown architecture report and a reusable graph.json.
Long context windows help coding agents read more code.
A persistent knowledge graph gives them a way to navigate what they've already analyzed.
And as coding agents spend more time working inside large repositories, avoiding repeated discovery of the same architecture becomes increasingly useful.
Here's the GitHub Repo: github.com/Graphify-Labs/…
89
57
826
61,503
Machine Learning from Scratch 🚀
A 33K-star GitHub repo rebuilds machine learning algorithms from scratch, from linear regression to GANs and deep reinforcement learning.
It's called ML-From-Scratch, created by Erik Linder-Noren, and the entire project is designed around mentations rather than optimizing them for speed.
The repository contains readable Python implementations of the algorithms normally hidden behind high-level machine learning libraries.
It covers -
→ Linear and logistic regression
→ Decision trees, random forests and gradient boosting
→ XGBoost and support vector machines
→ K-means, DBSCAN, PCA and Gaussian mixtures
→ Neural networks, CNNs and RNNs
→ Generative adversarial networks
→ Deep Q-learning for CartPole
The neural-network code is especially useful. It includes implementations of convolutional layers, pooling, batch normalization, dropout, activation functions, loss functions and optimizers.
You can follow how the components fit together to train a convolutional classifier, generate handwritten digits with a GAN, or train a reinforcement-learning agent.
Most of the numerical work is built around NumPy, with additional scientific Python dependencies where needed.
The implementations prioritize readability over performance. It's an older codebase, so some examples need dependency updates to run with current libraries.
For anyone who has studied machine learning theory and wants to examine the actual algorithms, this is an excellent reference to work through.
GitHub: github.com/eriklindernore…
A 33K-star GitHub repo rebuilds machine learning algorithms from scratch, from linear regression to GANs and deep reinforcement learning.
It's called ML-From-Scratch, created by Erik Linder-Noren, and the entire project is designed around mentations rather than optimizing them for speed.
The repository contains readable Python implementations of the algorithms normally hidden behind high-level machine learning libraries.
It covers -
→ Linear and logistic regression
→ Decision trees, random forests and gradient boosting
→ XGBoost and support vector machines
→ K-means, DBSCAN, PCA and Gaussian mixtures
→ Neural networks, CNNs and RNNs
→ Generative adversarial networks
→ Deep Q-learning for CartPole
The neural-network code is especially useful. It includes implementations of convolutional layers, pooling, batch normalization, dropout, activation functions, loss functions and optimizers.
You can follow how the components fit together to train a convolutional classifier, generate handwritten digits with a GAN, or train a reinforcement-learning agent.
Most of the numerical work is built around NumPy, with additional scientific Python dependencies where needed.
The implementations prioritize readability over performance. It's an older codebase, so some examples need dependency updates to run with current libraries.
For anyone who has studied machine learning theory and wants to examine the actual algorithms, this is an excellent reference to work through.
GitHub: github.com/eriklindernore…
5
73
443
10,056
Even the fastest GPUs can stall!
The same LLM can be memory-bound at batch size 1 and compute-bound at a larger batch size, on the same GPU.
During low-batch decoding, moving model weights and KV-cache data can take more time than the arithmetic itself. Increasing the batch to be reused across more computations, raising arithmetic intensity and potentially shifting the bottleneck toward compute.
This is why the FLOP count of a model cannot tell you how fast it will execute.
At the CUDA level, several details determine actual GPU performance.
A kernel launches a grid of thread blocks. Blocks are assigned to streaming multiprocessors (SMs), where threads execute in warps of 32. Warp schedulers issue instructions from ready warps while others wait on memory or dependencies.
Memory access patterns can make an enormous difference.
For 32 threads loading one 4-byte value each, aligned contiguous addresses touch four 32-byte sectors.
Change the stride to 32 bytes, and those same 128 bytes of requested data can touch 32 sectors, covering 1,024 bytes.
That's eight times the sector coverage for the same useful data. Actual DRAM traffic depends on caching.
Matrix multiplication addresses data movement through tiling. Efficient kernels reuse operands in registers and shared memory, reducing repeated transfers from device memory.
Tensor Cores accelerate supported matrix operations, but keeping them busy requires suitable tile sizes, data layouts and enough work.
Kernel fusion reduces intermediate memory traffic and launch overhead. FlashAttention applies IO-aware tiling and online softmax to avoid materializing the full attention score matrix in HBM, while retaining the quadratic arithmetic complexity of dense attention.
Then there's occupancy.
More resident warps can help hide latency, but increasing occupancy doesn't necessarily increase throughput. Register usage, shared memory, instruction dependencies and scheduling all influence execution.
The roofline model provides a useful upper bound based on peak compute throughput, memory bandwidth and arithmetic intensity. Actual kernels can perform well below that bound.
This is where profiling becomes essential.
Nsight Systems shows launches, transfers and execution gaps. Nsight Compute shows what happens inside kernels, including memory transactions, warp stalls and occupancy.
Optimizing AI workloads on GPUs requires understanding both the mathematical operations and how those operations map onto the memory hierarchy, execution units and scheduling hardware.
That's the engineering behind turning theoretical FLOPs into actual throughput.
The same LLM can be memory-bound at batch size 1 and compute-bound at a larger batch size, on the same GPU.
During low-batch decoding, moving model weights and KV-cache data can take more time than the arithmetic itself. Increasing the batch to be reused across more computations, raising arithmetic intensity and potentially shifting the bottleneck toward compute.
This is why the FLOP count of a model cannot tell you how fast it will execute.
At the CUDA level, several details determine actual GPU performance.
A kernel launches a grid of thread blocks. Blocks are assigned to streaming multiprocessors (SMs), where threads execute in warps of 32. Warp schedulers issue instructions from ready warps while others wait on memory or dependencies.
Memory access patterns can make an enormous difference.
For 32 threads loading one 4-byte value each, aligned contiguous addresses touch four 32-byte sectors.
Change the stride to 32 bytes, and those same 128 bytes of requested data can touch 32 sectors, covering 1,024 bytes.
That's eight times the sector coverage for the same useful data. Actual DRAM traffic depends on caching.
Matrix multiplication addresses data movement through tiling. Efficient kernels reuse operands in registers and shared memory, reducing repeated transfers from device memory.
Tensor Cores accelerate supported matrix operations, but keeping them busy requires suitable tile sizes, data layouts and enough work.
Kernel fusion reduces intermediate memory traffic and launch overhead. FlashAttention applies IO-aware tiling and online softmax to avoid materializing the full attention score matrix in HBM, while retaining the quadratic arithmetic complexity of dense attention.
Then there's occupancy.
More resident warps can help hide latency, but increasing occupancy doesn't necessarily increase throughput. Register usage, shared memory, instruction dependencies and scheduling all influence execution.
The roofline model provides a useful upper bound based on peak compute throughput, memory bandwidth and arithmetic intensity. Actual kernels can perform well below that bound.
This is where profiling becomes essential.
Nsight Systems shows launches, transfers and execution gaps. Nsight Compute shows what happens inside kernels, including memory transactions, warp stalls and occupancy.
Optimizing AI workloads on GPUs requires understanding both the mathematical operations and how those operations map onto the memory hierarchy, execution units and scheduling hardware.
That's the engineering behind turning theoretical FLOPs into actual throughput.
8
33
163
4,441
Calculus becomes a different subject once you start building things with it.
In a textbook, differentiation can feel like a collection of rules for manipulating symbols. In engineering, it answers a practical question: how sensitive is the result to a small change somewhere in etwork makes that question especially interesting. A single parameter can influence the result through a long sequence of calculations. Understanding the operation that contains it is not enough. We need to account for everything between that operation and the final loss.
The chain rule lets us assemble the answer from local information. Each operation contributes its derivative, and those derivatives fit together to describe the sensitivity of the whole computation.
Backpropagation makes this practical by organising the calculation so that shared work is reused. We do not have to change each parameter separately and run the network again to estimate its effect.
There is something beautiful about being able to reason about a large, complicated system this way. We can calculate how its loss responds to individual parameters without having a simple explanation of everything the network does.
And there is real engineering in making that calculation useful. The order of operations, the intermediate values we save, and the memory available all matter.
The chain rule is still the one taught in calculus. It becomes much easier to appreciate when you have a few million derivatives to calculate.
In a textbook, differentiation can feel like a collection of rules for manipulating symbols. In engineering, it answers a practical question: how sensitive is the result to a small change somewhere in etwork makes that question especially interesting. A single parameter can influence the result through a long sequence of calculations. Understanding the operation that contains it is not enough. We need to account for everything between that operation and the final loss.
The chain rule lets us assemble the answer from local information. Each operation contributes its derivative, and those derivatives fit together to describe the sensitivity of the whole computation.
Backpropagation makes this practical by organising the calculation so that shared work is reused. We do not have to change each parameter separately and run the network again to estimate its effect.
There is something beautiful about being able to reason about a large, complicated system this way. We can calculate how its loss responds to individual parameters without having a simple explanation of everything the network does.
And there is real engineering in making that calculation useful. The order of operations, the intermediate values we save, and the memory available all matter.
The chain rule is still the one taught in calculus. It becomes much easier to appreciate when you have a few million derivatives to calculate.
1
26
156
5,326
DeepSeek just open-sourced the inference engine behind a cluster it says serves 155 trillion tokens a day.
And this is becoming quite a pattern. DeepSeek has already released FlashMLA, DeepGEMM, DeepEP, and several other pieces of its inference infrastructure. Now it's publishing the framework that brings much of that engineering together.
The project is called DeepInfer, the high-performance inference framework powering DeepSeek's large-scale production services.
It currently supports DeepSeek-V4.1-Flash, with several interesting optimizations for running large Mixture-of-Experts models efficiently.
The implementation includes expert parallelism with dynamic load balancing, allowing MoE experts to be distributed across GPUs while adjusting for uneven workloads.
For decoding, DeepInfer uses DSpark speculative decoding with confidence-scheduled verification. Draft tokens are generated ahead of time, and confidence estimates help determine when verification should happen.
The scheduler is also worth studying. CPU-side batch scheduling overlaps with GPU execution, reducing the idle time that normally appears between decoding steps.
Other features include sliding-window attention bounded replay, batch-invariant inference, deterministic sampling, and constrained generation using JSON schemas, regular expressions, or EBNF grammars.
DeepSeek has also published unusually detailed deployment notes describing how its production system separates prefill and decode, distributes MoE experts across large GPU groups, and reuses cached context.
The framework supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
One important limitation: the public release currently provides offline inference. The KV cache system, complete prefill/decode disaggregation, and load balancing required for a full online service are not included.
DeepInfer was originally built primarily on vLLM, with additional ideas from SGLang and TensorRT-LLM.
So this isn't a complete reproduction of DeepSeek's production deployment. But it makes a substantial part of the actual inference implementation available for engineers to inspect, benchmark, and build upon.
Fully open source under Apache 2.0.
GitHub: github.com/deepseek-ai/De…
And this is becoming quite a pattern. DeepSeek has already released FlashMLA, DeepGEMM, DeepEP, and several other pieces of its inference infrastructure. Now it's publishing the framework that brings much of that engineering together.
The project is called DeepInfer, the high-performance inference framework powering DeepSeek's large-scale production services.
It currently supports DeepSeek-V4.1-Flash, with several interesting optimizations for running large Mixture-of-Experts models efficiently.
The implementation includes expert parallelism with dynamic load balancing, allowing MoE experts to be distributed across GPUs while adjusting for uneven workloads.
For decoding, DeepInfer uses DSpark speculative decoding with confidence-scheduled verification. Draft tokens are generated ahead of time, and confidence estimates help determine when verification should happen.
The scheduler is also worth studying. CPU-side batch scheduling overlaps with GPU execution, reducing the idle time that normally appears between decoding steps.
Other features include sliding-window attention bounded replay, batch-invariant inference, deterministic sampling, and constrained generation using JSON schemas, regular expressions, or EBNF grammars.
DeepSeek has also published unusually detailed deployment notes describing how its production system separates prefill and decode, distributes MoE experts across large GPU groups, and reuses cached context.
The framework supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
One important limitation: the public release currently provides offline inference. The KV cache system, complete prefill/decode disaggregation, and load balancing required for a full online service are not included.
DeepInfer was originally built primarily on vLLM, with additional ideas from SGLang and TensorRT-LLM.
So this isn't a complete reproduction of DeepSeek's production deployment. But it makes a substantial part of the actual inference implementation available for engineers to inspect, benchmark, and build upon.
Fully open source under Apache 2.0.
GitHub: github.com/deepseek-ai/De…
6
4
37
2,019
Tech with Mak retweeted
Stanford put its entire LLM engineering course on YouTube. The homework includes writing FlashAttention in Triton.
Its called CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto.
The Spring 2026 course covers the complete process of building and s, including the systems engineering required to make them efficient.
The lectures cover:
→ Tokenization and Transformer architectures
→ Attention alternatives and Mixture of Experts
→ GPU architecture, memory and compute
→ Triton kernels and FlashAttention
→ Distributed training and parallelism
→ Scaling laws and inference optimization
→ Pretraining data pipelines and evaluation
→ Supervised fine-tuning, RLHF and reasoning RL
But the assignments are where this course becomes particularly valuable.
Students implement their own tokenizer, Transformer and optimizer before training a small language model. They then profile the implementation, write FlashAttention-2 in Triton, and develop memory-efficient distributed training.
Later assignments involve fitting scaling laws, processing raw Common Crawl data, implementing filtering and deduplication, and using supervised fine-tuning and reinforcement learning to train models for mathematical reasoning.
Stanford provides minimal scaffolding and even recommends disabling AI autocomplete during the assignments so students actually understand the code they're writing.
The full lecture playlist, assignment handouts and lecture materials are publicly available. Some practical exercises require GPU compute.
For ML engineers who want to understand the full stack, from training data and attention kernels to inference and post-training, this is an excellent course to work through.
YouTube Playlist: youtube.com/playlist?list=…
Course & Assignments: cs336.stanford.edu
Lecture Code: github.com/stanford-cs336…
Its called CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto.
The Spring 2026 course covers the complete process of building and s, including the systems engineering required to make them efficient.
The lectures cover:
→ Tokenization and Transformer architectures
→ Attention alternatives and Mixture of Experts
→ GPU architecture, memory and compute
→ Triton kernels and FlashAttention
→ Distributed training and parallelism
→ Scaling laws and inference optimization
→ Pretraining data pipelines and evaluation
→ Supervised fine-tuning, RLHF and reasoning RL
But the assignments are where this course becomes particularly valuable.
Students implement their own tokenizer, Transformer and optimizer before training a small language model. They then profile the implementation, write FlashAttention-2 in Triton, and develop memory-efficient distributed training.
Later assignments involve fitting scaling laws, processing raw Common Crawl data, implementing filtering and deduplication, and using supervised fine-tuning and reinforcement learning to train models for mathematical reasoning.
Stanford provides minimal scaffolding and even recommends disabling AI autocomplete during the assignments so students actually understand the code they're writing.
The full lecture playlist, assignment handouts and lecture materials are publicly available. Some practical exercises require GPU compute.
For ML engineers who want to understand the full stack, from training data and attention kernels to inference and post-training, this is an excellent course to work through.
YouTube Playlist: youtube.com/playlist?list=…
Course & Assignments: cs336.stanford.edu
Lecture Code: github.com/stanford-cs336…
8
51
303
9,941
Tech with Mak retweeted
The π in the normal distribution has a geometric origin.
It comes from a famous mathematical result known as the Gaussian integral.
∫₋∞^∞ e^(−x²) dx = √π
The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line ly.
The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation.
This result is fundamental to probability theory.
The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1.
But its applications extend well beyond statistics.
In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals.
Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium.
In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering.
The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis.
And the √π in its integral ultimately comes from the geometry of the plane.
It comes from a famous mathematical result known as the Gaussian integral.
∫₋∞^∞ e^(−x²) dx = √π
The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line ly.
The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation.
This result is fundamental to probability theory.
The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1.
But its applications extend well beyond statistics.
In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals.
Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium.
In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering.
The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis.
And the √π in its integral ultimately comes from the geometry of the plane.
15
91
582
21,581
Stanford put its entire LLM engineering course on YouTube. The homework includes writing FlashAttention in Triton.
Its called CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto.
The Spring 2026 course covers the complete process of building and s, including the systems engineering required to make them efficient.
The lectures cover:
→ Tokenization and Transformer architectures
→ Attention alternatives and Mixture of Experts
→ GPU architecture, memory and compute
→ Triton kernels and FlashAttention
→ Distributed training and parallelism
→ Scaling laws and inference optimization
→ Pretraining data pipelines and evaluation
→ Supervised fine-tuning, RLHF and reasoning RL
But the assignments are where this course becomes particularly valuable.
Students implement their own tokenizer, Transformer and optimizer before training a small language model. They then profile the implementation, write FlashAttention-2 in Triton, and develop memory-efficient distributed training.
Later assignments involve fitting scaling laws, processing raw Common Crawl data, implementing filtering and deduplication, and using supervised fine-tuning and reinforcement learning to train models for mathematical reasoning.
Stanford provides minimal scaffolding and even recommends disabling AI autocomplete during the assignments so students actually understand the code they're writing.
The full lecture playlist, assignment handouts and lecture materials are publicly available. Some practical exercises require GPU compute.
For ML engineers who want to understand the full stack, from training data and attention kernels to inference and post-training, this is an excellent course to work through.
YouTube Playlist: youtube.com/playlist?list=…
Course & Assignments: cs336.stanford.edu
Lecture Code: github.com/stanford-cs336…
Its called CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto.
The Spring 2026 course covers the complete process of building and s, including the systems engineering required to make them efficient.
The lectures cover:
→ Tokenization and Transformer architectures
→ Attention alternatives and Mixture of Experts
→ GPU architecture, memory and compute
→ Triton kernels and FlashAttention
→ Distributed training and parallelism
→ Scaling laws and inference optimization
→ Pretraining data pipelines and evaluation
→ Supervised fine-tuning, RLHF and reasoning RL
But the assignments are where this course becomes particularly valuable.
Students implement their own tokenizer, Transformer and optimizer before training a small language model. They then profile the implementation, write FlashAttention-2 in Triton, and develop memory-efficient distributed training.
Later assignments involve fitting scaling laws, processing raw Common Crawl data, implementing filtering and deduplication, and using supervised fine-tuning and reinforcement learning to train models for mathematical reasoning.
Stanford provides minimal scaffolding and even recommends disabling AI autocomplete during the assignments so students actually understand the code they're writing.
The full lecture playlist, assignment handouts and lecture materials are publicly available. Some practical exercises require GPU compute.
For ML engineers who want to understand the full stack, from training data and attention kernels to inference and post-training, this is an excellent course to work through.
YouTube Playlist: youtube.com/playlist?list=…
Course & Assignments: cs336.stanford.edu
Lecture Code: github.com/stanford-cs336…
8
51
303
9,941
Someone built Git for AI agent memory.
ModernRelay has open-sourced OmniGraph, a graph database that gives AI agents something most shared-memory systems lack: branches, commits, diffs, and three-way merges.
The idea is simple. If multiple agents are working on the same ould be able to make changes independently without corrupting or overwriting each others work.
Imagine three agents researching the same subject. One extracts information from documents, another resolves duplicate entities, and a third connects claims to supporting evidence.
With OmniGraph, each agent works on its own branch of the knowledge graph. Changes to entities, relationships, and evidence remain isolated until they're reviewed and merged.
And this is where the engineering gets interesting.
The merge engine compares changes against a common ancestor, much like Git. It can detect conflicting entity updates, delete-versus-modify conflicts, orphaned relationships, and schema violations.
Conflicting changes are reported explicitly. Successful merges are committed atomically, and previous graph states remain available through version history.
OmniGraph also combines several retrieval methods in one engine:
→ Graph traversal for following relationships
→ Vector search for semantic similarity
→ BM25 for keyword-based retrieval
→ Reciprocal Rank Fusion for combining results
Under the hood, its written in Rust and uses versioned Lance files for storage, with support for local and S3-compatible object stores.
Typed schemas enforce valid relationships, Cedar policies control mutations, and an MCP server lets coding agents interact with the graph.
The important distinction is that semantic retrieval alone doesn't solve the problem of multiple agents changing shared knowledge.
Agents also need to know what changed, who changed it, which version they're reading, and whether two sets of updates can safely coexist.
OmniGraph brings those familiar source-control ideas into persistent agent memory.
Open source. MIT licensed.
Here's the GitHub Repo: github.com/ModernRelay/om…
ModernRelay has open-sourced OmniGraph, a graph database that gives AI agents something most shared-memory systems lack: branches, commits, diffs, and three-way merges.
The idea is simple. If multiple agents are working on the same ould be able to make changes independently without corrupting or overwriting each others work.
Imagine three agents researching the same subject. One extracts information from documents, another resolves duplicate entities, and a third connects claims to supporting evidence.
With OmniGraph, each agent works on its own branch of the knowledge graph. Changes to entities, relationships, and evidence remain isolated until they're reviewed and merged.
And this is where the engineering gets interesting.
The merge engine compares changes against a common ancestor, much like Git. It can detect conflicting entity updates, delete-versus-modify conflicts, orphaned relationships, and schema violations.
Conflicting changes are reported explicitly. Successful merges are committed atomically, and previous graph states remain available through version history.
OmniGraph also combines several retrieval methods in one engine:
→ Graph traversal for following relationships
→ Vector search for semantic similarity
→ BM25 for keyword-based retrieval
→ Reciprocal Rank Fusion for combining results
Under the hood, its written in Rust and uses versioned Lance files for storage, with support for local and S3-compatible object stores.
Typed schemas enforce valid relationships, Cedar policies control mutations, and an MCP server lets coding agents interact with the graph.
The important distinction is that semantic retrieval alone doesn't solve the problem of multiple agents changing shared knowledge.
Agents also need to know what changed, who changed it, which version they're reading, and whether two sets of updates can safely coexist.
OmniGraph brings those familiar source-control ideas into persistent agent memory.
Open source. MIT licensed.
Here's the GitHub Repo: github.com/ModernRelay/om…
23
19
108
5,032
The π in the normal distribution has a geometric origin.
It comes from a famous mathematical result known as the Gaussian integral.
∫₋∞^∞ e^(−x²) dx = √π
The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line ly.
The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation.
This result is fundamental to probability theory.
The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1.
But its applications extend well beyond statistics.
In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals.
Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium.
In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering.
The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis.
And the √π in its integral ultimately comes from the geometry of the plane.
It comes from a famous mathematical result known as the Gaussian integral.
∫₋∞^∞ e^(−x²) dx = √π
The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line ly.
The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation.
This result is fundamental to probability theory.
The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1.
But its applications extend well beyond statistics.
In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals.
Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium.
In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering.
The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis.
And the √π in its integral ultimately comes from the geometry of the plane.
15
91
582
21,581
I don't want AI deciding what I say. I do want to stop recording the same sentence five times.
Anyone who's made a course, tutorial, or product demo knows how much time disappears into retakes, corrections, and recording sessions.
The interesting part of voice cloning is being able to separate authorship from the physical act of speaking.
You write it. You approve it. Your voice delivers it.
Anyone who's made a course, tutorial, or product demo knows how much time disappears into retakes, corrections, and recording sessions.
The interesting part of voice cloning is being able to separate authorship from the physical act of speaking.
You write it. You approve it. Your voice delivers it.
Professional Voice Clone on HeyGen sounds exactly like you.
Upload 30 min to 2 hrs of audio and get a voice no one can tell apart from your own. Every breath, pause, laugh, and accent, preserved.
Comment "PVC" for a free month. First 100, web subscriptions only, ends 11/19.
4
2
19
1,642
Decision models give AI agents a way to evaluate defined outcomes without generating an answer as text.
The basic interface is straightforward. An application supplies a state, a question, and the possible answers. The model assigns probabilities to those answers, which the e for classification, tool selection, routing, severity assessment, or other decisions.
That sounds simple, but the mathematics has consequences across the entire agent system.
A binary decision produces a probability that can be compared against a threshold. A categorical decision distributes probability mass across the supplied alternatives. An ordinal decision produces a distribution over ordered levels, from which an expected score can be calculated. These are different output contracts, and each requires different handling by the application.
There are also important limitations. Softmax normalization does not guarantee that probabilities are calibrated. A high probability does not establish that a prediction is correct. Changing the available options changes the decision problem, and omitting the correct answer can force probability mass onto incorrect alternatives. Even when the prediction is reasonable, the appropriate action depends on error costs, thresholds, permissions, and escalation policies.
I put together a technical handbook on Decision Models for AI Agents, using Drex 2.0 and Drex Max to work through the mathematics and engineering in detail.
Drex 2.0 provides a useful implementation reference through its released model weights and runtime. The handbook examines how candidate answers are represented, how hidden-state readouts are used in pointer-head scoring, how logits become probabilities, and how question execution can reuse shared state.
Drex Max extends the typed decision interface to images. That introduces another set of problems around image preprocessing, context limits, visual ambiguity, and incomplete evidence. The handbook keeps its documented behavior separate from internal architectural details that have not been publicly established.
The material works through everything from first principles => binary, categorical, and ordinal decisions, probability distributions, expected risk, cost-sensitive thresholds, abstention, pointer-head mathematics, temperature scaling, shared-state inference, multimodal inputs, calibration, evaluation, and production failure modes.
The explanations include worked mathematical examples, with Drex-specific implementation details grounded in Nace AI's official code and documentation.
Sharing the handbook here 👇
The basic interface is straightforward. An application supplies a state, a question, and the possible answers. The model assigns probabilities to those answers, which the e for classification, tool selection, routing, severity assessment, or other decisions.
That sounds simple, but the mathematics has consequences across the entire agent system.
A binary decision produces a probability that can be compared against a threshold. A categorical decision distributes probability mass across the supplied alternatives. An ordinal decision produces a distribution over ordered levels, from which an expected score can be calculated. These are different output contracts, and each requires different handling by the application.
There are also important limitations. Softmax normalization does not guarantee that probabilities are calibrated. A high probability does not establish that a prediction is correct. Changing the available options changes the decision problem, and omitting the correct answer can force probability mass onto incorrect alternatives. Even when the prediction is reasonable, the appropriate action depends on error costs, thresholds, permissions, and escalation policies.
I put together a technical handbook on Decision Models for AI Agents, using Drex 2.0 and Drex Max to work through the mathematics and engineering in detail.
Drex 2.0 provides a useful implementation reference through its released model weights and runtime. The handbook examines how candidate answers are represented, how hidden-state readouts are used in pointer-head scoring, how logits become probabilities, and how question execution can reuse shared state.
Drex Max extends the typed decision interface to images. That introduces another set of problems around image preprocessing, context limits, visual ambiguity, and incomplete evidence. The handbook keeps its documented behavior separate from internal architectural details that have not been publicly established.
The material works through everything from first principles => binary, categorical, and ordinal decisions, probability distributions, expected risk, cost-sensitive thresholds, abstention, pointer-head mathematics, temperature scaling, shared-state inference, multimodal inputs, calibration, evaluation, and production failure modes.
The explanations include worked mathematical examples, with Drex-specific implementation details grounded in Nace AI's official code and documentation.
Sharing the handbook here 👇
16
11
64
2,796
You can read the handbook here:
drive.google.com/file/d/1YrcFbx…
drive.google.com/file/d/1YrcFbx…
0
0
2
420
You can literally play with fluid dynamics in your browser.
Pavel Dobryakov built a real-time fluid simulation that runs on the GPU with WebGL.
Click and drag and you can push dye through the velocity field, create vortices, mix colors and watch the flow evolve in real time.
based on GPU fluid-simulation techniques for the Navier-Stokes equations.
No install. Works on mobile too.
paveldogreat.github.io/WebGL-Fluid-Si…
Pavel Dobryakov built a real-time fluid simulation that runs on the GPU with WebGL.
Click and drag and you can push dye through the velocity field, create vortices, mix colors and watch the flow evolve in real time.
based on GPU fluid-simulation techniques for the Navier-Stokes equations.
No install. Works on mobile too.
paveldogreat.github.io/WebGL-Fluid-Si…
3
6
40
2,572
You are lucky if you work with a senior engineer who never makes you regret asking a basic question.
You ask something you think everyone else already knows.
They explain it without showing off.
Then somehow you leave understanding more than what you originally asked.
Those ou remember them for years.
You ask something you think everyone else already knows.
They explain it without showing off.
Then somehow you leave understanding more than what you originally asked.
Those ou remember them for years.
3
1
25
1,627
LLM quantization reduces the precision used to represent model weights, activations and KV-cache tensors.
The goal is to reduce memory requirements and data movement while keeping numerical error within acceptable limits.
But bit width alone doesn't determine accuracy, memory e speed.
There are several important engineering decisions.
1./ Quantization formats
INT4, INT8, NF4, FP4 and FP8 use different numerical representations.
Uniform integer quantization maps values through scales and, optionally, zero-points. Floating-point formats have exponent and fraction fields, while NF4 uses a nonuniform codebook.
The same bit width can produce different quantization errors.
2./ Scaling and granularity
Quantization can operate per tensor, per channel or per group.
Smaller groups can reduce reconstruction error, but require additional scaling metadata and compatible kernels.
Rounding and clipping introduce error. Outliers make this particularly important.
3./ Quantization algorithms
GPTQ: Uses approximate second-order information to reduce errors introduced by weight quantization.
AWQ: Uses activation statistics to identify salient weight channels and protect them during quantization.
SmoothQuant: Redistributes quantization difficulty between activations and weights to enable W8A8 quantization.
These methods address different aspects of the quantization problem.
4./ Training approaches
PTQ: Quantizes an already-trained model, often using calibration data.
QAT: Accounts for quantization effects during training, commonly using fake-quantization operations.
QLoRA: Keeps a quantized pretrained base model frozen while training low-rank adapters.
5./ Memory accounting
For a model with exactly 8 billion weights:
FP16 weights = 16 GB
Packed INT4 weights = 4 GB
One FP16 scale per 128 weights = 0.125 GB
Total = 4.125 GB for weights and scales, assuming no additional quantization metadata.
KV cache, activations and temporary buffers are separate memory costs.
6./ Storage versus computation
A W4A16 model uses 4-bit weights and 16-bit activations. That doesnt necessarily mean the matrix multiplication itself runs in 4-bit arithmetic.
Some kernels unpack and dequantize weight tiles before performing FP16/BF16 operations. Accumulation precision is another choice.
The actual speedup depends on kernel support, dequantization overhead, memory bandwidth, batch size and workload characteristics.
7./ Evaluation
Quantization needs to be evaluated for both model quality and serving performance.
Measure held-out task accuracy, numerical stability, time to first token, time per output token, throughput and peak memory under comparable conditions.
A 4-bit model isn't automatically four times faster than its 16-bit counterpart.
Quantization determines how values are represented. The hardware, kernels and workload determine how efficiently those values are processed.
The goal is to reduce memory requirements and data movement while keeping numerical error within acceptable limits.
But bit width alone doesn't determine accuracy, memory e speed.
There are several important engineering decisions.
1./ Quantization formats
INT4, INT8, NF4, FP4 and FP8 use different numerical representations.
Uniform integer quantization maps values through scales and, optionally, zero-points. Floating-point formats have exponent and fraction fields, while NF4 uses a nonuniform codebook.
The same bit width can produce different quantization errors.
2./ Scaling and granularity
Quantization can operate per tensor, per channel or per group.
Smaller groups can reduce reconstruction error, but require additional scaling metadata and compatible kernels.
Rounding and clipping introduce error. Outliers make this particularly important.
3./ Quantization algorithms
GPTQ: Uses approximate second-order information to reduce errors introduced by weight quantization.
AWQ: Uses activation statistics to identify salient weight channels and protect them during quantization.
SmoothQuant: Redistributes quantization difficulty between activations and weights to enable W8A8 quantization.
These methods address different aspects of the quantization problem.
4./ Training approaches
PTQ: Quantizes an already-trained model, often using calibration data.
QAT: Accounts for quantization effects during training, commonly using fake-quantization operations.
QLoRA: Keeps a quantized pretrained base model frozen while training low-rank adapters.
5./ Memory accounting
For a model with exactly 8 billion weights:
FP16 weights = 16 GB
Packed INT4 weights = 4 GB
One FP16 scale per 128 weights = 0.125 GB
Total = 4.125 GB for weights and scales, assuming no additional quantization metadata.
KV cache, activations and temporary buffers are separate memory costs.
6./ Storage versus computation
A W4A16 model uses 4-bit weights and 16-bit activations. That doesnt necessarily mean the matrix multiplication itself runs in 4-bit arithmetic.
Some kernels unpack and dequantize weight tiles before performing FP16/BF16 operations. Accumulation precision is another choice.
The actual speedup depends on kernel support, dequantization overhead, memory bandwidth, batch size and workload characteristics.
7./ Evaluation
Quantization needs to be evaluated for both model quality and serving performance.
Measure held-out task accuracy, numerical stability, time to first token, time per output token, throughput and peak memory under comparable conditions.
A 4-bit model isn't automatically four times faster than its 16-bit counterpart.
Quantization determines how values are represented. The hardware, kernels and workload determine how efficiently those values are processed.
11
23
202
5,058
Attention explained!!
I put together a handbook on how attention works, from the Q/K/V equation up through how it actually runs in modern LLMs.
Self vs. cross-attention, causal masking, multi-head attention, RoPE, KV caching, MHA/MQA/GQA/MLA, FlashAttention, worked through by hand, with citations pers.
Made it to be the explanation I wish I'd had.
6
58
427
14,840
Here's my free handbook:
drive.google.com/file/d/1mJxGsq…
For more such handbooks subscribe to my Free Newsletter. thecuriousmak.substack.com/subscribe
drive.google.com/file/d/1mJxGsq…
For more such handbooks subscribe to my Free Newsletter. thecuriousmak.substack.com/subscribe
0
2
4
594