← All questions
LLMOps & ProductionHard· CUDA· Kernel Development· Fusion

Walk me through how you would implement a custom activa...

A custom fused CUDA kernel requires a two-part structure. The CPU wrapper handles device checks, contiguous checks, empty_like allocation, grid computation, and the kernel launch. The GPU kernel computes the global index, performs bounds checking, executes the activation function, and writes the output. Fusion is essential because it eliminates multiple HBM round-trips that a naive implementation would incur for every sub-operation.

Code playground (Python)