Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical profiling methodology for DNN performance analysis, addressing the increasing complexity and computational demands of modern AI systems. The methodology integrates tools like cProfile, PyTorch Profiler, and NVIDIA Nsight Systems to analyze performance across different layers of abstraction, from code to GPU execution. A case study using VGGNet demonstrates the approach's ability to trace performance bottlenecks from high-level code to low-level GPU operations.
Uncover hidden DNN performance bottlenecks by systematically tracing inefficiencies from high-level code down to GPU kernel execution.
As AI systems proliferate, Deep Neural Networks (DNNs) continue to grow in complexity and computational demand, making performance optimization crucial for both training and inference—especially with the emergence of Large Language Models (LLMs) and generative AI. Because bottlenecks can arise across different layers of execution—from code and framework operations to GPU kernels and system I/O—effective diagnosis requires a structured, multi-level approach. In this paper, we present a hierarchical profiling methodology that organizes performance analysis across interconnected layers of abstraction, integrating tools such as cProfile, PyTorch Profiler, and NVIDIA Nsight Systems. Using VGGNet as a case study, we demonstrate how this approach enables systematic tracing from high-level code behavior to low-level GPU execution to identify the root causes of inefficiency.