What is a common method to reduce the computational cost of deep learning models during inference?
Pruning removes weights, neurons, or entire filters/channels that contribute minimally to model output --- identified via magnitude-based criteria (removing near-zero weights), sensitivity analysis, or more sophisticated importance scoring --- producing a smaller, sparser model that requires fewer computations and less memory at inference time while aiming to preserve accuracy through careful selection and, often, a fine-tuning step after pruning to recover any lost performance. Structured pruning (removing entire filters/channels) yields hardware-friendly speedups on standard accelerators, while unstructured pruning (removing individual weights) achieves higher sparsity ratios but requires specialized sparse-computation hardware or libraries to realize actual speed gains.
The remaining options move in the wrong direction or address a different concern: adding more convolutional filters (B) increases model capacity and parameter count, which increases computational cost, the opposite of the stated goal. Increasing batch size (D) affects training throughput and memory usage per step but does not reduce the per-sample computational cost of inference --- a larger batch does more total work, not less per inference call, and batch size at inference is often constrained by latency requirements rather than optimization goals. Option C's premise --- selectively replacing activation functions with simpler ones in 'some neurons' --- is not a standard or well-defined optimization technique; activation function choice is typically uniform within a layer and driven by training dynamics, not a piecemeal inference-cost lever.
Haley
2 days agoKristian
7 days agoRefugia
13 days agoJudy
18 days agoMalika
23 days agoOdette
28 days agoBrendan
1 month agoBrittani
1 month ago