Model Memory Footprint Estimation

EasyML Coding

Problem statement

When training or deploying large neural networks, we need to estimate how much memory a model will use to pick hardware and avoid out-of-memory errors. The total depends on the parameters and also on whether gradients and optimizer states must be stored during training.

Implement estimate_model_memory(num_params, bytes_per_param, optimizer, include_gradients, unit) that computes the total memory footprint.

Rules

  1. Parameter memory is always counted: num_params * bytes_per_param bytes.
  2. If include_gradients is True, add another num_params * bytes_per_param bytes for gradients (same precision as parameters).
  3. Optimizer states are stored in float32 (4 bytes per value):
    • 'none': 0 extra bytes
    • 'sgd': num_params * 4 bytes (one momentum buffer)
    • 'adam': num_params * 8 bytes (first and second moment buffers)
  4. Convert the total bytes to unit: 'MB' divides by 1024**2 and 'GB' by 1024**3.
  5. Return the result rounded to 2 decimal places.

Example

Input:

num_params = 124000000, bytes_per_param = 4, optimizer = 'none', include_gradients = False, unit = 'MB'

Output:

473.02

Reasoning: only parameters are stored, 124,000,000 * 4 = 496,000,000 bytes. Dividing by 1024^2 gives 473.0225 MB, which rounds to 473.02.

Show hint

Which separate buffers does a model need in memory during training, and which of them disappear at inference?

Relevant roles: ML Engineer, MLOps Engineer, LLM Engineer and AI Architect.

Related free problems