PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading.
Install the skill above, then run this in your AI tool:
/pytorch-patterns <your description>
About
Idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications.
Source
What gets added to your project
Preview every file this package writes on install before you run the command above.
Summary
PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading. Use when writing or reviewing PyTorch training loops, model architectures, or d...
Skill instructions
PyTorch Development Patterns
Idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications.
When to Activate
Writing new PyTorch models or training scripts
Reviewing deep learning code
Debugging training loops or data pipelines
Optimizing GPU memory usage or training speed
Setting up reproducible experiments
Core Principles
1. Device-Agnostic Code
Always write code that works on both CPU and GPU without hardcoding devices.
python
# Good: Device-agnostic
device = torch.device("cuda"if torch.cuda.is_available() else"cpu")
model = MyModel().to(device)
data = data.to(device)
# Bad: Hardcoded device
model = MyModel().cuda() # Crashes if no GPU
data = data.cuda()
2. Reproducibility First
Set all random seeds for reproducible results.
python
# Good: Full reproducibility setupdefset_seed(seed: int = 42) -> None:
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
np.random.seed(seed)
random.seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False# Bad: No seed control
model = MyModel() # Different weights every run
3. Explicit Shape Management
Always document and verify tensor shapes.
python
# Good: Shape-annotated forward passdefforward(self, x: torch.Tensor) -> torch.Tensor:
# x: (batch_size, channels, height, width)
x = self.conv1(x) # -> (batch_size, 32, H, W)
x = self.pool(x) # -> (batch_size, 32, H//2, W//2)
x = x.view(x.size(0), -1) # -> (batch_size, 32*H//2*W//2)returnself.fc(x) # -> (batch_size, num_classes)# Bad: No shape trackingdefforward(self, x):
x = self.conv1(x)
x = self.pool(x)
x = x.view(x.size(0), -1) # What size is this?returnself.fc(x) # Will this even work?
# Good: AMP with GradScaler
scaler = torch.amp.GradScaler("cuda")
for data, target in dataloader:
with torch.amp.autocast("cuda"):
output = model(data)
loss = criterion(output, target)
scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()
optimizer.zero_grad(set_to_none=True)
Gradient Checkpointing for Large Models
python
# Good: Trade compute for memoryfrom torch.utils.checkpoint import checkpoint
classLargeModel(nn.Module):
defforward(self, x: torch.Tensor) -> torch.Tensor:
# Recompute activations during backward to save memory
x = checkpoint(self.block1, x, use_reentrant=False)
x = checkpoint(self.block2, x, use_reentrant=False)
returnself.head(x)
torch.compile for Speed
python
# Good: Compile the model for faster execution (PyTorch 2.0+)
model = MyModel().to(device)
model = torch.compile(model, mode="reduce-overhead")
# Modes: "default" (safe), "reduce-overhead" (faster), "max-autotune" (fastest)
Quick Reference: PyTorch Idioms
Idiom
Description
model.train() / model.eval()
Always set mode before train/eval
torch.no_grad()
Disable gradients for inference
optimizer.zero_grad(set_to_none=True)
More efficient gradient clearing
.to(device)
Device-agnostic tensor/model placement
torch.amp.autocast
Mixed precision for 2x speed
pin_memory=True
Faster CPU→GPU data transfer
torch.compile
JIT compilation for speed (2.0+)
weights_only=True
Secure model loading
torch.manual_seed
Reproducible experiments
gradient_checkpointing
Trade compute for memory
Anti-Patterns to Avoid
python
# Bad: Forgetting model.eval() during validation
model.train()
with torch.no_grad():
output = model(val_data) # Dropout still active! BatchNorm uses batch stats!# Good: Always set eval mode
model.eval()
with torch.no_grad():
output = model(val_data)
# Bad: In-place operations breaking autograd
x = F.relu(x, inplace=True) # Can break gradient computation
x += residual # In-place add breaks autograd graph# Good: Out-of-place operations
x = F.relu(x)
x = x + residual
# Bad: Moving data to GPU inside the training loop repeatedlyfor data, target in dataloader:
model = model.cuda() # Moves model EVERY iteration!# Good: Move model once before the loop
model = model.to(device)
for data, target in dataloader:
data, target = data.to(device), target.to(device)
# Bad: Using .item() before backward
loss = criterion(output, target).item() # Detaches from graph!
loss.backward() # Error: can't backprop through .item()# Good: Call .item() only for logging
loss = criterion(output, target)
loss.backward()
print(f"Loss: {loss.item():.4f}") # .item() after backward is fine# Bad: Not using torch.save properly
torch.save(model, "model.pt") # Saves entire model (fragile, not portable)# Good: Save state_dict
torch.save(model.state_dict(), "model.pt")
Remember: PyTorch code should be device-agnostic, reproducible, and memory-conscious. When in doubt, profile with torch.profiler and check GPU memory with torch.cuda.memory_summary().
Add this badge to a GitHub README so readers can open the package page and install the same version.
[](https://www.aipm-registry.com/skills/affaan-m/pytorch-patterns/1.0.0?utm_source=github&utm_medium=readme&utm_campaign=package_badge)
This imported publisher has not claimed its AIPM account. A connected account confirms account control, but it does not guarantee that a package is safe.
AI assistant context
What this skill is for
@affaan-m/pytorch-patterns@1.0.0 is an AIPM skill package for All tools. Use it when a project needs the reusable AI behavior described as: PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading. Use when writing or reviewing PyTorch training loops, model architectures, or d...
Preview a UI component mockup in a local browser canvas where the human directly edits it (move, resize, edit text, delete, restyle), switches mobile/desktop/custom viewports, tests it, chats with you, and clicks Save and Finish. You the...
Run aipm add @affaan-m/pytorch-patterns@1.0.0 in a project that has AIPM initialized.
Supported tools
@affaan-m/pytorch-patterns@1.0.0 supports All tools.
Before installing
AIPM skills can add files to your project. Review the package name, target, description, license, and publisher before installing. Use a clean branch when trying a new skill.