PyTorch Tensor Corruption Bug: Metadata Mismatch On Failed Resize

Hey there, fellow PyTorch enthusiasts! Today, we're diving into a rather sneaky bug that can cause some serious headaches in your deep learning workflows. We're talking about a situation where PyTorch tries to update a tensor's shape metadata even when the underlying storage resize operation fails. This seemingly small oversight can lead to corrupted tensors, often dubbed "Zombie" tensors, and can result in dreaded segmentation faults or internal runtime errors when you least expect them. Let's break down what's happening, why it's a problem, and how this affects your code.

The Nitty-Gritty of the Bug: When Resize Meets Reality

The core of this issue lies in how PyTorch handles tensor resizing, specifically when a tensor is linked to storage that cannot be resized. You know how sometimes you might inject a NumPy array into a PyTorch tensor using set_()? Well, that NumPy array's storage might be fixed, meaning it can't just be expanded or shrunk on demand. When you then try to call resize_() on such a tensor, PyTorch correctly identifies this problem and throws a RuntimeError with the message: Trying to resize storage that is not resizable. This is exactly what we'd want to happen – an error is raised, and the operation stops.

However, the bug lies in the exception safety, or rather, the lack thereof. Before PyTorch even checks if the storage can actually be resized, it goes ahead and updates the tensor's shape and stride metadata. So, by the time the RuntimeError is thrown, the tensor's shape attribute might be telling you it's a nice, big tensor (like a 5x5x5 matrix), but its actual storage() is still empty, holding zero bytes of data. This creates a dangerous disconnect, a phantom tensor if you will, where the dimensions declared don't match the memory allocated.

Imagine this scenario: You're expecting a full loaf of bread (your tensor data), but you've only got the wrapper. The system knows the wrapper is for a full loaf (the updated metadata), but there's no bread inside (the 0-byte storage). When you try to slice that bread, or even just look at it, things fall apart spectacularly. This is precisely what happens with these "Zombie" tensors. Subsequent attempts to access, print, or use this corrupted tensor often lead to hard crashes like segmentation faults or more cryptic internal runtime errors. It’s a silent corruption that can propagate through your model, making debugging a nightmare.

We've seen this manifest as a RuntimeError on printing in minimal examples, but in more complex, real-world scenarios, it can escalate to a full-blown segmentation fault. This is particularly problematic in training loops or data processing pipelines where operations are chained together, and a single corrupted tensor can bring the whole process crashing down.

A Minimal Reproduction Case: Seeing is Believing

To really get a handle on this bug, let's look at the code that triggers it. It's surprisingly simple and highlights the core issue:

import torch
import numpy as np

# Create non-resizable storage (0 bytes)
locked_storage = torch.from_numpy(np.array([], dtype=np.int32)).untyped_storage()

# Inject into a fresh tensor
t = torch.tensor([], dtype=torch.int32)
t.set_(locked_storage)

# Attempt to resize (Expected: Fail, maintain original shape)
# (Actual: Fails, but updates shape to 5x5x5)
try:
    t.resize_((5, 5, 5))
except RuntimeError:
    pass

# Verify corruption
print(f"Shape: {t.shape}")       # Prints: torch.Size([5, 5, 5])
print(f"Storage: {t.untyped_storage().nbytes()}") # Prints: 0
print(t) # CRASH

In this snippet, we first create an empty NumPy array and then convert its untyped_storage() into a locked_storage. This effectively gives us a tensor with no actual data. We then create a new, empty PyTorch tensor and explicitly set its storage to this locked_storage using t.set_(locked_storage). The intention is to have a tensor that points to this fixed, zero-byte storage.

Next, we hit the problematic part: t.resize_((5, 5, 5)). As expected, because the underlying storage isn't resizable, PyTorch raises a RuntimeError. However, the critical failure here is that before the error is raised, the tensor's metadata is already altered. The shape is changed from its initial torch.Size([]) to torch.Size([5, 5, 5]). This is the moment the tensor becomes corrupted.

When we try to verify the state, we see the alarming output: Shape: torch.Size([5, 5, 5]) and Storage: 0. The shape claims we have a 5x5x5 tensor, which should contain 125 elements, but the storage size is still 0 bytes. The final print(t) line is where the program typically crashes, either with a RuntimeError or a segmentation fault, because it's trying to read data from a tensor that claims to have data but doesn't.

The Expected vs. Actual Behavior: Upholding Guarantees

In the world of robust software development, especially in libraries like PyTorch that deal with complex computations, strong exception guarantees are crucial. For an operation like resize_(), we'd expect one of two things to happen:

1. Success: The resize operation completes, and both the tensor's metadata (shape, stride) and its underlying storage are updated correctly.
2. Failure with No Side Effects: If the operation fails (e.g., due to unresizable storage), the tensor should remain in its original, unmodified state. This is often referred to as the