In data manipulation, it is typical to reshape or reorder the original data and create multiple copies. At any new step, a new copy is created. As the program grows, so does the occupied memory, and I seldom think to worry about this issue until an Out Of Memory error happens.
The amazing thing about tensors is that multiple tensors can refer to the same storage, allowing for much more efficient memory usage.
In the next article I will write about the even more amazing Tensor property of tracking the ancestors operations, but here I will mostly focus on memory optimization.