Standard Library
copy — Shallow vs Deep Copy
Python's copy module provides mechanisms to duplicate objects to avoid unintended side effects during mutation. Understanding the distinction between shallow and deep copies is critical because simple assignment only creates a reference to the original object. You should reach for these tools whenever you need to modify a collection or a nested structure while preserving the integrity of the source data.
The Assignment Fallacy
In Python, assigning a variable to another variable using the assignment operator does not create a copy of the data. Instead, it creates a new reference pointing to the exact same object in memory. Because many Python types, such as lists and dictionaries, are mutable, modifications made through one variable are immediately reflected in all other variables referencing that same object. This behavior is intentional to save memory and performance, as copying large data structures unnecessarily would be computationally expensive. However, this often leads to subtle bugs where altering a list inside a function unexpectedly changes the original list defined in the calling scope. To avoid these side effects, developers must explicitly create a new, independent copy when they intend to modify a container without affecting the initial source of that data.
original_list = [1, 2, 3]
reference = original_list # This is not a copy
reference.append(4)
print(original_list) # [1, 2, 3, 4] - original changed!Understanding Shallow Copies
A shallow copy, created using the copy module or specific constructors like list(), creates a new collection object but populates it with references to the same child objects found in the original. While the outer container (the top-level list or dictionary) is indeed a new object, the elements contained within are not duplicated. If the inner elements are immutable, such as integers or strings, a shallow copy behaves as if it were a full duplicate because you cannot change the underlying value of an integer. However, if the shallow copy contains other mutable objects, such as nested lists, changes made to those nested objects will propagate back to the original source. A shallow copy is sufficient only when you are modifying the structure of the top-level container, such as adding or removing elements, without intending to modify the internal contents themselves.
import copy
original = [[1, 2], [3, 4]]
shallow = copy.copy(original)
shallow[0][0] = 999 # Modifies nested list in both
print(original) # [[999, 2], [3, 4]]Deep Copies for Nested Structures
When your data structure contains multiple levels of nesting—such as a list of dictionaries, where those dictionaries contain other lists—a shallow copy fails to isolate the modifications entirely. This is where a deep copy becomes necessary. A deep copy recursively traverses the object tree, creating new copies of the container and every single nested object it encounters. This ensures that the newly created structure is entirely disconnected from the original. By duplicating every object in the hierarchy, deep copies guarantee that no matter how complex the structure is, modifying any part of the copy will have zero impact on the source. While this provides the highest level of safety and isolation, it is significantly more memory-intensive and slower than a shallow copy because it must construct an entirely new tree of objects during the execution of the command.
import copy
original = [[1, 2], [3, 4]]
deep = copy.deepcopy(original)
deep[0][0] = 999 # Only modifies the deep copy
print(original) # [[1, 2], [3, 4]]Performance and Resource Considerations
Choosing between copy types is a balance between safety and resource management. A shallow copy is highly efficient because it only creates the outer envelope of the data structure, leaving the heavy lifting of copying inner objects untouched. In high-performance systems or data-intensive applications, using deepcopy unnecessarily can lead to significant overhead, particularly when dealing with massive collections or deep recursions that could otherwise be handled with simple indexing or slice operations. However, performance should never come at the cost of correctness. If your application logic requires total isolation, the penalty of a deep copy is the correct price to pay. Developers must carefully evaluate if their code structure relies on shared state or if it can be redesigned to be functional and stateless, which often eliminates the need for expensive duplication altogether.
import timeit
# Shallow copy is significantly faster than deep copy
# for large nested structures.
data = [[i for i in range(100)] for _ in range(100)]
print(timeit.timeit(lambda: copy.copy(data), number=1000))Customizing Copy Behavior
The copy module is highly flexible and allows developers to control how their custom classes participate in the copying process. By implementing the special methods __copy__() and __deepcopy__() within a class, you can define exactly what happens when one of these operations is triggered. This is incredibly useful for objects that contain internal resources, such as file handles, network connections, or database sockets, which should not be duplicated blindly. For instance, you might want to perform a shallow copy of an object's metadata but prevent the copying of the underlying resource itself, or initialize a new connection during a deep copy. Defining these methods gives you granular control over state management, ensuring that your objects remain consistent even when they are duplicated across different parts of your application logic or testing frameworks.
class ResourceManager:
def __deepcopy__(self, memo):
# Custom logic for deep copy
return "Resource handle cannot be copied"
obj = ResourceManager()
print(copy.deepcopy(obj)) # Returns custom stringKey points
- Assignment in Python merely creates a new reference to the existing object in memory.
- Mutable objects share their state across all references until a true copy is performed.
- A shallow copy creates a new container but holds references to the original child objects.
- Shallow copies are safe if the nested elements are immutable types like strings or integers.
- Deep copies recursively duplicate every object within the hierarchy to ensure total isolation.
- Deep copies are memory-intensive and slower due to the overhead of traversing and cloning structures.
- The copy module provides standard interfaces for both shallow and deep copying operations.
- Classes can override __copy__ and __deepcopy__ to define custom behavior for their instances.
Common mistakes
- Mistake: Assuming the assignment operator (=) creates a copy. Why it's wrong: In Python, '=' only creates a new reference to the same object in memory. Fix: Use the copy module or slicing/constructors to create a new object.
- Mistake: Thinking copy.copy() is a deep copy. Why it's wrong: copy.copy() is a shallow copy and only copies the outer container, not the nested objects. Fix: Use copy.deepcopy() if you need to duplicate nested data structures.
- Mistake: Modifying a nested list after a shallow copy. Why it's wrong: Shallow copies share references to mutable child objects, so changes in one affect the other. Fix: Perform a deep copy if the nested structures need to be independent.
- Mistake: Copying custom class instances using slicing like [:] or list(). Why it's wrong: These methods only work for built-in container types; they do not trigger a copy of a custom object's internal state. Fix: Define __copy__ or __deepcopy__ methods in the class or use the copy module.
- Mistake: Using copy.deepcopy() on objects containing recursive references. Why it's wrong: It can cause infinite recursion if not handled, though Python's deepcopy usually tracks memoized objects, manual logic errors often occur. Fix: Ensure your object graph is managed and use deepcopy only when necessary.
Interview questions
What is the fundamental difference between a shallow copy and a deep copy in Python?
A shallow copy creates a new object, but it inserts references into that new object for the child objects found in the original. Essentially, the container is copied, but the items inside remain the same objects. A deep copy, conversely, recursively copies the original object and all objects found within it, creating a completely independent clone. This means changing a nested object in a deep copy will never affect the original, whereas a shallow copy might leave you vulnerable to shared references.
Can you explain how to perform a shallow copy in Python and why you might choose it over a deep copy?
To perform a shallow copy, you typically use the 'copy' module's 'copy()' function or slicing methods like 'list_a[:]'. You choose a shallow copy when you have a flat data structure, such as a list of integers or strings, because it is significantly faster and more memory-efficient than a deep copy. Since there are no nested mutable objects, the shallow copy behaves effectively like a deep copy, making it a pragmatic choice for performance-sensitive tasks where full recursion is unnecessary.
When dealing with a nested list in Python, what happens if you perform a shallow copy and then modify an inner list?
If you perform a shallow copy on a list containing nested lists, both the original and the copy will reference the exact same sub-lists in memory. For instance, if you have 'list_a = [[1, 2], 3]' and perform 'list_b = copy.copy(list_a)', modifying 'list_b[0][0] = 9' will also update 'list_a' because they point to the same internal list object. This behavior often leads to subtle, hard-to-track bugs where unintentional side effects occur in your data structures.
Compare the performance and memory implications of using shallow copy versus deep copy.
Shallow copy is highly efficient because it only copies the top-level structure, essentially creating a new container of references. It operates in constant time relative to the number of top-level elements. Deep copy, however, requires a recursive traversal of the entire object graph, keeping track of visited objects to handle circular references. This leads to higher CPU usage and significant memory overhead, making deep copy much slower, especially for large, complex, or deeply nested objects in your Python application.
How does Python handle circular references during a deep copy operation?
Python’s 'copy.deepcopy()' function is designed to handle circular references by maintaining an internal dictionary, often referred to as a memo dictionary, which tracks objects that have already been copied. When the deepcopy process encounters an object, it first checks if the object is in this memo dictionary. If it is, it returns the reference to the previously copied instance instead of recursing indefinitely. This mechanism is crucial for preventing stack overflow errors and ensuring that the object identity remains consistent throughout the entire duplicated structure.
Describe a scenario where neither shallow nor deep copy is sufficient and how you might approach custom object cloning.
Sometimes, you need to copy an object but exclude specific fields, like database connections or file handles, which cannot be serialized or replicated. In these cases, you can implement the '__copy__' and '__deepcopy__' methods in your class. By defining these magic methods, you gain fine-grained control over the cloning process, allowing you to explicitly instantiate a new object and selectively copy only the attributes that are meaningful or safe to duplicate, thereby bypassing the limitations of the default copy module.
Check yourself
1. Given a list 'a = [[1, 2], [3, 4]]' and 'b = a.copy()', what happens if you execute 'b[0][0] = 9'?
- A.Only b[0][0] becomes 9
- B.Both a[0][0] and b[0][0] become 9
- C.An error is raised because a is a nested list
- D.Only a[0][0] becomes 9
Show answer
B. Both a[0][0] and b[0][0] become 9
A shallow copy creates a new outer list but shares the references to the nested lists. Because they point to the same list objects, modifying 'b' affects 'a'. The other options are incorrect because the outer structure is copied, but the inner elements are not.
2. Which of the following scenarios absolutely requires a deep copy to ensure complete independence?
- A.Copying a list of integers
- B.Copying a list of strings
- C.Copying a dictionary containing lists as values
- D.Copying a tuple of immutable integers
Show answer
C. Copying a dictionary containing lists as values
Lists are mutable. In a dictionary of lists, a shallow copy duplicates the dictionary, but the nested lists remain shared. Integers, strings, and tuples of integers are immutable, so sharing references is safe and functionally identical to a deep copy.
3. What is the result of using 'list(original_list)' compared to a shallow copy?
- A.It performs a deep copy
- B.It results in the exact same shallow copy behavior
- C.It creates a reference, not a copy
- D.It causes an error if the list contains objects
Show answer
B. It results in the exact same shallow copy behavior
Both the list constructor and .copy() create a new list object with references to the original items, making them both shallow copies. It is not a deep copy, nor is it merely a reference.
4. If you define a custom class 'Data' and use 'copy.deepcopy(instance)', what is required for it to work?
- A.The class must inherit from a special copyable base class
- B.The class must implement a __deepcopy__ method
- C.By default, Python handles custom objects by recursively copying their __dict__
- D.It will fail because custom objects cannot be deep copied
Show answer
C. By default, Python handles custom objects by recursively copying their __dict__
The copy module's deepcopy function is designed to recursively traverse the object's dictionary and copy attributes. The other options are incorrect because no special inheritance is needed and it does not fail by default.
5. Consider 'x = [1, 2, 3]'. If you perform 'y = x[:]', which statement is true?
- A.x and y occupy the same memory address
- B.y is a new list object, but contains the same integer references as x
- C.y is a deep copy of x
- D.Modifying x will affect y since they share memory
Show answer
B. y is a new list object, but contains the same integer references as x
Slicing with [:] creates a new list (a shallow copy). Since integers are immutable in Python, sharing the references is perfectly safe and efficient. Option 0 is false as they are different objects; option 2 is false as it's only shallow; option 3 is false because integers cannot be modified in place.