Python Interview Questions For Experienced

Core Python & Advanced Concepts

1. What is the difference between __new__ and __init__? When would you use __new__?

• Answer:

  • __new__ is a static method responsible for creating and returning a new instance of the class. It's the first step in object instantiation.

  • __init__ is an instance method responsible for initializing the newly created object (i.e., setting initial values). It's called after __new__.

  • You would use __new__ when you need to control the creation of a new instance. Common use cases are:

    • Implementing the Singleton or Factory design patterns.

    • Subclassing immutable types (like tuple, str, int) where you must set the value at the moment of creation.

    • Returning an object of a different class from the constructor.

2. Explain the Global Interpreter Lock (GIL). What are its implications for multi-threading?

• Answer:
  The GIL is a mutex that allows only one thread to hold control of the Python interpreter at a time. This means that even in a multi-threaded Python program, only one thread can execute Python bytecode at any given moment.

  • Implications:

    • CPU-bound tasks: Multi-threading is ineffective for CPU-intensive tasks (e.g., mathematical computations) because threads cannot run in parallel on multiple cores. They end up taking turns, which can even add overhead.

    • I/O-bound tasks: Multi-threading can still be beneficial for I/O-bound tasks (e.g., network requests, file reading) because while one thread is waiting for I/O, the GIL is released, allowing other threads to run.

  • Workarounds: To achieve true parallelism for CPU-bound work, use the multiprocessing module (which uses separate processes, each with its own Python interpreter and GIL) or leverage libraries like numpy that perform computations in external, optimized C libraries that release the GIL.

3. How do Metaclasses work in Python? Can you give a practical example of their use?

• Answer:
  Metaclasses are the "classes of classes." They define how a class itself behaves. Just as a class defines the behavior of its instances, a metaclass defines the behavior of classes and their instances.

  • The type metaclass: The built-in type is the default metaclass. You can use type to create classes dynamically: MyClass = type('MyClass', (), {'attr': 100}).

  • Creating a custom metaclass: You create one by subclassing type.

  • Practical Example: A common use case is creating an API where you want to automatically register all subclasses in a registry.

  python

  class PluginRegistry(type):
      plugins = []
      def __init__(cls, name, bases, attrs):
          super().__init__(name, bases, attrs)
          # Automatically register every class using this metaclass
          if name != 'BasePlugin':
              PluginRegistry.plugins.append(cls)
  
  class BasePlugin(metaclass=PluginRegistry):
      pass
  
  class EmailPlugin(BasePlugin):
      pass
  
  class ChatPlugin(BasePlugin):
      pass
  
  print(PluginRegistry.plugins)  # Output: [<class '__main__.EmailPlugin'>, <class '__main__.ChatPlugin'>]

   

4. What are context managers? How do you create one?

• Answer:
  Context managers are objects that define the runtime context to be established when entering and exiting a with block. They are used to manage resources, ensuring they are properly acquired and released (e.g., files, locks, database connections).

  • Using a class: Define __enter__(self) and __exit__(self, exc_type, exc_val, exc_tb) methods.

  • Using contextlib: The contextlib module provides utilities like @contextmanager decorator for creating context managers with a generator function.

  python

  # Method 1: Using a class
  class FileManager:
      def __init__(self, filename, mode):
          self.filename = filename
          self.mode = mode
      def __enter__(self):
          self.file = open(self.filename, self.mode)
          return self.file
      def __exit__(self, exc_type, exc_val, exc_tb):
          self.file.close()
  
  # Method 2: Using contextlib
  from contextlib import contextmanager
  @contextmanager
  def open_file(filename, mode):
      try:
          f = open(filename, mode)
          yield f
      finally:
          f.close()

   

5. Explain Descriptors in Python. How are they related to properties?

• Answer:
  A descriptor is an object attribute with "binding behavior", accessed through the protocol methods __get__(), __set__(), and __delete__(). They are the underlying mechanism for properties, methods, static methods, and class methods.

  • Relation to @property: The property() function is a concise way to build a data descriptor. property(fget=None, fset=None, fdel=None, doc=None) internally creates a descriptor object.

  • Example of a custom descriptor:

  python

  class PositiveNumber:
      def __init__(self, name):
          self.name = name
      def __get__(self, instance, owner):
          return instance.__dict__[self.name]
      def __set__(self, instance, value):
          if value <= 0:
              raise ValueError("Positive number required.")
          instance.__dict__[self.name] = value
  
  class Order:
      quantity = PositiveNumber('quantity') # Descriptor instance
      price = PositiveNumber('price')
  
  order = Order()
  order.quantity = 10  # Calls PositiveNumber.__set__
  print(order.quantity) # Calls PositiveNumber.__get__
  # order.price = -5   # Would raise ValueError

   

Memory Management & Performance

6. How does Python manage memory?

• Answer:
  Python uses private heap space for memory management. All Python objects and data structures are located in this private heap. The Python memory manager handles the allocation and deallocation of this heap space.

  • Reference Counting: The primary garbage collection mechanism. Each object has a count of the number of references to it. When this count drops to zero, the memory is deallocated immediately.

  • Generational Garbage Collector (gc module): A secondary mechanism to handle reference cycles (e.g., object A refers to B, and B refers to A). It detects and collects cycles that reference counting cannot handle.

7. What are the differences between __slots__ and __dict__?

• Answer:

  • __dict__ is a dictionary present in (almost) every object by default, storing all its instance attributes. It provides flexibility to add attributes dynamically but consumes more memory.

  • __slots__ is a class variable that can be assigned a sequence of strings. It tells the interpreter to use a fixed-size array for storing attributes instead of a __dict__.

  • Key Differences:

    • Memory: __slots__ can significantly reduce memory footprint, especially when creating many small objects.

    • Performance: Attribute access is generally faster with __slots__.

    • Flexibility: Using __slots__ prevents the creation of __dict__, so you cannot add new attributes to instances dynamically. It's a trade-off between performance/memory and flexibility.

Concurrency & Parallelism

8. Compare and contrast multiprocessing vs multithreading vs asyncio. When would you use each?

• Answer:

  • multithreading: Best for I/O-bound tasks where threads spend most of their time waiting for external resources (APIs, disks, databases). It cannot achieve true parallelism for CPU work due to the GIL. Use concurrent.futures.ThreadPoolExecutor.

  • multiprocessing: Best for CPU-bound tasks that require true parallelism across multiple CPU cores. It bypasses the GIL by creating separate Python processes, each with its own memory space. The overhead is higher than threading. Use concurrent.futures.ProcessPoolExecutor.

  • asyncio: Best for high-concurrency I/O-bound tasks, especially those involving a large number of network connections (e.g., web servers, chat bots). It uses a single-threaded event loop and async/await syntax to handle many tasks concurrently. It's more efficient than threading for this specific use case because it avoids the overhead of thread context switching.

9. What is the difference between concurrent.futures and asyncio?

• Answer:

  • concurrent.futures: This is a high-level interface for asynchronously executing callables using threads (ThreadPoolExecutor) or processes (ProcessPoolExecutor). It is blocking by nature; you submit tasks and wait for the results. It's simpler to use for parallelizing a function over a dataset.

  • asyncio: This is a library to write concurrent code using the async/await syntax. It is non-blocking and single-threaded. It's designed for structured, event-driven programming where you have fine-grained control over when tasks yield control back to the event loop. It's more complex but more powerful for building async applications.

OOP & Design Patterns

10. Explain the Monkey Patching in Python. What are its pros and cons?

• Answer:
  Monkey patching is the dynamic modification of a class or module at runtime.

  • Example: json.dumps = my_custom_dumps_function

  • Pros:

    • Useful for testing (e.g., patching a network call to return mock data).

    • Can be used to apply hot-fixes in a running application.

  • Cons:

    • Can make code harder to understand and debug ("spooky action at a distance").

    • Can lead to compatibility issues if the original class/module changes.

    • Can create subtle bugs if not done carefully. It should be used sparingly.

11. What is the Method Resolution Order (MRO) and how does the C3 linearization algorithm work?

• Answer:
  MRO is the order in which Python looks for a method in a hierarchy of classes. It is determined by the C3 linearization algorithm.

  • C3 Algorithm Rules: It ensures a consistent and monotonic order. For a class C(B1, B2, ...), the MRO is the merge of [C] + the merge of the MROs of its bases [B1, B2, ...] + the list of bases [B1, B2, ...].

  • The merge preserves the order from the parent MROs and ensures a class always appears before its parents.

  • You can view the MRO of a class with Class.__mro__ or Class.mro().

12. How would you implement a Singleton pattern in Python?

• Answer:
  There are several ways. A common and simple one is to override the __new__ method.

  python

  class Singleton:
      _instance = None
      def __new__(cls, *args, **kwargs):
          if cls._instance is None:
              cls._instance = super().__new__(cls, *args, **kwargs)
          return cls._instance
  
  obj1 = Singleton()
  obj2 = Singleton()
  print(obj1 is obj2)  # Output: True

  • Other methods: Using a metaclass, or a decorator. The __new__ method is often the most straightforward.

Decorators & Generators

13. How would you write a decorator that can be used with or without parameters?

• Answer:
  You need to create a decorator that can be called in two ways: @decorator or @decorator(arg1, arg2). This requires nesting the decorator logic.

  python

  from functools import wraps
  
  def configurable_decorator(arg=None):
      def actual_decorator(func):
          @wraps(func)
          def wrapper(*args, **kwargs):
              # Do something with 'arg' here
              print(f"Decorator argument: {arg}")
              return func(*args, **kwargs)
          return wrapper
      # If called without parentheses, 'arg' is the function
      if callable(arg):
          return actual_decorator(arg) # Called as @decorator
      else:
          return actual_decorator       # Called as @decorator(...)
  
  # Usage without parameters
  @configurable_decorator
  def func1():
      pass
  
  # Usage with parameters
  @configurable_decorator("hello")
  def func2():
      pass

   

14. What is the difference between a generator and a list comprehension?

• Answer:

  • List Comprehension: [x*2 for x in range(5)] creates and returns a full list in memory all at once. For large data, this can be memory-intensive.

  • Generator Expression: (x*2 for x in range(5)) returns a generator iterator. It yields one item at a time, on-demand (lazy evaluation). It is much more memory-efficient for large or infinite sequences but does not support indexing or len().

  python

  # List comprehension - memory heavy
  big_list = [x**2 for x in range(1000000)]
  # Generator expression - memory efficient
  big_gen = (x**2 for x in range(1000000))

   

Testing & Debugging

15. What are some common best practices for writing unit tests in Python?

• Answer:

  • Isolation: Tests should be independent and not rely on the state from other tests.

  • Naming: Use clear, descriptive names (e.g., test_function_should_raise_value_error_on_invalid_input).

  • Arrange-Act-Assert (AAA) Pattern: Structure tests clearly into these three sections.

  • Use unittest or pytest: pytest is very popular for its simplicity and powerful features (fixtures, parametrization).

  • Test Edge Cases: Don't just test the "happy path". Test for invalid inputs, boundary conditions, and exceptions.

  • Use Mocking: Use unittest.mock to isolate the code under test by replacing dependencies with mock objects.

16. How do you profile a Python application to find performance bottlenecks?

• Answer:

  • cProfile: A built-in module that provides deterministic profiling. It shows the number of calls and total time spent in each function. python -m cProfile -s time my_script.py

  • line_profiler (external): A module that shows line-by-line execution times. It's more granular than cProfile.

  • memory_profiler (external): A module that monitors memory consumption on a line-by-line basis.

  • Visualization Tools: Tools like snakeviz can visualize cProfile output to make it easier to understand.

System Design & Architecture

17. How does Python's "duck typing" system influence your software design?

• Answer:
  Duck typing ("if it walks like a duck and quacks like a duck, then it must be a duck") means that an object's suitability is determined by the presence of certain methods and properties, rather than the type of the object itself.

  • Influence on Design:

    • Flexibility and Polymorphism: It allows for very flexible and reusable code. A function can work with any object that has the required behavior (methods/attributes), without needing a common base class.

    • Loose Coupling: Code is less dependent on rigid inheritance hierarchies.

    • Responsibility: The developer must be more careful to document expected interfaces and handle cases where an object doesn't support the required operations (often using try/except blocks or hasattr() checks).

18. Explain the concept of "Data Classes" (introduced in Python 3.7). What problem do they solve?

• Answer:
  Data Classes (@dataclass) are a decorator that automatically generates special methods (__init__, __repr__, __eq__) for classes that are primarily used to store data.

  • Problem Solved: They eliminate the need to write repetitive boilerplate code.

  • Example:

  python

  from dataclasses import dataclass
  
  @dataclass
  class InventoryItem:
      name: str
      unit_price: float
      quantity_on_hand: int = 0
  
      def total_cost(self) -> float:
          return self.unit_price * self.quantity_on_hand
  
  # __init__, __repr__, __eq__ are automatically generated.
  item = InventoryItem("Widget", 19.99, 10)
  print(item) # Output: InventoryItem(name='Widget', unit_price=19.99, quantity_on_hand=10)

   

19. What is the difference between a shallow copy and a deep copy?

• Answer:

  • Shallow Copy: copy.copy(x) creates a new object, but inserts references into it to the objects found in the original. If the original contains mutable objects, changes to those mutable objects will be reflected in both the copy and the original.

  • Deep Copy: copy.deepcopy(x) creates a new object and then, recursively, inserts copies of the objects found in the original. The copy is completely independent of the original.

  python

  import copy
  original = [[1, 2], [3, 4]]
  shallow = copy.copy(original)
  deep = copy.deepcopy(original)
  
  original[0][0] = 99
  print(shallow)  # Output: [[99, 2], [3, 4]] -> Changed!
  print(deep)     # Output: [[1, 2], [3, 4]] -> Unchanged!

   

20. Describe a challenging Python project you worked on. What was the architecture, and what specific Python features or libraries made it successful?

• Answer:
  (This is a behavioral question. Tailor your answer to your experience. Here is a template/structure.)

  • Project Goal: Briefly describe the project's purpose and scale.

  • Architecture: "We built a distributed data processing pipeline. It involved a Flask/FastAPI service for receiving data, a Celery queue with Redis/RabbitMQ for handling background tasks, and several worker services that used Pandas and NumPy for data transformation before loading it into a PostgreSQL database."

  • Key Python Features/Libraries:

    • asyncio/aiohttp: "We used these to build a highly concurrent ingestion service that could handle thousands of simultaneous connections."

    • pandas/numpy: "These were crucial for performing efficient data manipulation and numerical computations."

    • SQLAlchemy: "We used its ORM for safe and convenient database interactions and its core for complex, performance-critical queries."

    • pytest & unittest.mock: "This combination allowed us to build a comprehensive, reliable test suite that ran quickly."

  • Challenge & Solution: "The main challenge was memory usage in our workers. We solved it by using generators to stream large datasets instead of loading them entirely into memory, and we implemented __slots__ in key data-holding classes to reduce overhead."

Good luck with your interview! Remember to practice explaining these concepts out loud.