Introduction to how python works

#python #random

Introduction to Python's Internal Mechanics

Overview

Python is a high-level, dynamically typed, interpreted language known for its readability and flexibility. Under the hood, Python uses a combination of compilation, a virtual machine, and a dynamic object system that make it both powerful and adaptable.

How Python Executes Commands

Compilation to Bytecode

• Source Code Parsing: When you run a Python script, the interpreter first parses your code into an Abstract Syntax Tree (AST).

• Bytecode Generation: The AST is compiled into bytecode, a low-level, platform-independent representation.

• Execution by the Virtual Machine: This bytecode is executed by the Python Virtual Machine (PVM), which interprets the instructions and interacts with the underlying system.

example of how a python script is parsed into an Abstract Syntax Tree (AST)

#just a normal python code
x = 5 + 3
print(x)

then We can use Python's built-in ast module to see how this code is structured internally...

import ast
import astpretty

code = """
x = 5 + 3
print(x)
"""

# Parse into AST
tree = ast.parse(code)

# Pretty print the AST
astpretty.pprint(tree)

this how the output would be:

Module(
    body=[
        Assign(
            targets=[Name(id='x', ctx=Store())],
            value=BinOp(left=Constant(value=5), op=Add(), right=Constant(value=3))
        ),
        Expr(
            value=Call(
                func=Name(id='print', ctx=Load()),
                args=[Name(id='x', ctx=Load())],
                keywords=[]
            )
        )
    ]
)

• Explanation:

  • Module: The top-level structure of the Python script.

  • Assign: Represents an assignment (x = 5 + 3).

    • BinOp: A binary operation (5 + 3).

  • Expr: Represents an expression (print(x)).

    • Call: A function call (print(...)).

This AST representation is what Python processes before converting it into bytecode for execution.

The Role of the Interpreter

The Python interpreter first compiles source code into bytecode, which it then executes to run the program. This process enables dynamic features such as runtime modifications and introspection, supporting rapid development, flexible testing, and interactive execution.

The Python Object Model

Everything is an Object

In Python, all data types—integers, strings, functions, classes, etc.—are implemented as objects. This unified model allows Python to handle various data types in a consistent manner, supporting dynamic attribute assignment and introspection.

The object Class

• Base Class: object is the most fundamental base class in Python. Every new-style class (all classes in Python 3) implicitly inherits from object unless another superclass is specified.

• Default Behaviors: It provides implementations for many built-in methods (like __init__, __str__, and __eq__), which can be overridden by subclasses to customize behavior.

The type Class and Its Relationship with object

• Metaclass Functionality: type is not only the type of all classes but also serves as the default metaclass. This means that every class in Python is an instance of type including object class.

• Circular Relationship:

  • The object class is an instance of type.

  • The type class itself is a subclass of object.

  This might seem paradoxical at first, but it is a deliberate design that supports Python's dynamic and reflective capabilities.

• type is an instance of itself (type points to itself).

• type is a subclass of object (type(object)).

• object is the base of everything.

• object itself is instance of type class

• Example Demonstration:

  print(isinstance(object, type))  # Output: True
  print(issubclass(type, object))  # Output: True
  print(type(type))                # Output: <class 'type'>

Subclasses and Memory Management

Loading Subclasses into Memory

When a class is defined in Python, it is stored in memory as an instance of type. Subclasses inherit methods and attributes from their parent classes, and their structure is stored in memory at runtime.

How Python Loads Subclasses:

1. Definition: When a class is defined, Python creates an entry in memory for it.

2. Metaclass Assignment: The class is assigned to a metaclass (typically type).

3. Attribute Resolution: Python dynamically resolves attributes and methods by checking the method resolution order (MRO).

4. Object Instantiation: When an object of the class is created, memory is allocated for its instance variables.

Example:

class A:
    pass
    
class B(A):
    pass

print(isinstance(B, type))  # Output: True
print(issubclass(B, A))     # Output: True

• B is an instance of type and a subclass of A, inheriting all attributes.

• When B is created, Python loads it into memory and associates it with its parent class.

Memory Optimization with __slots__

By default, Python stores instance attributes in a dynamic dictionary (__dict__). However, using __slots__ can reduce memory usage by preventing the creation of an instance dictionary.

class C:
    __slots__ = ('x', 'y')

c = C()
c.x = 10
c.y = 20
# c.z = 30  # Raises AttributeError, since 'z' is not in __slots__

• __slots__ limits attributes to a predefined set, reducing memory overhead.

Conclusion

Python's architecture—compiling source code to bytecode and executing it on a virtual machine—combined with its robust object model, underpins its flexibility and power. The unique, circular relationship between object and type is a cornerstone of Python’s design, facilitating a dynamic and introspective programming environment. Subclasses are dynamically loaded into memory, with optimizations like __slots__ available for efficient memory usage.