Language Concepts

Python aims to be simple, readable, and easy to learn. That simplicity comes with trade-offs and idiosyncrasies. These are my notes on the idioms and conventions behind idiomatic Python (a.k.a pythonic code).

Data Types & Data Structures

Python supports all the standard data types, including integers, floats, and strings, which are used to store and manipulate individual pieces of data in your code.

Additionally, Python includes several built-in data structures, such as lists, tuples, dictionaries, and sets, which are used to organize and manage collections of data efficiently.

# A list is a collection of items that are ordered and changeable (mutable).
# In other languages, it can be similar to a dynamic array or slice.
my_list = [1, 2, 3]

# A tuple is a collection of items that are ordered and unchangeable (immutable).
my_tuple = (1, 2, 3)

# A dictionary is a collection of key-value pairs that are ordered (as of Python 3.7),
# changeable (mutable), and indexed. In other languages, it can be similar to a hash table or associative array.
my_dict = {'a': 1, 'b': 2}

# A set is a collection of unique items that are unordered and unindexed.
# In other languages, it can be similar to a hash set.
my_set = {1, 2, 3}

Indentation and Code Blocks

Python uses indentation to define blocks of code, not curly braces {} or begin/end keywords. The structure of the code is literally its indentation, so consistency is not optional. Odd at first coming from brace languages, but it does force readable code. Example:

def check_number(num):
    if num > 0:
        print("The number is positive.")
    elif num == 0:
        print("The number is zero.")
    else:
        print("The number is negative.")

# Main block of the script
if __name__ == "__main__":
    # Input: Enter a number
    number = int(input("Enter a number: "))
    # Call the function to check the number
    check_number(number)

Dynamic Typing and Duck Typing

Python is dynamically typed: no type declarations, the type of a variable is determined at runtime from whatever value gets assigned. Easier to read, but two costs: the code runs slower, and bugs that a compiler or static analysis would catch elsewhere only show up at runtime here.

Duck typing follows from that. What matters is what an object can do, not what it is. If it has the method, the call works, no inheritance required. From the saying “If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.”. Powerful, but also a source of runtime surprises. Example:

# Duck typing example
class Duck:
    def quack(self):
        print("Quack!")

class Person:
    def quack(self):
        print("I can quack like a duck!")

def make_it_quack(duck):
    duck.quack()

# Dynamic typing example
def print_type_and_value(val):
    if isinstance(val, int):
        print(f"Type: int, Value: {val}")
    elif isinstance(val, str):
        print(f"Type: string, Value: {val}")
    elif isinstance(val, bool):
        print(f"Type: bool, Value: {val}")
    else:
        print("Unknown type")

if __name__ == "__main__":
    duck = Duck()
    person = Person()

    make_it_quack(duck)
    make_it_quack(person)

    print_type_and_value(42)
    print_type_and_value("Hello, World!")
    print_type_and_value(True)

Functions

Functions are declared with def, the function name, parameters in parentheses, and a colon. The body is indented and returns with the return keyword. Example:

def add(a, b):
    return a + b

result = add(2, 3)
print(result)

Python functions support several kinds of arguments: positional, keyword, default, and variable-length. Take this signature:

def my_function(positional, keyword=value, *args, **kwargs):
    pass

The pass keyword is the common way to define an empty block of code, a placeholder for a function or class to be implemented later.

positional is required for this function.
keyword is optional, since it has a default value.
*args is a tuple of variable-length positional arguments.
**kwargs is a dictionary of variable-length keyword arguments.

Positional Arguments

Positional arguments are passed by position: order matters, and the count has to match. Example:

def greet(name, message):
    print(f"{message}, {name}!")

greet("John", "Hello")
greet("Jane", "Goodbye")

Positional arguments can also be passed by name at the call site. Example:

greet(name="Alice", message="Hi")

Considered an anti-pattern though; for functions with few arguments, plain positional style reads better.

Keyword Arguments

Keyword arguments are passed by name rather than by position. They earn their keep in functions with many arguments, where a bare positional call stops being readable. Example:

def greet(name, message="Hello"):
    print(f"{message}, {name}!")

greet("John")
greet("Jane", "Goodbye")
greet(message="Hi", name="Alice")

Default Arguments

Default arguments take a fallback value when the caller omits them, and can still be overridden. Example:

def greet(name, message="Hello"):
    print(f"{message}, {name}!")

greet("John")
greet("Jane", "Goodbye")

Variable-Length Arguments

*args and **kwargs accept a variable number of arguments, handy for functions that take any number of inputs or forward their arguments to another function. Example:

def sum(*args):
    total = 0
    for arg in args:
        total += arg
    return total

print(sum(1, 2, 3, 4, 5))

def print_info(**kwargs):
    for key, value in kwargs.items():
        print(f"{key}: {value}")

print_info(name="Alice", age=30, city="New York")

def print_all(*args, **kwargs):
    for arg in args:
        print(arg)
    for key, value in kwargs.items():
        print(f"{key}: {value}")

print_all(1, 2, 3, name="Alice", age=30)

Functions and Variables scoping

Python uses lexical scoping: a variable’s scope is determined by where it’s defined. Variables defined inside a function are local to that function; making one visible outside requires declaring it global.

Scoping follows the LEGB rule: Local, Enclosing, Global, Built-in. Python resolves a name by searching these scopes in order:

Local: Variables that are defined inside the current function.
Enclosing: Variables that are defined in the enclosing function (if any).
Global: Variables that are defined at the top level of the module.
Built-in: Variables that are built into Python (like len, range, etc.).

If the variable is not found in any of these scopes, Python will raise a NameError.

Example:

# Global scope
global_var = "global"

def outer_function():
    # Enclosing scope
    enclosing_var = "enclosing"

    # nested function
    def inner_function():
        # Local scope
        local_var = "local"
        print("Inner function:", local_var)

    inner_function()
    print("Outer function:", enclosing_var)

outer_function()
print("Global scope:", global_var)

# Modifying global variable
def modify_global():
    # here the keyword global is used to modify the global variable
    # without it, the variable would be treated as a local variable
    global global_var
    global_var = "modified global"

modify_global()
print("Modified Global scope:", global_var)

# Accessing a built-in scope
def access_builtin():
    print("Built-in scope (length of list):", len([1, 2, 3]))

access_builtin()

The nonlocal keyword modifies a variable in the enclosing scope, without promoting it all the way to global.

def outer_function():
    outer_var = "I am outer"

    def inner_function():
        # here the keyword nonlocal is used to modify the variable in the enclosing scope
        nonlocal outer_var
        outer_var = "I have been modified by inner"
        print(outer_var)

    inner_function()
    print(outer_var)

outer_function()

The built-in scope holds everything predefined in the language (len, range, etc.), available everywhere with no import.

def print_builtin():
    # Using the built-in len function
    print("Length of the list:", len([1, 2, 3]))

print_builtin()

List Comprehensions

List comprehensions build a new list by applying an expression to each item of an iterable. Usually more readable and faster than the equivalent for loop. Example:

# Using a for loop
squares = []
for i in range(10):
    squares.append(i ** 2)

print("Squares using for loop:", squares)

# Using list comprehension
squares = [i ** 2 for i in range(10)]
print("Squares using list comprehension:", squares)

They also take a condition to filter items. Example:

# Using a for loop
even_squares = []
for i in range(10):
    if i % 2 == 0:
        even_squares.append(i ** 2)

print("Even squares using for loop:", even_squares)

# Using list comprehension
even_squares = [i ** 2 for i in range(10) if i % 2 == 0]
print("Even squares using list comprehension:", even_squares)

Iterators and Generators

Generators are a concise way to create iterators. Like a list comprehension, but instead of materializing a list they yield items one at a time, which keeps memory flat on large datasets.

A generator is just a function containing the yield keyword; calling it returns a generator object to iterate over. Example:

# Using a generator function
def squares(n):
    for i in range(n):
        yield i ** 2

# Using a generator expression
squares_gen = (i ** 2 for i in range(10))

print("Squares using generator function:")
for square in squares(10):
    print(square)

print("Squares using generator expression:")
for square in squares_gen:
    print(square)

Under the hood, an iterator is any object implementing __iter__ and __next__. Generators implement both automatically; writing them by hand looks like this:

class UpperCaseIterator:
    def __init__(self, strings):
        self.strings = strings
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index < len(self.strings):
            result = self.strings[self.index].upper()
            self.index += 1
            return result
        else:
            raise StopIteration

# List of strings to iterate over
strings = ["hello", "world", "python", "iterator"]

# Create an instance of the iterator
uppercase_strings = UpperCaseIterator(strings)

# Use the iterator
for string in uppercase_strings:
    print(string)

# Output:
# HELLO
# WORLD
# PYTHON
# ITERATOR

Decorators

Decorators modify or extend the behavior of a function without touching its code, applied with the @ symbol above the definition. Example:

# Decorator function
def uppercase_decorator(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result.upper()
    return wrapper

# Using the decorator
@uppercase_decorator
def greet(name):
    return f"Hello, {name}!"

print(greet("John"))

Decorators can also take arguments, at the cost of one more level of nesting. Example:

# Decorator function with arguments
def repeat_decorator(n):
    def decorator(func):
        def wrapper(*args, **kwargs):
            result = func(*args, **kwargs)
            return result * n
        return wrapper
    return decorator

# Using the decorator with arguments
@repeat_decorator(3)
def greet(name):
    return f"Hello, {name}!"

print(greet("John"))

Context Managers

Context managers handle resources like files, network connections, or database connections: acquire on entry, release on exit, no manual cleanup to forget. Used via the with statement, and created either with the contextlib module or a class with __enter__ and __exit__ methods.

The built-in open function is the classic example, it opens the file and closes it when the block exits:

# Here we are using the built-in open function as a context manager.
# This will automatically close the file when the block is exited.
# Notice the `with` keyword
with open("example.txt", "w") as f:
    f.write("Hello, World!")

A custom one as a class:

class FileManager:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode
        self.file = None

    def __enter__(self):
        self.file = open(self.filename, self.mode)
        return self.file

    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.file:
            self.file.close()

# Using the custom context manager
with FileManager('example.txt', 'w') as file:
    file.write('Hello, World!')

Or the shorter route, contextlib with a generator function:

from contextlib import contextmanager

@contextmanager
def file_manager(filename, mode):
    file = open(filename, mode)
    try:
        yield file
    finally:
        file.close()

# Using the context manager
with file_manager('example.txt', 'w') as file:
    file.write('Hello, World!')

Lambda Functions

Lambdas are anonymous functions: any number of arguments, exactly one expression, no def needed. Example:

# Using a lambda function
add = lambda x, y: x + y
print(add(2, 3))

# Using a lambda function with a list comprehension
squares = [(lambda x: x ** 2)(i) for i in range(10)]
print(squares)

Where they show up most: as the function argument to map, filter, and reduce. Example:

# Using the map function with a lambda function
numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x ** 2, numbers))
print(squared)

# Using the filter function with a lambda function
even = list(filter(lambda x: x % 2 == 0, numbers))
print(even)

# Using the reduce function with a lambda function
from functools import reduce
sum = reduce(lambda x, y: x + y, numbers)
print(sum)

Modules and Packages

Python code is organized into 📄 modules: files defining functions, classes, and variables for other modules to use, imported with the import keyword and accessed with dot notation. Example:

# Importing a module
import math

# Using a function from the math module
print(math.sqrt(16))

Modules group into 📁 packages: directories holding modules and, possibly, other packages, forming a hierarchy imported with dot notation.

A package is essentially a directory with modules plus an __init__.py file, which can be empty or hold initialization code. The hierarchy structures the module namespace.

my_package/
    # This file indicates that the directory should be treated as a package.
    __init__.py
    module1.py
    module2.py

Importing Modules from a Package:

from my_package import module1
from my_package import module2

# specific functions or variables can also be imported from a module:
# from my_package.module1 import function1
# from my_package.module2 import function2

module1.function1()
module2.function2()

Object-Oriented Programming

Python is an object-oriented programming language, which means that it supports the creation of classes and objects.

class Dog:
    def __init__(self, name):
        self.name = name

    def bark(self):
        print(f"{self.name} says woof!")

my_dog = Dog("Buddy")
my_dog.bark()

Error Handling

Python handles errors with Exceptions, caught and handled through try, except, and finally blocks. Example:

try:
    # code that may raise an exception
    x = 1 / 0
except ZeroDivisionError:
    # handle the exception
    print("Cannot divide by zero!")
finally:
    # code that will always run
    print("Done!")

Custom exceptions are just classes inheriting from Exception. Example:

class MyError(Exception):
    pass

And raised with the raise keyword to signal a specific error condition. Example:

def divide(x, y):
    if y == 0:
        raise ZeroDivisionError("Cannot divide by zero!")
    return x / y

try:
    result = divide(1, 0)
except ZeroDivisionError as e:
    print(e)

EAFP: Easier to Ask Forgiveness than Permission

Coming from other languages, my instinct is to check that an operation is safe before doing it. Turns out the pythonic way is the opposite: just do it, and handle the exception if it fails. This is called EAFP (Easier to Ask Forgiveness than Permission), as opposed to LBYL (Look Before You Leap).

person = {"name": "Alice"}

# LBYL: check first, then act
if "age" in person:
    age = person["age"]
else:
    age = None

# EAFP: act first, handle failure
try:
    age = person["age"]
except KeyError:
    age = None

At first I thought these were equivalent, but EAFP has a real advantage: no gap between the check and the action. With LBYL, things can change in between. The classic example is checking if a file exists before opening it:

import os

# LBYL: the file could be deleted between the check and the open
if os.path.exists("config.txt"):
    with open("config.txt") as f:
        data = f.read()

# EAFP: no gap, so no race condition
try:
    with open("config.txt") as f:
        data = f.read()
except FileNotFoundError:
    data = ""

Also interesting: a try block costs almost nothing unless an exception is actually raised, so EAFP is often faster when the operation usually succeeds. It fits duck typing too, instead of checking isinstance(obj, Duck), just call obj.quack() and catch the AttributeError.

It’s a preference, not a rule though. When the failure case is common or the check reads more clearly (like if y == 0 before dividing), LBYL is fine. One thing to watch out for: always catch specific exceptions, a bare except: swallows bugs I actually want to see.

EAFP is baked into the language itself: dict.get() is a shortcut for the KeyError dance above, and every for loop works by calling next() until StopIteration is raised.