Identifying Module Dependencies and Requirements Sets

🧭 Context Introduction

Before you can share your Python automation script with anyone—whether it's a teammate, a reviewer, or a production system—you need to know exactly what external modules your script relies on. Think of it like packing for a trip: you need to know which ingredients are required before you can run your recipe on a different kitchen. This section will help you identify, document, and manage those dependencies so your script runs smoothly everywhere.

⚙️ What Are Module Dependencies?

A module dependency is any external Python package or library that your script imports to function. If your script uses requests to make HTTP calls or pandas to process data, those are dependencies.

Key points to remember: - Python's standard library (like os, sys, json) is built-in and does not need to be listed as a dependency. - Any package you install via pip (like requests, boto3, paramiko) must be documented. - Dependencies can have their own dependencies—these are called transitive dependencies.

🕵️ How to Identify Dependencies in Your Script

There are two main approaches to find out what your script needs:

Manual inspection approach: - Open your script and look for all import statements at the top of the file. - Separate standard library imports from third-party imports. - For third-party imports, note the exact package name (often the same as the import name, but not always—for example, you import yaml but the package is PyYAML).

Automated approach using tools: - Run a tool like pipreqs or pip-tools to scan your script and generate a list automatically. - These tools parse your imports and match them to installed packages in your environment.

📊 The Requirements File: Your Dependency Manifest

The standard way to document dependencies is a requirements.txt file. This file lives in the root of your project and lists every package your script needs.

What a well-written requirements file includes: - Each package on its own line - Version pinning using operators like ==, >=, or ~= - Comments to explain why a specific version is needed

Example structure (not code, just a list): - requests==2.31.0 (pinned to exact version for stability) - boto3>=1.28.0 (any version 1.28 or newer is acceptable) - pyyaml~=6.0 (compatible with version 6.x)

🛠️ Requirements Sets: When One File Isn't Enough

As your automation scripts grow, you may need different dependency sets for different environments. This is where requirements sets come in.

Common requirements sets you might create:

Best practices for managing multiple sets: - Use a base file that other sets reference using the -r directive - Keep development dependencies separate from runtime dependencies - Document which set to use for which purpose in your project README

📋 Comparison: Manual vs Automated Dependency Management

✅ Checklist for Engineers New to Dependency Management

• [ ] Scan your script and list all import statements
• [ ] Identify which imports are third-party (not in Python's standard library)
• [ ] Check the installed version of each third-party package using pip list
• [ ] Create a requirements.txt file with pinned versions
• [ ] Test your script in a fresh virtual environment using pip install -r requirements.txt
• [ ] If you have multiple environments, create separate requirements sets
• [ ] Add a comment at the top of your requirements file explaining its purpose
• [ ] Commit the requirements file to your version control repository

💡 Pro Tips for Beginners

• Always use a virtual environment before generating your requirements file. This prevents capturing packages you don't actually need.
• If you're unsure whether a package is standard library or third-party, run pip show . If it returns details, it's third-party.
• For scripts that will run on different operating systems, test your requirements file on each OS before sharing.
• Consider using pip freeze to generate a complete list, but be aware it captures everything in your environment—not just what your script needs. Always trim it down.

🚀 Next Steps After Identifying Dependencies

Once you have your dependencies documented, you're ready to: - Share your script with confidence, knowing others can install everything they need - Set up automated testing that installs dependencies from your requirements file - Create a Docker container that uses your requirements file to build the environment - Hand off your project to another engineer with clear setup instructions

Dependency management is one of those small habits that separates a script from a professional automation tool. Master this, and your code becomes portable, reproducible, and truly shareable.

This topic shows how to find what external modules your Python script needs and how to list them in a requirements file.

🔧 Example 1: Checking if a module is installed

This example checks whether the requests module is available on your system.

import importlib
spec = importlib.util.find_spec("requests")
if spec is None:
    print("Module not installed")
else:
    print("Module is installed")

📤 Output: Module not installed (if requests is not installed)

🔧 Example 2: Listing all installed modules with pip

This example uses pip to show every module currently installed in your Python environment.

import subprocess
result = subprocess.run(["pip", "list"], capture_output=True, text=True)
print(result.stdout)

📤 Output: Package Version \n--- --- \npip 23.0.1 \nsetuptools 58.1.0 ... (partial list)

🔧 Example 3: Generating a requirements.txt file

This example creates a requirements file that lists all installed modules and their versions.

import subprocess
with open("requirements.txt", "w") as f:
    subprocess.run(["pip", "freeze"], stdout=f)
print("requirements.txt created")

📤 Output: requirements.txt created

🔧 Example 4: Finding dependencies for a specific script

This example scans a Python script and identifies which imported modules are not part of Python's standard library.

import ast
import sys
script_code = """
import os
import requests
import json
from datetime import datetime
"""
tree = ast.parse(script_code)
imports = []
for node in ast.walk(tree):
    if isinstance(node, ast.Import):
        for alias in node.names:
            imports.append(alias.name)
    elif isinstance(node, ast.ImportFrom):
        imports.append(node.module)
standard_libs = sys.stdlib_module_names
external_modules = [m for m in imports if m not in standard_libs]
print(external_modules)

📤 Output: ['requests']

🔧 Example 5: Creating a requirements file for a project folder

This example finds all Python files in a folder, collects their external imports, and writes a requirements file.

import os
import ast
import sys
import subprocess
project_folder = "./my_script_folder"
all_imports = set()
for root, dirs, files in os.walk(project_folder):
    for file in files:
        if file.endswith(".py"):
            filepath = os.path.join(root, file)
            with open(filepath, "r") as f:
                try:
                    tree = ast.parse(f.read())
                    for node in ast.walk(tree):
                        if isinstance(node, ast.Import):
                            for alias in node.names:
                                all_imports.add(alias.name)
                        elif isinstance(node, ast.ImportFrom):
                            all_imports.add(node.module)
                except:
                    pass
standard_libs = sys.stdlib_module_names
external_modules = [m for m in all_imports if m and m not in standard_libs]
with open("requirements.txt", "w") as f:
    for module in sorted(external_modules):
        f.write(f"{module}\n")
print("requirements.txt written with", len(external_modules), "modules")

📤 Output: requirements.txt written with 3 modules

Quick Comparison Table

🧭 Context Introduction

Before you can share your Python automation script with anyone—whether it's a teammate, a reviewer, or a production system—you need to know exactly what external modules your script relies on. Think of it like packing for a trip: you need to know which ingredients are required before you can run your recipe on a different kitchen. This section will help you identify, document, and manage those dependencies so your script runs smoothly everywhere.

⚙️ What Are Module Dependencies?

A module dependency is any external Python package or library that your script imports to function. If your script uses requests to make HTTP calls or pandas to process data, those are dependencies.

Key points to remember: - Python's standard library (like os, sys, json) is built-in and does not need to be listed as a dependency. - Any package you install via pip (like requests, boto3, paramiko) must be documented. - Dependencies can have their own dependencies—these are called transitive dependencies.

🕵️ How to Identify Dependencies in Your Script

There are two main approaches to find out what your script needs:

Manual inspection approach: - Open your script and look for all import statements at the top of the file. - Separate standard library imports from third-party imports. - For third-party imports, note the exact package name (often the same as the import name, but not always—for example, you import yaml but the package is PyYAML).

Automated approach using tools: - Run a tool like pipreqs or pip-tools to scan your script and generate a list automatically. - These tools parse your imports and match them to installed packages in your environment.

📊 The Requirements File: Your Dependency Manifest

The standard way to document dependencies is a requirements.txt file. This file lives in the root of your project and lists every package your script needs.

What a well-written requirements file includes: - Each package on its own line - Version pinning using operators like ==, >=, or ~= - Comments to explain why a specific version is needed

Example structure (not code, just a list): - requests==2.31.0 (pinned to exact version for stability) - boto3>=1.28.0 (any version 1.28 or newer is acceptable) - pyyaml~=6.0 (compatible with version 6.x)

🛠️ Requirements Sets: When One File Isn't Enough

As your automation scripts grow, you may need different dependency sets for different environments. This is where requirements sets come in.

Common requirements sets you might create:

Best practices for managing multiple sets: - Use a base file that other sets reference using the -r directive - Keep development dependencies separate from runtime dependencies - Document which set to use for which purpose in your project README

📋 Comparison: Manual vs Automated Dependency Management

✅ Checklist for Engineers New to Dependency Management

• [ ] Scan your script and list all import statements
• [ ] Identify which imports are third-party (not in Python's standard library)
• [ ] Check the installed version of each third-party package using pip list
• [ ] Create a requirements.txt file with pinned versions
• [ ] Test your script in a fresh virtual environment using pip install -r requirements.txt
• [ ] If you have multiple environments, create separate requirements sets
• [ ] Add a comment at the top of your requirements file explaining its purpose
• [ ] Commit the requirements file to your version control repository

💡 Pro Tips for Beginners

• Always use a virtual environment before generating your requirements file. This prevents capturing packages you don't actually need.
• If you're unsure whether a package is standard library or third-party, run pip show . If it returns details, it's third-party.
• For scripts that will run on different operating systems, test your requirements file on each OS before sharing.
• Consider using pip freeze to generate a complete list, but be aware it captures everything in your environment—not just what your script needs. Always trim it down.

🚀 Next Steps After Identifying Dependencies

Once you have your dependencies documented, you're ready to: - Share your script with confidence, knowing others can install everything they need - Set up automated testing that installs dependencies from your requirements file - Create a Docker container that uses your requirements file to build the environment - Hand off your project to another engineer with clear setup instructions

Dependency management is one of those small habits that separates a script from a professional automation tool. Master this, and your code becomes portable, reproducible, and truly shareable.

This topic shows how to find what external modules your Python script needs and how to list them in a requirements file.

🔧 Example 1: Checking if a module is installed

This example checks whether the requests module is available on your system.

import importlib
spec = importlib.util.find_spec("requests")
if spec is None:
    print("Module not installed")
else:
    print("Module is installed")

📤 Output: Module not installed (if requests is not installed)

🔧 Example 2: Listing all installed modules with pip

This example uses pip to show every module currently installed in your Python environment.

import subprocess
result = subprocess.run(["pip", "list"], capture_output=True, text=True)
print(result.stdout)

📤 Output: Package Version \n--- --- \npip 23.0.1 \nsetuptools 58.1.0 ... (partial list)

🔧 Example 3: Generating a requirements.txt file

This example creates a requirements file that lists all installed modules and their versions.

import subprocess
with open("requirements.txt", "w") as f:
    subprocess.run(["pip", "freeze"], stdout=f)
print("requirements.txt created")

📤 Output: requirements.txt created

🔧 Example 4: Finding dependencies for a specific script

This example scans a Python script and identifies which imported modules are not part of Python's standard library.

import ast
import sys
script_code = """
import os
import requests
import json
from datetime import datetime
"""
tree = ast.parse(script_code)
imports = []
for node in ast.walk(tree):
    if isinstance(node, ast.Import):
        for alias in node.names:
            imports.append(alias.name)
    elif isinstance(node, ast.ImportFrom):
        imports.append(node.module)
standard_libs = sys.stdlib_module_names
external_modules = [m for m in imports if m not in standard_libs]
print(external_modules)

📤 Output: ['requests']

🔧 Example 5: Creating a requirements file for a project folder

This example finds all Python files in a folder, collects their external imports, and writes a requirements file.

import os
import ast
import sys
import subprocess
project_folder = "./my_script_folder"
all_imports = set()
for root, dirs, files in os.walk(project_folder):
    for file in files:
        if file.endswith(".py"):
            filepath = os.path.join(root, file)
            with open(filepath, "r") as f:
                try:
                    tree = ast.parse(f.read())
                    for node in ast.walk(tree):
                        if isinstance(node, ast.Import):
                            for alias in node.names:
                                all_imports.add(alias.name)
                        elif isinstance(node, ast.ImportFrom):
                            all_imports.add(node.module)
                except:
                    pass
standard_libs = sys.stdlib_module_names
external_modules = [m for m in all_imports if m and m not in standard_libs]
with open("requirements.txt", "w") as f:
    for module in sorted(external_modules):
        f.write(f"{module}\n")
print("requirements.txt written with", len(external_modules), "modules")

📤 Output: requirements.txt written with 3 modules

Quick Comparison Table