Python Backend Development Skill

File Organization

This skill uses a split structure for HIGH-RISK requirements:

• SKILL.md: Core principles, patterns, and essential security (this file)
• references/security-examples.md: Complete CVE details and OWASP implementations
• references/advanced-patterns.md: Advanced Python patterns and optimization
• references/threat-model.md: Attack scenarios and STRIDE analysis

Validation Gates

1. Overview

Risk Level: HIGH

Justification: Python backend services handle authentication, database access, file operations, and external API communication. Vulnerabilities in input validation, deserialization, command execution, and cryptography can lead to data breaches and system compromise.

You are an expert Python backend developer specializing in secure, maintainable, and performant services.

Core Expertise Areas

• Type annotations and runtime validation
• Async programming with asyncio
• Security: input validation, cryptography, secrets management
• Testing: pytest, property-based testing, security testing
• Database access with SQLAlchemy/asyncpg
• API development with FastAPI/Starlette

2. Core Responsibilities

Fundamental Principles

1. TDD First: Write tests before implementation, design API through test cases
2. Performance Aware: Use async, generators, efficient data structures by default
3. Type Safety: Use type hints everywhere, validate at runtime boundaries
4. Defense in Depth: Multiple validation layers, fail securely
5. Secure Defaults: Use safe libraries, reject unsafe operations
6. Explicit over Implicit: Clear error handling, explicit dependencies
7. Testability: Design for testing, write security tests

Decision Framework

2.1 Implementation Workflow (TDD)

Step 1: Write Failing Test First

python
1import pytest
2from my_service import UserService, UserNotFoundError
3
4class TestUserService:
5    @pytest.mark.asyncio
6    async def test_get_user_returns_user_when_exists(self, db_session):
7        service = UserService(db_session)
8        user_id = await service.create_user("alice", "alice@example.com")
9        user = await service.get_user(user_id)
10        assert user.username == "alice"
11
12    @pytest.mark.asyncio
13    async def test_get_user_raises_when_not_found(self, db_session):
14        service = UserService(db_session)
15        with pytest.raises(UserNotFoundError):
16            await service.get_user(99999)
17
18    @pytest.mark.asyncio
19    async def test_create_user_validates_email(self, db_session):
20        service = UserService(db_session)
21        with pytest.raises(ValueError, match="Invalid email"):
22            await service.create_user("bob", "not-an-email")

Step 2: Implement Minimum to Pass

python
1class UserNotFoundError(Exception): pass
2
3class UserService:
4    def __init__(self, db: AsyncSession):
5        self.db = db
6
7    async def get_user(self, user_id: int) -> User:
8        user = await self.db.get(User, user_id)
9        if not user:
10            raise UserNotFoundError(f"User {user_id} not found")
11        return user
12
13    async def create_user(self, username: str, email: str) -> int:
14        if "@" not in email:
15            raise ValueError("Invalid email format")
16        # ... minimal implementation to pass tests

Step 3: Refactor if Needed

• Extract common patterns, add type hints, ensure errors don't leak internals

Step 4: Run Full Verification

bash
1pytest --cov=src           # All tests pass
2mypy src/ --strict         # Type check passes
3bandit -r src/ -ll         # Security scan passes
4pip-audit && safety check  # Dependencies clean

2.2 Performance Patterns

Pattern 1: Async I/O with asyncio.gather

python
1# BAD: Sequential requests (slow)
2for url in urls:
3    response = await client.get(url)  # Waits for each one
4
5# GOOD: Concurrent requests with gather
6tasks = [client.get(url) for url in urls]
7responses = await asyncio.gather(*tasks)  # All at once

Pattern 2: Generators for Large Data Processing

python
1# BAD: Load all into memory
2return [process(line) for line in f.readlines()]  # OOM risk
3
4# GOOD: Generator yields one at a time
5def process_large_file(filepath: str) -> Iterator[dict]:
6    with open(filepath) as f:
7        for line in f:
8            yield process(line)  # Memory efficient

Pattern 3: Efficient Data Structures

python
1# BAD: List for membership testing - O(n)
2required in user_perms_list  # Slow for large lists
3
4# GOOD: Set for membership testing - O(1)
5required in user_perms_set  # Fast lookup
6
7# BAD: Repeated string concatenation
8result = ""; for f in fields: result += f + ", "  # Creates new string each time
9
10# GOOD: Join for string building
11", ".join(fields)  # Single allocation

Pattern 4: Connection Pooling

python
1# BAD: New connection per request
2engine = create_async_engine(DATABASE_URL)  # Connection overhead each time
3
4# GOOD: Reuse pooled connections
5engine = create_async_engine(DATABASE_URL, pool_size=20, max_overflow=10)
6async_session = sessionmaker(engine, class_=AsyncSession)
7
8async def get_user(user_id: int):
9    async with async_session() as session:  # Reuses pooled connection
10        return await session.get(User, user_id)

Pattern 5: Batch Database Operations

python
1# BAD: Individual inserts (N round trips)
2for user in users:
3    db.add(User(**user)); await db.commit()  # N commits = slow
4
5# GOOD: Batch insert (1 round trip)
6stmt = insert(User).values(users)
7await db.execute(stmt); await db.commit()  # Single commit
8
9# GOOD: Chunked for very large datasets
10for i in range(0, len(users), 1000):
11    await db.execute(insert(User).values(users[i:i+1000]))
12await db.commit()

3. Technical Foundation

Version Recommendations

Security Dependencies

toml
1# pyproject.toml
2[project]
3dependencies = [
4    "pydantic>=2.0", "email-validator>=2.0",      # Validation
5    "cryptography>=41.0", "argon2-cffi>=21.0",    # Cryptography
6    "PyJWT>=2.8", "sqlalchemy>=2.0", "asyncpg>=0.28",
7    "httpx>=0.25", "bandit>=1.7",
8]
9
10[project.optional-dependencies]
11dev = ["pytest>=7.0", "pytest-asyncio>=0.21", "hypothesis>=6.0", "safety>=2.0", "pip-audit>=2.0"]

4. Implementation Patterns

Pattern 1: Type-Safe Input Validation

python
1from pydantic import BaseModel, Field, field_validator, EmailStr
2from typing import Annotated
3import re
4
5class UserCreate(BaseModel):
6    """Validated user creation request."""
7    username: Annotated[str, Field(min_length=3, max_length=50)]
8    email: EmailStr
9    password: Annotated[str, Field(min_length=12)]
10
11    @field_validator('username')
12    @classmethod
13    def validate_username(cls, v: str) -> str:
14        if not re.match(r'^[a-zA-Z0-9_-]+$', v):
15            raise ValueError('Username must be alphanumeric')
16        return v
17
18    @field_validator('password')
19    @classmethod
20    def validate_password_strength(cls, v: str) -> str:
21        if not all([re.search(r'[A-Z]', v), re.search(r'[a-z]', v), re.search(r'\d', v)]):
22            raise ValueError('Password needs uppercase, lowercase, and digit')
23        return v

Pattern 2: Secure Password Hashing

python
1from argon2 import PasswordHasher
2from argon2.exceptions import VerifyMismatchError
3
4ph = PasswordHasher(time_cost=3, memory_cost=65536, parallelism=4)
5
6def hash_password(password: str) -> str:
7    return ph.hash(password)
8
9def verify_password(password: str, hash: str) -> bool:
10    try:
11        ph.verify(hash, password)
12        return True
13    except VerifyMismatchError:
14        return False

Pattern 3: Safe Database Queries

python
1from sqlalchemy import select, text
2from sqlalchemy.ext.asyncio import AsyncSession
3
4# NEVER: f"SELECT * FROM users WHERE username = '{username}'"
5
6async def get_user_safe(db: AsyncSession, username: str) -> User | None:
7    stmt = select(User).where(User.username == username)
8    result = await db.execute(stmt)
9    return result.scalar_one_or_none()
10
11async def search_users(db: AsyncSession, pattern: str) -> list:
12    stmt = text("SELECT * FROM users WHERE username LIKE :pattern")
13    result = await db.execute(stmt, {"pattern": f"%{pattern}%"})
14    return result.fetchall()

Pattern 4: Safe File Operations

python
1from pathlib import Path
2
3def safe_read_file(base_dir: Path, user_filename: str) -> str:
4    if '..' in user_filename or user_filename.startswith('/'):
5        raise ValueError("Invalid filename")
6
7    file_path = (base_dir / user_filename).resolve()
8    if not file_path.is_relative_to(base_dir.resolve()):
9        raise ValueError("Path traversal detected")
10
11    return file_path.read_text()

Pattern 5: Safe Subprocess Execution

python
1import subprocess
2
3ALLOWED_PROGRAMS = {'git', 'python', 'pip'}
4
5def run_command_safe(program: str, args: list[str]) -> str:
6    if program not in ALLOWED_PROGRAMS:
7        raise ValueError(f"Program not allowed: {program}")
8
9    result = subprocess.run(
10        [program, *args],
11        capture_output=True, text=True, timeout=30, check=True,
12    )
13    return result.stdout

5. Security Standards

5.1 Domain Vulnerability Landscape

See references/security-examples.md for complete CVE details and mitigation code

5.2 OWASP Top 10 Mapping

5.3 Essential Security Patterns

python
1from pydantic import BaseModel, field_validator
2import os, logging
3
4# Secure base model - reject unknown fields, strip whitespace
5class SecureInput(BaseModel):
6    model_config = {'extra': 'forbid', 'str_strip_whitespace': True}
7
8    @field_validator('*', mode='before')
9    @classmethod
10    def reject_null_bytes(cls, v):
11        if isinstance(v, str) and '\x00' in v:
12            raise ValueError('Null bytes not allowed')
13        return v
14
15# Secrets from environment (NEVER hardcode)
16API_KEY = os.environ["API_KEY"]
17DB_URL = os.environ["DATABASE_URL"]
18
19# Safe error handling - log details, return safe message
20class AppError(Exception):
21    def __init__(self, message: str, internal: str = None):
22        self.message = message
23        if internal:
24            logging.error(f"{message}: {internal}")
25
26    def to_response(self) -> dict:
27        return {"error": self.message}

See references/advanced-patterns.md for secrets manager integration

6. Testing & Validation

Security Testing Commands

bash
1bandit -r src/ -ll          # Static analysis
2pip-audit && safety check   # Dependency vulnerabilities
3mypy src/ --strict          # Type checking

Security Test Examples

python
1import pytest
2from pathlib import Path
3
4def test_sql_injection_prevented(db):
5    for payload in ["'; DROP TABLE users; --", "' OR '1'='1", "admin'--"]:
6        assert get_user_safe(db, payload) is None
7
8def test_path_traversal_blocked():
9    base = Path("/app/data")
10    for attack in ["../etc/passwd", "..\\windows\\system32", "foo/../../etc/passwd"]:
11        with pytest.raises(ValueError, match="traversal|Invalid"):
12            safe_read_file(base, attack)
13
14def test_command_injection_blocked():
15    with pytest.raises(ValueError, match="not allowed"):
16        run_command_safe("rm", ["-rf", "/"])

See references/security-examples.md for comprehensive test patterns

7. Common Mistakes & Anti-Patterns

8. Pre-Deployment Checklist

Phase 1: Before Writing Code

• Requirements understood and documented
• API design reviewed (inputs, outputs, errors)
• Security threat model considered
• Test cases written first (TDD)
• Edge cases and error scenarios identified

Phase 2: During Implementation

• Following TDD workflow (test -> implement -> refactor)
• Using performance patterns (async, generators, pooling)
• All inputs validated with Pydantic
• DB queries parameterized/ORM
• File ops check path containment
• Subprocess uses list args
• Passwords hashed with Argon2id
• Secrets from environment only

Phase 3: Before Committing

• All tests pass: pytest --cov=src
• Type check passes: mypy src/ --strict
• Security scan passes: bandit -r src/ -ll
• Dependency audit passes: pip-audit && safety check
• No hardcoded secrets in code
• Errors don't leak internal details
• Debug mode disabled
• Logging configured (no PII/secrets)

9. Summary

Create Python code that is type safe, secure, testable, and maintainable.

Security Essentials:

1. Validate and sanitize all user input
2. Use parameterized queries for database ops
3. Never use shell=True with user input
4. Hash passwords with Argon2id
5. Load secrets from environment
6. Keep dependencies updated and audited

For attack scenarios and threat modeling, see references/threat-model.md