Development Guidelines

Code Quality Standards

File Headers

All Python source files must include a copyright header:

################################################################################
# Copyright (c) 2026 Wind River Systems, Inc.
#
# SPDX-License-Identifier: Apache-2.0
#
################################################################################

Module Docstrings

Every Python module must have a docstring describing its purpose:

"""Module to parse CloudFront log files.

Provides functionality to parse gzipped CloudFront standard log files
and convert them into structured data objects.
"""

Import Organization

• Use from __future__ import annotations at the top of every Python file for forward compatibility
• Group imports in standard order: standard library, third-party, local modules
• Use explicit imports rather than wildcard imports

Example:

from __future__ import annotations

import datetime
import gzip
import logging
from pathlib import Path
from typing import Any, Generator

from elxr_metrics.cloudfront_log import parse_cloudfront_log

Code Formatting

• Line length: 120 characters maximum (configured in pyproject.toml)
• Indentation: 4 spaces (no tabs)
• Use Black formatter for consistent code style
• Follow PEP 8 conventions with project-specific overrides

Naming Conventions

• Functions and variables: snake_case
• Classes: PascalCase
• Constants: UPPER_CASE
• Private functions: prefix with underscore _function_name
• Module names: snake_case or any style allowed

Examples:

# Functions
def parse_cloudfront_log(file_path: Path) -> Generator:
    pass

def _to_datetime(value: str) -> datetime.datetime:
    pass

# Classes
class CloudFrontLogEntry:
    pass

# Constants
BUCKET_INTERVAL = 6

Type Annotations

• Use type hints for all function parameters and return values
• Use | None for optional types (Python 3.10+ union syntax)
• Use from __future__ import annotations to enable forward references

Examples:

def _to_datetime(value: str) -> datetime.datetime:
    """Convert string to datetime object."""
    return datetime.datetime.fromisoformat(value.rstrip("Z"))

def parse_cloudfront_log(file_path: Path) -> Generator[CloudFrontLogEntry, Any, None]:
    """Parse CloudFront log file and yield log entries."""
    pass

@dataclass(frozen=True)
class CloudFrontLogEntry:
    date: datetime.date | None = None
    time: datetime.time | None = None
    sc_bytes: int | None = None

Docstrings

Use Google-style docstrings with sections for Args, Returns, Raises, Note, and Yields:

def parse_cloudfront_log(file_path: Path) -> Generator[CloudFrontLogEntry, Any, None]:
    """Parse CloudFront log file and yield log entries.

    Args:
        file_path: Path to the gzipped CloudFront log file.

    Yields:
        CloudFrontLogEntry objects for each valid log line.

    Raises:
        Exception: If file_path does not exist or is not a file.

    Note:
        Skips comment lines (starting with #) and empty lines.
    """
    pass

Semantic Patterns

Dataclass Usage

Use frozen dataclasses for immutable data structures:

@dataclass(frozen=True)
class CloudFrontLogEntry:
    """CloudFront log entry data structure.

    Represents a single log entry from CloudFront standard logs with all
    available fields as optional attributes.
    """
    date: datetime.date | None = None
    time: datetime.time | None = None
    x_edge_location: str | None = None

Pattern Frequency: Used in 100% of data model definitions

Property Methods

Use @property decorator for computed attributes:

@property
def timestamp(self) -> datetime.datetime:
    """Get the combined datetime representation of date and time fields.

    Returns:
        UTC datetime combining the date and time fields, or minimum datetime if fields are None.
    """
    return datetime.datetime.combine(
        self.date or datetime.date.min, self.time or datetime.time.min, datetime.timezone.utc
    )

Pattern Frequency: Used for all computed attributes in dataclasses

Generator Functions

Use generators for processing large datasets:

def parse_cloudfront_log(file_path: Path) -> Generator[CloudFrontLogEntry, Any, None]:
    """Parse CloudFront log file and yield log entries."""
    with gzip.open(file_path, "rt", encoding="utf-8") as file:
        for line in file:
            if line.startswith("#") or not line:
                continue
            col = line.split("\t")
            yield CloudFrontLogEntry(*[_to_object(value, field) for value, field in zip(col, model_fields)])

Pattern Frequency: Used in all log parsing functions

Context Managers

Always use context managers for file operations:

# Python - file operations
with gzip.open(file_path, "rt", encoding="utf-8") as file:
    for line in file:
        process(line)

# Python - temporary files
with tempfile.NamedTemporaryFile(mode="w", delete=False) as tmp_file:
    tmp_file.write(content)
    tmp_file_path = Path(tmp_file.name)

Pattern Frequency: 100% of file operations use context managers

Error Handling

Use specific exception types and provide meaningful error messages:

def _to_object(value: str, field: Field):
    """Convert string value to appropriate Python object based on field type.

    Raises:
        ValueError: If field type is not handled.
    """
    if field_type == "str":
        return value
    if field_type == "int":
        return int(value)
    raise ValueError(f"unhandled value:type {value}:{field_type}")

Pattern Frequency: All functions that can fail include explicit error handling

Logging

Use module-level logger with appropriate log levels:

import logging

logger = logging.getLogger(__name__)

def parse_cloudfront_log(file_path: Path):
    logger.info("parsing cloudfront log file: {%s}", file_path)
    try:
        # processing logic
    except Exception as e:
        logger.warning("failed to parse: {%s}", file_path)
        logger.exception("parsing error: %s", e)

Pattern Frequency: All modules that perform I/O or complex operations use logging

Argument Parsing

Use argparse with custom validation functions:

def is_dir(parser: argparse.ArgumentParser, path: str) -> Path:
    """Check if path is a directory.

    Args:
        parser: The argument parser instance for error reporting.
        path: The path string to validate.

    Returns:
        Path object if valid directory.

    Raises:
        SystemExit: If path is empty, doesn't exist, or is not a directory.
    """
    if not path:
        parser.error("The path is empty!")
    d = Path(path)
    if not d.exists():
        parser.error(f"The path does not exist! ({path})")
    elif not d.is_dir():
        parser.error(f"The path is not directory! ({path})")
    return d

parser.add_argument(
    "log_path",
    nargs=1,
    type=lambda x: is_dir(parser, x),
    help="the directory contains log files",
)

Pattern Frequency: Used in all CLI entry points

Path Handling

Use pathlib.Path for all file system operations:

from pathlib import Path

def process_logs(log_path: Path, csv_path: Path):
    """Process log files from directory."""
    if not log_path.exists():
        raise FileNotFoundError(f"Log path not found: {log_path}")

    for log_file in log_path.glob("*.gz"):
        process_file(log_file)

Pattern Frequency: 100% of file operations use pathlib.Path

JavaScript Patterns

Variable Declarations

Use const for immutable references, let for mutable variables:

const dateRangeSelect = document.getElementById('date-range');
const chartCanvas = document.getElementById('my-chart');
let chartData = [];
let viewChart;

Pattern Frequency: Used consistently throughout JavaScript code

Event Listeners

Use addEventListener for event handling:

window.addEventListener('load', () => {
    fetch('elxr_org_view.csv')
        .then(response => response.text())
        .then(csvData => {
            chartData = parseCSV(csvData);
            drawChart(chartData);
        })
        .catch(error => {
            console.error('Error fetching CSV data:', error);
        });
});

startDateInput.addEventListener('change', rangeChange);

Pattern Frequency: All event handling uses addEventListener

Async/Await Pattern

Use async/await for asynchronous operations:

async function loadCountryData() {
    try {
        const response = await fetch('country.csv');
        const csvData = await response.text();
        countryData = parseCSV(csvData);
        return countryData;
    } catch (error) {
        console.error('Error fetching CSV data:', error);
    }
}

Pattern Frequency: Used for all fetch operations

Chart.js Integration

Destroy existing charts before creating new ones to prevent memory leaks:

function drawChart(data) {
    const ctx = chartCanvas.getContext('2d');
    if (viewChart) {
        viewChart.destroy();  // Destroy existing chart to prevent duplication
    }
    viewChart = new Chart(ctx, {
        type: 'line',
        data: {
            labels: data.map(item => item.TimeBucket),
            datasets: [/* ... */]
        },
        options: {/* ... */}
    });
}

Pattern Frequency: Used in all chart rendering functions

CSV Parsing

Normalize line endings before parsing:

function parseCSV(csvData) {
    const normalized = csvData.replace(/\r\n|\r/g, '\n');
    const lines = normalized.split('\n');
    const headers = lines[0].split(',');
    const dataArray = [];
    for (let i = 1; i < lines.length; i++) {
        const values = lines[i].split(',');
        const item = {};
        for (let j = 0; j < headers.length; j++) {
            item[headers[j]] = values[j];
        }
        dataArray.push(item);
    }
    return dataArray;
}

Pattern Frequency: Standard pattern for CSV parsing

Testing Patterns

Pytest Fixtures

Use fixtures for reusable test setup:

@pytest.fixture
def list_str_field():
    """Fixture to temporarily modify field type and restore it after test."""
    mock_field = CloudFrontLogEntry.__dataclass_fields__["date"]
    original_type = mock_field.type
    object.__setattr__(mock_field, "type", "list[str]")
    yield mock_field
    object.__setattr__(mock_field, "type", original_type)

Pattern Frequency: Used for complex test setup requiring cleanup

Parametrized Tests

Use pytest.mark.parametrize for testing multiple inputs:

@pytest.mark.parametrize(
    "value, field, result",
    [
        ("2024-01-01", CloudFrontLogEntry.__dataclass_fields__["date"], datetime.date(2024, 1, 1)),
        ("200", CloudFrontLogEntry.__dataclass_fields__["sc_status"], 200),
        ("-", CloudFrontLogEntry.__dataclass_fields__["cs_method"], None),
    ],
)
def test_to_object(value, field, result):
    actual = _to_object(value, field)
    assert actual == result

Pattern Frequency: Used in 80% of test functions for comprehensive coverage

Temporary Files in Tests

Use tempfile and pathlib for test file creation:

def test_parse_log_empty_file(tmp_path):
    """Test parsing empty log file."""
    log_file = tmp_path / "empty.gz"
    with gzip.open(log_file, "wt") as f:
        f.write("")

    entries = list(parse_cloudfront_log(log_file))
    assert len(entries) == 0

Pattern Frequency: Used in all tests requiring file I/O

Exception Testing

Use pytest.raises for testing exceptions:

def test_to_datetime_invalid():
    """Test _to_datetime with an invalid datetime string. Should raise ValueError."""
    dt_str = "invalid-datetime"
    with pytest.raises(ValueError):
        _to_datetime(dt_str)

Pattern Frequency: Used for all negative