---

name: coding-guidelines description: Standardized Python & Stata coding practices for empirical research projects

Research Project Coding Guidelines

Version: 1.0 Last Updated: January 2026 Purpose: Standardized coding practices for empirical research projects

---

Table of Contents

1. Project Structure
2. Python Guidelines
3. Stata Guidelines
4. General Best Practices
5. Quick Reference Templates

---

Project Structure

Directory Organization

ProjectName/
├── Code/                           # All analysis scripts
│   ├── [Number]_[Name].py         # Data processing (Python)
│   ├── AN_[Number]_[Name].do      # Analysis scripts (Stata)
│   ├── AN_[Number]_[Name].py      # Analysis scripts (Python)
│   ├── LogFiles/                  # Stata log files
│   └── README.md                  # Project documentation
├── Data/
│   ├── Raw/                       # Original data (never modify)
│   ├── Intermediate/              # Partial processing
│   └── Clean/                     # Analysis-ready data
└── Results/
    ├── Tables/                    # Regression tables
    └── Figures/                   # Visualizations

Script Numbering Convention

• 0: Initial data extraction
• 1a, 1b, 1c: Data cleaning and preparation
• 2a, 2b: Data merging and linking
• 3a, 3b: Feature extraction and engineering
• 4a, 4b: Final data preparation
• 5a, 5b: Descriptive analysis
• AN_1, AN_2: Formal analysis and regressions

Use letter suffixes (a, b, c) for parallel steps Use number suffixes (1, 2, 3) for sequential substeps

Tool Preferences by Task

| Task | Preferred Tool | Rationale | |------|---------------|-----------| | Figures/Visualizations | Python | Better control, publication-quality with matplotlib/seaborn | | Regression Tables | Stata | More efficient with outreg2/esttab, standard in economics | | Data Cleaning | Python | Better for large datasets, flexible transformations | | Panel Regressions | Stata | reghdfe package is gold standard |

---

Python Guidelines

1. Script Template

#!/usr/bin/env python3
"""
ScriptName.py

Brief description of what this script does.

Input files:
- Data/Raw/input1.csv
- Data/Intermediate/input2.csv

Output files:
- Data/Clean/output.csv (description)

Author: Your Name
Date: YYYY-MM-DD
"""

import pandas as pd
import numpy as np
from pathlib import Path
import matplotlib.pyplot as plt
import seaborn as sns

def get_project_root():
    """Automatically detect the project root directory."""
    return Path(__file__).parent.absolute()


def main():
    """Main processing function."""
    print("=" * 70)
    print("SCRIPT TITLE")
    print("=" * 70)

    # Setup paths
    base_dir = get_project_root()
    data_clean_dir = base_dir / ".." / "Data" / "Clean"

    # Your code here

    print("\n" + "=" * 70)
    print("PROCESSING COMPLETE")
    print("=" * 70)


if __name__ == "__main__":
    main()

2. Path Management (CRITICAL)

Always use this pattern for portability:

def get_project_root():
    """Automatically detect the project root directory."""
    return Path(__file__).parent.absolute()

# Then use relative paths
base_dir = get_project_root()
data_raw_dir = base_dir / ".." / "Data" / "Raw"
data_clean_dir = base_dir / ".." / "Data" / "Clean"
data_intermediate_dir = base_dir / ".." / "Data" / "Intermediate"
results_tables_dir = base_dir / ".." / "Results" / "Tables"
results_figures_dir = base_dir / ".." / "Results" / "Figures"

# Create directories if they don't exist
data_intermediate_dir.mkdir(parents=True, exist_ok=True)

3. Data Loading & Saving

# Loading with error handling
if not input_file.exists():
    raise FileNotFoundError(f"Input file not found: {input_file}")

try:
    df = pd.read_csv(input_file, low_memory=False)
    print(f"Loaded {len(df):,} records")
except Exception as e:
    print(f"Error reading file: {e}")
    return

# Saving with confirmation
df_sorted = df.sort_values(['company', 'year'])
df_sorted.to_csv(output_file, index=False)
print(f"Saved {len(df_sorted):,} records to: {output_file}")

4. Progress Reporting

Use consistent formatting for readability:

# Section headers
print("\n" + "=" * 70)
print("DATA PROCESSING")
print("=" * 70)

# Progress with comma formatting
print(f"\nLoaded {len(df):,} records")
print(f"  After filtering: {len(df_filtered):,} ({len(df_filtered)/len(df)*100:.1f}%)")

# Summary statistics
print("\n=== SUMMARY ===")
print(f"Total companies: {df['company'].nunique():,}")
print(f"Date range: {df['date'].min()} to {df['date'].max()}")
print(f"Match rate: {match_rate*100:.1f}%")

5. Function Documentation

def clean_company_name(name_str):
    """
    Clean and standardize company names for matching.

    Parameters:
    - name_str: Raw company name string

    Returns:
    - Cleaned company name (uppercase, no punctuation)
    """
    if pd.isna(name_str):
        return ""

    # Remove common suffixes
    name = str(name_str).upper()
    name = re.sub(r'\b(INC|CORP|LTD|LLC)\b', '', name)
    name = re.sub(r'[^\w\s]', '', name)  # Remove punctuation

    return name.strip()

6. Data Validation

# Check for required columns
required_cols = ['company', 'year', 'value']
missing_cols = [col for col in required_cols if col not in df.columns]
if missing_cols:
    raise ValueError(f"Missing required columns: {missing_cols}")

# Report data quality
print("\nData Quality Checks:")
print(f"  Missing values in key column: {df['key_col'].isna().sum():,}")
print(f"  Duplicate records: {df.duplicated().sum():,}")
print(f"  Unique companies: {df['company'].nunique():,}")

7. Merging Pattern

# Prepare keys
df1['merge_key'] = df1['company'].astype(str).str.strip().str.upper()
df2['merge_key'] = df2['company'].astype(str).str.strip().str.upper()

# Merge with reporting
print(f"\nMerging datasets:")
print(f"  Dataset 1: {len(df1):,} records")
print(f"  Dataset 2: {len(df2):,} records")

df_merged = df1.merge(df2, on='merge_key', how='inner', indicator=True)

print(f"  Merged: {len(df_merged):,} records")
print(f"  Match rate: {len(df_merged)/len(df1)*100:.1f}%")

# Check merge results
print("\nMerge indicator breakdown:")
print(df_merged['_merge'].value_counts())

8. Visualization Standards

# Setup (at top of script)
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 7)

# Create publication-quality figures
fig, ax = plt.subplots(figsize=(14, 8))

ax.plot(x, y, marker='o', linewidth=2, markersize=8,
        color='#2E86AB', label='Series Name')

ax.set_xlabel('X-axis Label', fontsize=12, fontweight='bold')
ax.set_ylabel('Y-axis Label', fontsize=12, fontweight='bold')
ax.set_title('Figure Title', fontsize=14, fontweight='bold', pad=20)

# Format y-axis with commas
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f'{int(x):,}'))

ax.legend(loc='best', frameon=True, fancybox=True, shadow=True)
ax.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig(output_path, dpi=300, bbox_inches='tight')
plt.close()

print(f"Saved figure to: {output_path}")

9. Variable Naming Conventions

| Type | Convention | Examples | |------|-----------|----------| | DataFrames | df_ prefix | df, df_filtered, df_merged, df_agg | | Paths | _dir or _file suffix | base_dir, input_file, output_path | | Functions | snake_case verbs | clean_data(), load_files(), calculate_returns() | | Variables | snake_case | company_name, year_founded, total_assets | | Constants | UPPER_CASE | START_YEAR, MIN_OBSERVATIONS |

---

Stata Guidelines

1. Script Template

/*
================================================================================
ScriptName.do

Description of the analysis performed in this script.

Inputs:
- ../Data/Clean/input_data.csv

Outputs:
- ../Results/Tables/Table1_MainResults.xls
- ../Code/LogFiles/ScriptName.log

Author: Your Name
Date: YYYY-MM-DD
================================================================================
*/

*** Set up paths
global repodir "/Users/zrsong/MIT Dropbox/Zirui Song/Research Projects/PROJECT_NAME"
global datadir "$repodir/Data"
global cleandir "$datadir/Clean"
global intdir "$datadir/Intermediate"
global tabdir "$repodir/Results/Tables"
global figdir "$repodir/Results/Figures"
global logdir "$repodir/Code/LogFiles"

*** Start log
log using "$logdir/ScriptName.log", text replace

/*==============================================================================
    Data Preparation
==============================================================================*/

import delimited "$cleandir/input_data.csv", clear

[Your code here]

*** Close log
log close

2. Global Path Setup (CRITICAL)

Always define these at the top:

global repodir "/Full/Path/To/Project"
global datadir "$repodir/Data"
global cleandir "$datadir/Clean"
global intdir "$datadir/Intermediate"
global rawdir "$datadir/Raw"
global tabdir "$repodir/Results/Tables"
global figdir "$repodir/Results/Figures"
global logdir "$repodir/Code/LogFiles"

Note: Update repodir for each user/computer

3. Regression Structure

/*==============================================================================
    Main Regressions - Table 1
==============================================================================*/

*** Define variable lists
local borr_controls "log_assets leverage tangibility profitability"
local loan_controls "log_amount maturity"
local all_controls "`borr_controls' `loan_controls'"

*** Column 1: No controls
reghdfe outcome treatment_var, ///
    absorb(industry year) ///
    vce(cluster firm_id)

outreg2 using "$tabdir/Table1_MainResults.xls", replace excel ///
    ctitle("(1) No Controls") label dec(3) ///
    addtext(Industry FE, YES, Year FE, YES) ///
    keep(treatment_var)

*** Column 2: With controls
reghdfe outcome treatment_var `all_controls', ///
    absorb(industry year) ///
    vce(cluster firm_id)

outreg2 using "$tabdir/Table1_MainResults.xls", append excel ///
    ctitle("(2) Full Controls") label dec(3) ///
    addtext(Industry FE, YES, Year FE, YES, Controls, YES) ///
    keep(treatment_var `all_controls')

4. Output Table Conventions

Two output workflows:

| Stage | Command | Output Format | Use Case | |-------|---------|---------------|----------| | Working/Exploratory | outreg2 | Excel (.xls) | Quick iteration, reviewing results | | Final Paper | esttab | LaTeX (.tex) | Publication-ready tables |

---

A. Working Tables: outreg2 with Excel

Use outreg2 with the excel option for exploratory analysis and quick iterations:

outreg2 using "$tabdir/TableName.xls", [replace/append] excel ///
    ctitle("Column Title") ///              // Column header
    label ///                                // Use variable labels
    dec(3) ///                              // 3 decimal places
    keep(vars_to_show) ///                  // Variables to display
    addtext(Industry FE, YES, ///           // Notes for fixed effects
            Year FE, YES, ///
            Controls, YES)

Working table naming:

• Table1_MainResults.xls
• Table2_Robustness.xls
• TableA1_DescriptiveStats.xls (appendix)

---

B. Final Paper Tables: esttab for LaTeX

Use esttab to generate publication-ready LaTeX tables:

*** Store regression results
eststo clear

eststo m1: reghdfe outcome treatment, absorb(industry year) vce(cluster firm_id)
eststo m2: reghdfe outcome treatment `controls', absorb(industry year) vce(cluster firm_id)
eststo m3: reghdfe outcome treatment `controls', absorb(firm_id year) vce(cluster firm_id)

*** Output La