Development Workflow: tf-kube-any-compute

Development Setup

Prerequisites

# Required tools
terraform >= 1.0
kubectl >= 1.20
helm >= 3.0
make
bash >= 4.0

# Recommended tools
jq
yq
git
pre-commit

Initial Setup

1. Clone Repository

git clone https://github.com/gannino/tf-kube-any-compute.git
cd tf-kube-any-compute

2. Install Pre-commit Hooks

# Option 1: Using pip
pip install pre-commit
pre-commit install
pre-commit install --hook-type commit-msg

# Option 2: Using setup script
./setup-pre-commit.sh

# Option 3: Using Makefile
make pre-commit-install

3. Initialize Terraform

make init
# or
terraform init

4. Configure Environment

# Copy example configuration
cp terraform.tfvars.example terraform.tfvars

# Create development workspace
terraform workspace new dev

Development Workflow

1. Code Development

Create New Feature Branch

git checkout -b feature/add-new-service

Edit Files

# Edit module files
vi helm-new-service/main.tf
vi helm-new-service/variables.tf
vi helm-new-service/values.yaml.tpl

Run Local Tests

# Quick validation
make test-quick

# Safe tests (no deployment)
make test-safe

# Full validation
make test-lint
make test-validate

2. Pre-commit Validation

Automatic Checks

# Stage files
git add .

# Pre-commit runs automatically on commit
git commit -m "feat(new-service): add new service module"

Manual Pre-commit Run

# Run all checks manually
pre-commit run --all-files

# Run specific hook
pre-commit run terraform-fmt --all-files
pre-commit run tflint-optimized --all-files

Pre-commit Hooks

• terraform-fmt: Auto-format Terraform files
• terraform-docs: Generate documentation
• tflint-optimized: Fast linting (changed files only)
• terraform-validate: Validate configuration
• checkov: Security scanning
• terrascan: Security scanning
• detect-secrets: Detect secrets in code
• commit-msg: Validate commit message format

3. Testing Strategy

Unit Tests

# Run unit tests
make test-unit

# Run specific test file
terraform test -filter=tests.tftest.hcl -verbose

# Run architecture tests
terraform test -filter=tests-architecture.tftest.hcl

Scenario Tests

# Run scenario tests
make test-scenarios

# Test specific scenario
terraform test -filter=test-scenarios.tftest.hcl -verbose

Integration Tests

# Run integration tests (requires deployed infrastructure)
make test-integration

# Test specific service
./scripts/integration-tests.sh --service traefik

Performance Tests

# Run performance tests (requires k6)
make test-performance

# Run with custom configuration
k6 run --vus 10 --duration 30s scripts/performance-test.js

Security Tests

# Run security scanning
make test-security

# Run individual security tools
./scripts/security-scan.sh

4. Commit Process

Commit Message Format

# Conventional commits
feat: add new service module
fix: resolve traefik authentication issue
docs: update contributing guide
test: add integration tests for monitoring
refactor: optimize resource limits
style: format terraform files
chore: update dependencies

Commit Examples

# Feature addition
git commit -m "feat(prometheus): add high availability configuration"

# Bug fix
git commit -m "fix(traefik): resolve dashboard authentication issue"

# Documentation
git commit -m "docs(readme): update installation instructions"

# Test addition
git commit -m "test(integration): add ARM64 deployment scenarios"

5. Code Review

Pull Request Checklist

• [ ] All tests pass locally
• [ ] Pre-commit hooks succeed
• [ ] Documentation updated
• [ ] Examples provided
• [ ] Tests added/updated
• [ ] Backward compatibility maintained
• [ ] Security review completed
• [ ] Performance considered

PR Template

## Description
Brief description of changes

## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Documentation update
- [ ] Breaking change

## Testing
- [ ] Unit tests pass
- [ ] Integration tests pass
- [ ] Manual testing completed

## Architecture Support
- [ ] Tested on AMD64
- [ ] Tested on ARM64
- [ ] Tested on mixed clusters

## Documentation
- [ ] README updated
- [ ] Code comments added
- [ ] Examples provided

## Breaking Changes
List any breaking changes and migration steps

Module Development

Creating New Service Module

1. Create Module Structure

mkdir helm-new-service
cd helm-new-service

# Create standard files
touch main.tf
touch variables.tf
touch outputs.tf
touch locals.tf
touch version.tf
touch values.yaml.tpl
touch README.md
touch .tflint.hcl
mkdir -p templates

2. Implement Main Resources

# main.tf
###########################
#  New Service - Purpose  #
###########################

# Namespace
resource "kubernetes_namespace" "this" {
  metadata {
    name = local.namespace
    labels = local.common_labels
  }
}

# Helm release
resource "helm_release" "this" {
  name       = local.release_name
  namespace  = kubernetes_namespace.this.metadata[0].name
  repository = var.chart_repository
  chart      = var.chart_name
  version    = local.chart_version

  # Timeout and wait configuration
  timeout          = local.helm_timeout
  wait             = local.helm_wait
  wait_for_jobs    = local.helm_wait_for_jobs
  cleanup_on_fail  = local.helm_cleanup_on_fail

  # Values from template
  values = [
    templatefile("${path.module}/templates/values.yaml.tpl", {
      namespace     = local.namespace
      cpu_arch      = local.cpu_arch
      storage_class = local.storage_class
    })
  ]

  depends_on = [
    kubernetes_namespace.this
  ]
}

3. Define Variables

# variables.tf
###########################
#  Variables - Input     #
###########################

variable "chart_name" {
  description = "Helm chart name"
  type        = string
  default     = "service-chart"
}

variable "chart_repository" {
  description = "Helm chart repository URL"
  type        = string
  default     = "https://charts.example.com"
}

variable "chart_version" {
  description = "Helm chart version (empty = latest)"
  type        = string
  default     = ""

  validation {
    condition     = var.chart_version == "" || can(regex("^\\d+\\.\\d+\\.\\d+", var.chart_version))
    error_message = "Chart version must be empty or in semantic version format (e.g., 1.2.3)."
  }
}

variable "cpu_arch" {
  description = "CPU architecture for node selection"
  type        = string
  default     = ""

  validation {
    condition     = var.cpu_arch == "" || contains(["amd64", "arm64"], var.cpu_arch)
    error_message = "CPU architecture must be 'amd64', 'arm64', or empty for auto-detection."
  }
}

variable "storage_class" {
  description = "Storage class for persistent volumes"
  type        = string
  default     = ""
}

4. Define Outputs

# outputs.tf
###########################
#  Outputs - Results     #
###########################

output "namespace" {
  description = "The namespace where the service is deployed"
  value       = kubernetes_namespace.this.metadata[0].name
}

output "release_name" {
  description = "The Helm release name"
  value       = helm_release.this.name
}

output "chart_version" {
  description = "The deployed chart version"
  value       = helm_release.this.metadata[0].version
}

output "status" {
  description = "The Helm release status"
  value       = helm_release.this.status
}

output "service_info" {
  description = "Complete service information"
  value = {
    namespace     = kubernetes_namespace.this.metadata[0].name
    release_name  = helm_release.this.name
    chart_version = helm_release.this.metadata[0].version
    status        = helm_release.this.status
  }
}

5. Define Locals

# locals.tf
###########################
#  Locals - Computed     #
###########################

locals {
  # Namespace configuration
  namespace = "${var.environment}-${var.name}-system"

  # Release name
  release_name = "${var.environment}-${var.name}"

  # Chart version (default to latest if not specified)
  chart_version = var.chart_version != "" ? var.chart_version : null

  # Helm timeout
  helm_timeout = coalesce(
    var.helm_timeout,
    var.default_helm_timeout,
    600
  )

  # Common labels
  common_labels = {
    "app.kubernetes.io/name"       = var.name
    "app.kubernetes.io/instance"   = local.release_name
    "app.kubernetes.io/managed-by" = "terraform"
  }

  # Resource limits based on architecture
  resource_limits = var.cpu_arch == "arm64" ? {
    cpu_limit      = "200m"
    memory_limit   = "256Mi"
    cpu_request    = "100m"
    memory_request = "128Mi"
  } : {
    cpu_limit      = "500m"
    memory_limit   = "512Mi"
    cpu_request    = "250m"
    memory_request = "256Mi"
  }
}

6. Create Helm Values Template

# templates/values.yaml.tpl
###########################
#  Helm Values Template    #
###########################

image:
  repository: ${image_repository}
  tag: ${image_tag}
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: ${service_port}

resources:
%{ if enable_resource_limits ~}
  limits:
    cpu: ${cpu_limit}
    memory: ${memory_limit}
  requests:
    cpu: ${cpu_request}
    memory: ${memory_request}
%{ endif ~}

%{ if enable_persistence ~}
persistence:
  enabled: true
  storageClass: ${storage_class}
  size: ${storage_size}
%{ endif ~}

nodeSelector:
%{ if cpu_arch != "" ~}
  kubernetes.io/arch: ${cpu_arch}
%{ endif ~}

tolerations: []
affinity: {}

7. Create Module README

# New Service Helm Module

Brief description of the service and its purpose.

## Features
- **Feature 1**: Description
- **Feature 2**: Description

## Usage

### Basic Usage
```hcl
module "new_service" {
  source = "./helm-new-service"

  name        = "new-service"
  environment = "prod"
}

Advanced Configuration

module "new_service" {
  source = "./helm-new-service"

  name        = "new-service"
  environment = "prod"

  cpu_arch      = "arm64"
  storage_class = "nfs-csi"

  helm_timeout = 900
}

Requirements

| Name | Version | |------|---------| | terraform | >= 1.0 | | helm | ~> 3.0 | | kubernetes | ~> 2.0 |

Providers

| Name | Version | |------|---------| | kubernetes | ~> 2.0 | | helm | ~> 3.0 |

Inputs

| Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | name | Service name | string | n/a | yes | | environment | Environment name | string | n/a | yes | | chart_name | Helm chart name | string | "service-chart" | no | | chart_repository | Helm chart repository | string | "https://charts.example.com" | no | | chart_version | Helm chart version | string | "" | no | | cpu_arch | CPU architecture | string | "" | no | | storage_class | Storage class | string | "" | no |

Outputs

| Name | Description | |------|-------------| | namespace | The namespace where the service is deployed | | release_name | The Helm release name | | chart_version | The deployed chart version | | status | The Helm release status | | service_info | Complete service information |

Architecture Support

ARM64

• Resource limits: 200m CPU / 256Mi memory
• Optimized for Raspberry Pi clusters
• MicroK8s mode compatible

AMD64

• Resource limits: 500m CPU / 512Mi memory
• Optimized for cloud environments
• Full feature support

Troubleshooting

Common Issues

Issue: Service fails to start

# Check pod logs
kubectl logs -n prod-new-service-system -l app=new-service

# Check pod status
kubectl get pods -n prod-new-service-system -l app=new-service

Issue: Storage not accessible

# Check PVC status
kubectl get pvc -n prod-new-service-system

# Check storage class
kubec