Story 1.18: Codebase Structure Consolidation

# Status

Draft

# Story

As a developer, I want the codebase structure to be organized, consistent, and aligned with maintainability best practices, so that the project is easier to navigate, understand, and maintain as it grows.

# Domain Context

# Primary Domain

Infrastructure & Development Operations - Codebase organization and structural improvements

# Scope

This is a REFACTORING STORY focused on improving codebase organization without changing functionality:

• Service directory naming standardization
• Test consolidation into unified structure
• Infrastructure stack organization
• Documentation alignment with actual structure

# What This Story Does NOT Include

• Excluded: Deleting or moving BATspa-old/ (legacy Angular app - keep as-is)
• Excluded: Deleting or moving apps/projectdoc/ (project documentation - keep as-is)
• Excluded: Functionality changes or feature additions
• Excluded: Breaking changes to deployed systems

# Requirements Context

# Problem Statement

Current codebase structure analysis (2025-11-01) revealed:

1. Service Naming Inconsistency: All 5 services have -service suffix not present in documentation
2. Test Fragmentation: E2E tests scattered across 3 locations (bruno-tests/, web-frontend/e2e/, infrastructure/test/e2e/)
3. Infrastructure Organization: All CDK stacks flat in single directory, not organized by concern (global/core/domain)
4. Documentation Misalignment: source-tree.md doesn’t match actual structure

# Goals

• Maintainability: Reduce cognitive load for developers navigating the codebase
• Consistency: Align service naming with documentation standards
• Organization: Group related code together (tests, stacks)
• Clarity: Make project structure self-documenting

# Related Documentation

• Source Tree: docs/architecture/source-tree.md (needs update)
• Analysis Report: Comprehensive structure analysis completed 2025-11-01 (see Dev Notes)

# Architecture Context

# Current Structure Issues

Services Directory:

services/
├── attendee-experience-service/        # Should be: attendee-experience/
├── company-user-management-service/    # Should be: company-management/
├── event-management-service/           # Should be: event-management/
├── partner-coordination-service/       # Should be: partner-coordination/
└── speaker-coordination-service/       # Should be: speaker-coordination/

Test Fragmentation:

• bruno-tests/ - API integration tests (not documented)
• web-frontend/e2e/ - Frontend e2e tests
• infrastructure/test/e2e/ - Infrastructure tests
• Documentation expects: unified e2e-tests/ directory

Infrastructure Organization:

• Current: All stacks in flat infrastructure/lib/stacks/ (19 files)
• Target: Organized by concern (global/, core/, domain/, per documentation)

# Target Structure (Meet in Middle Approach)

Services (Remove -service suffix):

services/
├── attendee-experience/
├── company-management/
├── event-management/
├── partner-coordination/
└── speaker-coordination/

Tests (Consolidated):

e2e-tests/
├── api/                    # From bruno-tests/
├── workflows/              # From web-frontend/e2e/
├── infrastructure/         # From infrastructure/test/e2e/
├── fixtures/               # Shared test data
└── README.md              # Test documentation

Infrastructure (Organized by Concern):

infrastructure/lib/stacks/
├── global/                 # DNS, Secrets, CI/CD
├── core/                   # Network, Database, Cognito, EventBus
├── domain/                 # Service-specific stacks
└── frontend/               # Frontend deployment

# Architecture Patterns

• DDD Alignment: Service names match bounded contexts (no technical suffixes)
• Test Organization: Tests organized by type (api, workflows, infrastructure)
• Infrastructure as Code: CDK stacks organized by deployment lifecycle
• Documentation Accuracy: source-tree.md reflects reality

# Acceptance Criteria

# Phase 1: Service Naming Standardization

• AC1.1: All service directories renamed to remove -service **AC1.1**: All service directories renamed to remove `-service` suffix

  • event-management-service/ → event-management/
  • speaker-coordination-service/ → speaker-coordination/
  • partner-coordination-service/ → partner-coordination/
  • attendee-experience-service/ → attendee-experience/
  • company-user-management-service/ → company-management/

• AC1.2: Fix company-management **AC1.2**: Fix `company-management` service internal issues

  • Consolidate duplicate event/ and events/ directories into single events/
  • Update Java package name from companyuser to company (if feasible)

• AC1.3 **AC1.3**: Update all references to old service names

  • Gradle build files (settings.gradle, build.gradle)
  • Docker Compose configurations
  • CDK infrastructure stack references
  • GitHub Actions workflows
  • Documentation files

• AC1.4 **AC1.4**: Verify all services build successfully after rename

  • ./gradlew build passes for all services
  • No broken imports or package references

# Phase 2: Test Consolidation

• AC2.1: Create unified e2e-tests/ **AC2.1**: Create unified `e2e-tests/` directory structure

  e2e-tests/
  ├── api/                  # API integration tests
  ├── workflows/            # User workflow tests
  ├── infrastructure/       # Infrastructure tests
  ├── fixtures/             # Shared test data
  ├── .env.example         # Test environment template
  └── README.md            # Test documentation
  

• AC2.2 **AC2.2**: Migrate API tests

  • Move bruno-tests/* → e2e-tests/api/
  • Update Bruno collection paths
  • Verify API tests still run

• AC2.3 **AC2.3**: Migrate frontend e2e tests

  • Move web-frontend/e2e/* → e2e-tests/workflows/
  • Update Playwright configuration paths
  • Verify frontend e2e tests still run

• AC2.4 **AC2.4**: Migrate infrastructure tests

  • Move infrastructure/test/e2e/* → e2e-tests/infrastructure/
  • Update test import paths
  • Verify infrastructure tests still run

• AC2.5 **AC2.5**: Update test execution scripts

  • Update package.json test scripts
  • Update scripts/test/* paths
  • Update GitHub Actions test workflows
  • Create e2e-tests/README.md with test documentation

• AC2.6 **AC2.6**: Clean up old test directories

  • Remove empty bruno-tests/ directory
  • Remove empty web-frontend/e2e/ directory
  • Remove empty infrastructure/test/e2e/ directory

# Phase 3: Infrastructure Reorganization

• AC3.1 **AC3.1**: Create organized stack directory structure

  infrastructure/lib/stacks/
  ├── global/              # Region-independent resources
  ├── core/                # Regional shared resources
  ├── domain/              # Service-specific stacks
  └── frontend/            # Frontend deployment
  

• AC3.2 **AC3.2**: Categorize and move stacks to appropriate directories

  • Global stacks → global/

    • DNS/Route53 configurations
    • Secrets/Parameter Store
    • CI/CD pipeline stacks

  • Core stacks → core/

    • Network/VPC stack
    • Database stack
    • Cognito stack
    • EventBus stack
    • Monitoring stack

  • Domain stacks → domain/

    • Event management stack
    • Company management stack
    • Service-specific infrastructure

  • Frontend stack → frontend/

    • CloudFront/S3 frontend deployment

• AC3.3 **AC3.3**: Update CDK imports and references

  • Update bin/batbern-infrastructure.ts import paths
  • Update stack file imports (cross-stack references)
  • Verify CDK synth works: npx cdk synth

• AC3.4 **AC3.4**: Update infrastructure documentation

  • Document stack organization rationale
  • Update deployment guides with new paths

# Phase 4: Documentation Alignment

• AC4.1: Update **AC4.1**: Update `docs/architecture/source-tree.md`

  • Reflect actual service names (without -service suffix)
  • Document e2e-tests/ unified structure
  • Document infrastructure stacks/ organization
  • Add sections for undocumented directories:
    • .bmad-core/ - Workflow automation
    • security-reports/ - Trivy scan results
    • config/ - Checkstyle configuration
    • mockups/ - HTML mockups
  • Note legacy directories as “archive” (BATspa-old, apps/projectdoc)

• AC4.2 **AC4.2**: Document actual vs. ideal structure

  • Add comments explaining pragmatic choices
  • Note deviations from original architecture plan
  • Document rationale for “meet in middle” decisions

• AC4.3 **AC4.3**: Update related documentation

  • Update developer onboarding docs
  • Update README.md if it references old structure
  • Update contribution guidelines

# Phase 5: Validation & Cleanup

• AC5.1 **AC5.1**: Verify all builds pass

  • ./gradlew build - All services compile
  • npm run build - Frontend builds
  • npx cdk synth - Infrastructure validates

• AC5.2 **AC5.2**: Verify all tests pass

  • Run API tests from new e2e-tests/api/ location
  • Run frontend e2e tests from new e2e-tests/workflows/ location
  • Run infrastructure tests from new e2e-tests/infrastructure/ location
  • Run unit tests: ./gradlew test

• AC5.3 **AC5.3**: Verify git hooks still work

  • Pre-commit hooks execute
  • Commit message validation works

• AC5.4 **AC5.4**: Clean up empty directories

  • Remove any empty directories created during migration
  • Verify git status is clean

• AC5.5 **AC5.5**: Create migration guide

  • Document all changes made
  • Provide mapping of old → new paths
  • Note breaking changes for other developers

# Dev Notes

# Analysis Summary (2025-11-01)

Comprehensive structure analysis identified:

• 5 services with naming mismatches (all have -service suffix)
• 3 test directories to consolidate (bruno-tests, web-frontend/e2e, infrastructure/test/e2e)
• 19 infrastructure stacks in flat directory structure
• 8+ undocumented directories that should be documented

# User Preferences

• Keep as-is: BATspa-old/ and apps/projectdoc/ (legacy/archive)
• Priority: Service naming, test consolidation, infrastructure organization
• Approach: Meet in middle (some code changes, some doc updates)
• Environment: Development only (no production deployments to worry about)

# Key Constraints

• Development-only environment (no live deployments)
• No functionality changes (refactoring only)
• Must maintain all existing tests passing
• Git history should be preserved (git mv commands)

# Risk Mitigation

1. Breaking Changes: Use git mv to preserve history, update all references systematically
2. Build System: Update Gradle, CDK, package.json incrementally with validation
3. Test Breakage: Verify tests after each phase before proceeding
4. CI/CD: Update GitHub Actions workflows alongside code changes

# Related Stories

• Story 1.1: Shared-Kernel Infrastructure Setup
• Story 1.3: Multi-Environment CDK Infrastructure
• Story 1.7: TDD Developer Workflow Setup
• Story 2.2: Event Management Service Architecture Compliance (naming reference)

# Testing

# Pre-Migration Tests (Baseline)

• Run full test suite and record results
  • API tests (bruno-tests)
  • Frontend e2e tests (web-frontend/e2e)
  • Infrastructure tests
  • Unit tests (all services)
• Verify builds succeed
  • Gradle build
  • Frontend build
  • CDK synth

# Phase Testing Strategy

Each phase must validate:

1. After Service Rename: All services build, imports resolve
2. After Test Migration: All tests run from new locations
3. After Infrastructure Reorganization: CDK synth succeeds
4. After Documentation Update: No broken links

# Post-Migration Validation

• All baseline tests still pass
• All builds still succeed
• Git hooks functional
• CI/CD pipelines run successfully

# Tasks

# Task 0: Preparation & Baseline

• Create feature branch: Create feature branch: `feature/story-1.18-structure-consolidation`
• Run and record baseline test results
• Document current directory structure
• Review git hooks and CI/CD workflows to identify all references

# Task 1: Service Naming Standardization

• Rename service directories using Rename service directories using `git mv`

  • git mv services/event-management-service services/event-management
  • git mv services/speaker-coordination-service services/speaker-coordination
  • git mv services/partner-coordination-service services/partner-coordination
  • git mv services/attendee-experience-service services/attendee-experience
  • git mv services/company-user-management-service services/company-management

• Fix company-management internal duplicates

  • Consolidate event/ and events/ directories

• Update Gradle configuration

  • Update settings.gradle project names
  • Update any cross-service build dependencies

• Update Docker Compose

  • Update service names in docker-compose.yml
  • Update volume mappings
  • Update network configurations

• Update infrastructure references

  • Search CDK code for old service paths
  • Update Lambda function paths
  • Update ECS service definitions

• Update CI/CD workflows

  • Update GitHub Actions service paths
  • Update test paths
  • Update build paths

• Verify builds

  • Run ./gradlew build
  • Fix any broken imports
  • Verify all services compile

• Commit: "refactor: standardize service naming (remove -service suffix)"

# Task 2: Test Consolidation

• Create e2e-tests/ Create `e2e-tests/` directory structure

  bash

  mkdir -p e2e-tests/{api,workflows,infrastructure,fixtures}
  

• Migrate API tests

  • git mv bruno-tests/* e2e-tests/api/
  • Update Bruno collection paths
  • Test: Run API tests from new location

• Migrate frontend e2e tests

  • git mv web-frontend/e2e/* e2e-tests/workflows/
  • Update playwright.config.ts paths
  • Test: Run frontend e2e tests from new location

• Migrate infrastructure tests

  • git mv infrastructure/test/e2e/* e2e-tests/infrastructure/
  • Update test import paths
  • Test: Run infrastructure tests from new location

• Create test documentation

  • Create e2e-tests/README.md
  • Document test organization
  • Document how to run each test suite
  • Create .env.example for test configuration

• Update test scripts

  • Update package.json test scripts
  • Update scripts/test/* paths
  • Update GitHub Actions test workflows

• Clean up old directories

  • Remove empty bruno-tests/
  • Remove empty web-frontend/e2e/
  • Remove empty infrastructure/test/e2e/

• Verify all tests

  • Run API tests
  • Run frontend e2e tests
  • Run infrastructure tests
  • Verify all pass from new locations

• Commit: "refactor: consolidate e2e tests into unified structure"

# Task 3: Infrastructure Reorganization

• Create stack organization directories

  bash

  mkdir -p infrastructure/lib/stacks/{global,core,domain,frontend}
  

• Categorize existing stacks

  • Review each stack in infrastructure/lib/stacks/
  • Categorize as global/core/domain/frontend
  • Document categorization decisions

• Move global stacks

  • Identify global stacks (DNS, Secrets, CI/CD)
  • git mv to stacks/global/
  • Update imports in moved files

• Move core stacks

  • Identify core stacks (Network, Database, Cognito, EventBus)
  • git mv to stacks/core/
  • Update imports in moved files

• Move domain stacks

  • Identify domain stacks (service-specific)
  • git mv to stacks/domain/
  • Update imports in moved files

• Move frontend stack

  • git mv frontend stack to stacks/frontend/
  • Update imports in moved files

• Update CDK bin entry point

  • Update imports in bin/batbern-infrastructure.ts
  • Update stack instantiation paths
  • Verify all stacks imported

• Update cross-stack references

  • Search for stack imports across all files
  • Update import paths
  • Fix any broken references

• Verify CDK synth

  • Run npx cdk synth
  • Fix any import errors
  • Verify all stacks synthesize

• Commit: "refactor: organize infrastructure stacks by concern"

# Task 4: Documentation Alignment

• Update source-tree.md

  • Update services section (remove -service suffix)
  • Update test section (document e2e-tests/)
  • Update infrastructure section (document stack organization)
  • Add undocumented directories section
  • Note legacy directories

• Add architecture decision context

  • Document “meet in middle” approach
  • Explain pragmatic choices
  • Note deviations from ideal structure

• Update developer documentation

  • Update onboarding guides
  • Update README.md
  • Update contribution guidelines

• Verify documentation accuracy

  • Check all file paths in docs
  • Verify no broken links
  • Test documentation examples

• Commit: "docs: update source-tree and guides to reflect actual structure"

# Task 5: Final Validation & Migration Guide

• Run complete test suite

  • Unit tests: ./gradlew test
  • API tests: e2e-tests/api/
  • Frontend e2e: e2e-tests/workflows/
  • Infrastructure tests: e2e-tests/infrastructure/
  • Record and compare with baseline

• Verify builds

  • Gradle: ./gradlew build
  • Frontend: npm run build
  • CDK: npx cdk synth

• Test git hooks

  • Make test commit
  • Verify pre-commit hooks run
  • Verify commit message validation

• Clean up

  • Remove any empty directories
  • Verify git status clean
  • Remove temporary files

• Create migration guide

  • Document all path changes (old → new)
  • List breaking changes
  • Provide migration checklist for other developers
  • Add troubleshooting section

• Commit: "docs: add structure consolidation migration guide"

• Final verification

  • Review all commits
  • Ensure no functionality changes
  • Verify tests pass
  • Verify builds succeed

# Dev Agent Record

# Agent Model Used

[Claude model version used during implementation]

# Debug Log References

[Links to debug log entries if debugging was needed]

# Completion Notes

[Notes added by dev agent upon completion]

# File List

[Complete list of all files created, modified, or deleted during this story]

# Created:

# Modified:

# Deleted:

# Change Log

[Detailed log of changes made during implementation]

# Migration Path Reference

# Service Directory Mappings

OLD PATH                                    NEW PATH
services/event-management-service/       → services/event-management/
services/speaker-coordination-service/   → services/speaker-coordination/
services/partner-coordination-service/   → services/partner-coordination/
services/attendee-experience-service/    → services/attendee-experience/
services/company-user-management-service/ → services/company-management/

# Test Directory Mappings

OLD PATH                          NEW PATH
bruno-tests/                   → e2e-tests/api/
web-frontend/e2e/              → e2e-tests/workflows/
infrastructure/test/e2e/       → e2e-tests/infrastructure/

# Infrastructure Stack Mappings

OLD PATH                                NEW PATH
infrastructure/lib/stacks/*.ts       → infrastructure/lib/stacks/{category}/*.ts
                                        (where category = global|core|domain|frontend)

# Estimated Effort

• Task 0 (Preparation): 30 minutes
• Task 1 (Service Naming): 2-3 hours
• Task 2 (Test Consolidation): 1-2 hours
• Task 3 (Infrastructure Reorganization): 2-3 hours
• Task 4 (Documentation): 30 minutes
• Task 5 (Validation): 1 hour

Total: 7-10 hours