Source: data_layer/README.md

Data Layer Architecture

Overview

This data layer provides a unified schema management system for AltSportsData, with version control and multi-format generation.

Structure

data_layer/
├── output_styles/
│   └── schemas/
│       └── domains/
│           ├── v1/                          ← Version 1 (Current)
│           │   ├── python/                  ← SOURCE OF TRUTH (Canonical)
│           │   │   ├── base/               # Core models
│           │   │   ├── sports/             # Sports domain
│           │   │   ├── leagues/            # League domain
│           │   │   ├── business/           # Business domain
│           │   │   ├── api/                # API models
│           │   │   ├── workflows/          # Workflows
│           │   │   ├── ml/                 # ML models
│           │   │   ├── generators/         # Schema generators
│           │   │   └── tests/              # Tests
│           │   │
│           │   ├── typescript/             ← GENERATED from python/
│           │   ├── json/                   ← Existing JSON schemas
│           │   ├── drizzle/                ← Existing Drizzle ORM
│           │   ├── graphql/                ← Can generate from python/
│           │   ├── neo4j/                  ← Existing Neo4j schemas
│           │   └── converters/             ← Schema converters
│           │
│           └── v2/                          ← Version 2 (Future)
│               ├── python/                  # V2 models (when needed)
│               ├── typescript/              # Generated TypeScript
│               └── ...

Single Source of Truth

v1/python/ is the canonical schema source:

All models defined in Python/Pydantic
Other formats generated automatically
Version controlled alongside code

Generation Workflow

1. Define Models (Python/Pydantic)

# v1/python/leagues/questionnaires.py
from pydantic import Field
from ..base import BaseModel
 
class LeagueQuestionnaire(BaseModel):
    league_name: str = Field(..., description="League name")
    # ... more fields

2. Generate to Any Format

cd data_layer/output_styles/schemas/domains/v1/python
 
# Generate TypeScript to v1
python generate_schemas.py --format typescript --output ../typescript/
 
# Generate SQL to v1
python generate_schemas.py --format sql --output ../sql/
 
# Generate GraphQL to v1
python generate_schemas.py --format graphql --output ../graphql/
 
# Generate all formats
python generate_schemas.py --all

3. Use Generated Schemas

Backend (Python):

from apps.backend.core.models import LeagueQuestionnaire
# Imports from v1/python automatically

Frontend (TypeScript):

import { LeagueQuestionnaire } from '@/types/generated/leagues';
// Uses generated TypeScript from v1/typescript/

Version Management

v1 (Current - Production)

Stable API
Used by all current applications
Generate from v1/python/

v2 (Future - Next Generation)

Breaking changes
New features
Gradual migration path

Migration Path

v1/python/                  v2/python/
    ↓                          ↓
v1/typescript/   →  →  →   v2/typescript/
    ↓                          ↓
Frontend v1              Frontend v2

Generator Outputs

From v1/python/, you can generate:

Format	Output Location	Status
TypeScript + Zod	`v1/typescript/`	✓ Working
JSON Schema	`v1/json/`	Ready
SQL (PostgreSQL)	`v1/sql/`	Stub
GraphQL	`v1/graphql/`	Stub
Drizzle ORM	`v1/drizzle/`	Can integrate
Prisma	`v1/prisma/`	Future

Integration Points

Backend Integration

# apps/backend/core/models/__init__.py
from data_layer.output_styles.schemas.domains.v1.python.base import BaseModel

Frontend Integration

// tsconfig.json
{
  "compilerOptions": {
    "paths": {
      "@/types/generated/*": ["./data_layer/output_styles/schemas/domains/v1/typescript/*"]
    }
  }
}

Database Integration

// Use generated Drizzle schemas
import { leagueTable } from '@/data_layer/schemas/domains/v1/drizzle/leagues';

Best Practices

1. Always Define in Python First

# ✓ DO: Define in v1/python/
class NewModel(BaseModel):
    field: str
 
# ✗ DON'T: Manually create TypeScript
interface NewModel { ... }  // Will be overwritten by generator

2. Generate After Changes

# After modifying Python models
python generate_schemas.py --all

3. Version Breaking Changes

# For breaking changes, use v2
cp -r v1/python/ v2/python/
# Make breaking changes in v2/python/
# Generate v2 schemas separately

4. Test Generated Schemas

# Test Python models
pytest v1/python/tests/
 
# Test generated TypeScript
npm test

Quick Commands

# Navigate to Python schemas
cd data_layer/output_styles/schemas/domains/v1/python
 
# Preview migration from old models
python migrate_models.py --dry-run
 
# Execute migration
python migrate_models.py
 
# Generate all schemas
python generate_schemas.py --all
 
# Test everything
python quick_test.py
 
# Generate specific format
python generate_schemas.py --format typescript
python generate_schemas.py --format sql
python generate_schemas.py --format graphql

Benefits

✓ Single Source of Truth - Define once in Python ✓ Version Control - v1, v2, etc. for breaking changes ✓ Auto-Generation - TypeScript, SQL, GraphQL from Python ✓ Type Safety - Strong typing everywhere ✓ No Drift - Frontend/backend always in sync ✓ Easy Migration - Clear path from v1 to v2

Next Steps

Migrate existing models from apps/backend/core/models_backup/:

cd v1/python
python migrate_models.py --dry-run  # Preview
python migrate_models.py            # Execute

Generate all formats:
```
python generate_schemas.py --all
```
Update imports in backend/frontend to use generated schemas
Set up CI/CD to auto-generate schemas on commit

Data Layer & Schema Registry Database Directory Reorganization Proposal