# Import System V2 Architecture

## Overview

ImportServiceV2 is a refactored, database-driven import processing system that replaces the monolithic ImportProcessor.php with a modular, maintainable architecture.

## Key Features

- **Database-driven configuration**: Entity processing rules, validation, and handlers configured in `import_entities` table
- **Pluggable handlers**: Each entity type has its own handler class implementing `EntityHandlerInterface`
- **Queue support**: Large imports can be processed asynchronously via `ProcessLargeImportJob`
- **Validation**: Entity-level validation rules stored in database
- **Priority-based processing**: Entities processed in configured priority order
- **Extensible**: Easy to add new entity types without modifying core service

## Directory Structure

```
app/Services/Import/
├── Contracts/
│   └── EntityHandlerInterface.php    # Handler contract
├── Handlers/
│   ├── ContractHandler.php           # Contract entity handler
│   ├── AccountHandler.php            # Account entity handler
│   ├── PaymentHandler.php            # Payment handler (to be implemented)
│   ├── ActivityHandler.php           # Activity handler (to be implemented)
│   └── ...                            # Additional handlers
├── BaseEntityHandler.php             # Base handler with common logic
└── ImportServiceV2.php               # Main import service
```

## Database Schema

### import_entities Table

| Column | Type | Description |
|--------|------|-------------|
| id | bigint | Primary key |
| key | string | UI key (plural, e.g., "contracts") |
| canonical_root | string | Canonical root for processor (singular, e.g., "contract") |
| label | string | Human-readable label |
| fields | json | Array of field names |
| field_aliases | json | Field alias mappings |
| aliases | json | Root aliases |
| supports_multiple | boolean | Whether entity supports multiple items per row |
| meta | boolean | Whether entity is metadata |
| rules | json | Suggestion rules |
| ui | json | UI configuration |
| handler_class | string | Fully qualified handler class name |
| validation_rules | json | Laravel validation rules |
| processing_options | json | Handler-specific options |
| is_active | boolean | Whether entity is enabled |
| priority | integer | Processing priority (higher = first) |
| created_at | timestamp | Creation timestamp |
| updated_at | timestamp | Update timestamp |

## Handler Interface

All entity handlers must implement `EntityHandlerInterface`:

```php
interface EntityHandlerInterface
{
    public function process(Import $import, array $mapped, array $raw, array $context = []): array;
    public function validate(array $mapped): array;
    public function getEntityClass(): string;
    public function resolve(array $mapped, array $context = []): mixed;
}
```

### Handler Methods

- **process()**: Main processing method, returns result with action (inserted/updated/skipped) and entity
- **validate()**: Validates mapped data before processing
- **getEntityClass()**: Returns the model class name this handler manages
- **resolve()**: Resolves existing entity by key/reference

## Creating a New Handler

1. Create handler class extending `BaseEntityHandler`:

```php
<?php

namespace App\Services\Import\Handlers;

use App\Models\YourEntity;
use App\Models\Import;
use App\Services\Import\BaseEntityHandler;

class YourEntityHandler extends BaseEntityHandler
{
    public function getEntityClass(): string
    {
        return YourEntity::class;
    }

    public function resolve(array $mapped, array $context = []): mixed
    {
        // Implement entity resolution logic
        return YourEntity::where('key', $mapped['key'])->first();
    }

    public function process(Import $import, array $mapped, array $raw, array $context = []): array
    {
        $existing = $this->resolve($mapped, $context);

        if ($existing) {
            // Update logic
            $payload = $this->buildPayload($mapped, $existing);
            $appliedFields = $this->trackAppliedFields($existing, $payload);

            if (empty($appliedFields)) {
                return [
                    'action' => 'skipped',
                    'entity' => $existing,
                    'message' => 'No changes detected',
                ];
            }

            $existing->fill($payload);
            $existing->save();

            return [
                'action' => 'updated',
                'entity' => $existing,
                'applied_fields' => $appliedFields,
            ];
        }

        // Create logic
        $entity = new YourEntity;
        $payload = $this->buildPayload($mapped, $entity);
        $entity->fill($payload);
        $entity->save();

        return [
            'action' => 'inserted',
            'entity' => $entity,
            'applied_fields' => array_keys($payload),
        ];
    }

    protected function buildPayload(array $mapped, $model): array
    {
        // Map fields to model attributes
        return [
            'field1' => $mapped['field1'] ?? null,
            'field2' => $mapped['field2'] ?? null,
        ];
    }
}
```

2. Add configuration to `import_entities` table:

```php
ImportEntity::create([
    'key' => 'your_entities',
    'canonical_root' => 'your_entity',
    'label' => 'Your Entities',
    'fields' => ['field1', 'field2'],
    'handler_class' => \App\Services\Import\Handlers\YourEntityHandler::class,
    'validation_rules' => [
        'field1' => 'required|string',
        'field2' => 'nullable|integer',
    ],
    'processing_options' => [
        'update_mode' => 'update',
    ],
    'is_active' => true,
    'priority' => 100,
]);
```

## Usage

### Synchronous Processing

```php
use App\Services\Import\ImportServiceV2;

$service = app(ImportServiceV2::class);
$results = $service->process($import, $user);
```

### Queue Processing (Large Imports)

```php
use App\Jobs\ProcessLargeImportJob;

ProcessLargeImportJob::dispatch($import, $user->id);
```

## Processing Options

Handler-specific options stored in `processing_options` JSON column:

### Contract Handler
- `update_mode`: 'update' | 'skip' | 'error'
- `create_missing`: boolean

### Account Handler
- `update_mode`: 'update' | 'skip'
- `require_contract`: boolean

### Payment Handler (planned)
- `deduplicate_by`: array of fields
- `create_booking`: boolean
- `create_activity`: boolean

## Migration Path

### Phase 1: Setup (Current)
- ✅ Create directory structure
- ✅ Add v2 columns to import_entities
- ✅ Create base interfaces and classes
- ✅ Implement ContractHandler and AccountHandler
- ✅ Create ProcessLargeImportJob
- ✅ Create seeder for entity configurations

### Phase 2: Implementation
- [ ] Implement remaining handlers (Payment, Activity, Person, Contacts)
- [ ] Add comprehensive tests
- [ ] Update controllers to use ImportServiceV2
- [ ] Add feature flag to toggle between v1 and v2

### Phase 3: Migration
- [ ] Run both systems in parallel
- [ ] Compare results and fix discrepancies
- [ ] Migrate all imports to v2
- [ ] Remove ImportProcessor.php (v1)

## Testing

```bash
# Run migrations
php artisan migrate

# Seed entity configurations
php artisan db:seed --class=ImportEntitiesV2Seeder

# Run tests
php artisan test --filter=ImportServiceV2
```

## Benefits Over V1

1. **Maintainability**: Each entity has its own handler, easier to understand and modify
2. **Testability**: Handlers can be tested independently
3. **Extensibility**: New entities added without touching core service
4. **Configuration**: Business rules in database, no code deployment needed
5. **Queue Support**: Built-in queue support for large imports
6. **Validation**: Entity-level validation separate from processing logic
7. **Priority Control**: Process entities in configurable order
8. **Reusability**: Handlers can be reused across different import scenarios

## Simulation Service

ImportSimulationServiceV2 provides a way to preview what an import would do without persisting any data to the database. This is useful for:
- Validating mappings before processing
- Previewing create/update actions
- Detecting errors before running actual import
- Testing handler logic

### Usage

```php
use App\Services\Import\ImportSimulationServiceV2;

$service = app(ImportSimulationServiceV2::class);

// Simulate first 100 rows (default)
$result = $service->simulate($import);

// Simulate 50 rows with verbose output
$result = $service->simulate($import, limit: 50, verbose: true);

// Result structure:
// [
//     'success' => true,
//     'total_simulated' => 50,
//     'limit' => 50,
//     'summaries' => [
//         'contract' => ['create' => 10, 'update' => 5, 'skip' => 0, 'invalid' => 1],
//         'account' => ['create' => 20, 'update' => 3, 'skip' => 0, 'invalid' => 0],
//     ],
//     'rows' => [
//         [
//             'row_number' => 2,
//             'entities' => [
//                 'contract' => [
//                     'action' => 'update',
//                     'reference' => 'CNT-001',
//                     'existing_id' => 123,
//                     'data' => ['reference', 'title', 'amount'],
//                     'changes' => ['title' => ['old' => 'Old', 'new' => 'New']],
//                 ],
//             ],
//             'warnings' => [],
//             'errors' => [],
//         ],
//     ],
//     'meta' => [
//         'has_header' => true,
//         'delimiter' => ',',
//         'mappings_count' => 8,
//     ],
// ]
```

### CLI Command

```bash
# Simulate import with ID 123
php artisan import:simulate-v2 123

# Simulate with custom limit
php artisan import:simulate-v2 123 --limit=50

# Verbose mode shows field-level changes
php artisan import:simulate-v2 123 --verbose
```

### Action Types

- **create**: Entity doesn't exist, would be created
- **update**: Entity exists, would be updated
- **skip**: Entity exists but update_mode is 'skip'
- **invalid**: Validation failed
- **error**: Processing error occurred

### Comparison with V1 Simulation

| Feature | ImportSimulationService (V1) | ImportSimulationServiceV2 |
|---------|------------------------------|---------------------------|
| Handler-based | ❌ Hardcoded logic | ✅ Uses V2 handlers |
| Configuration | ❌ In code | ✅ From database |
| Validation | ❌ Manual | ✅ Handler validation |
| Extensibility | ❌ Modify service | ✅ Add handlers |
| Change detection | ✅ Yes | ✅ Yes |
| Priority ordering | ❌ Fixed | ✅ Configurable |
| Error handling | ✅ Basic | ✅ Comprehensive |

## Original ImportProcessor.php

The original file remains at `app/Services/ImportProcessor.php` and can be used as reference for implementing remaining handlers.