Teren-app/app/Services/Import

Import System V2 Architecture

Overview

ImportServiceV2 is a refactored, database-driven import processing system that replaces the monolithic ImportProcessor.php with a modular, maintainable architecture.

Key Features

• Database-driven configuration: Entity processing rules, validation, and handlers configured in import_entities table
• Pluggable handlers: Each entity type has its own handler class implementing EntityHandlerInterface
• Queue support: Large imports can be processed asynchronously via ProcessLargeImportJob
• Validation: Entity-level validation rules stored in database
• Priority-based processing: Entities processed in configured priority order
• Extensible: Easy to add new entity types without modifying core service

Directory Structure

app/Services/Import/
├── Contracts/
│   └── EntityHandlerInterface.php    # Handler contract
├── Handlers/
│   ├── ContractHandler.php           # Contract entity handler
│   ├── AccountHandler.php            # Account entity handler
│   ├── PaymentHandler.php            # Payment handler (to be implemented)
│   ├── ActivityHandler.php           # Activity handler (to be implemented)
│   └── ...                            # Additional handlers
├── BaseEntityHandler.php             # Base handler with common logic
└── ImportServiceV2.php               # Main import service

Database Schema

import_entities Table

Column	Type	Description
id	bigint	Primary key
key	string	UI key (plural, e.g., "contracts")
canonical_root	string	Canonical root for processor (singular, e.g., "contract")
label	string	Human-readable label
fields	json	Array of field names
field_aliases	json	Field alias mappings
aliases	json	Root aliases
supports_multiple	boolean	Whether entity supports multiple items per row
meta	boolean	Whether entity is metadata
rules	json	Suggestion rules
ui	json	UI configuration
handler_class	string	Fully qualified handler class name
validation_rules	json	Laravel validation rules
processing_options	json	Handler-specific options
is_active	boolean	Whether entity is enabled
priority	integer	Processing priority (higher = first)
created_at	timestamp	Creation timestamp
updated_at	timestamp	Update timestamp

Handler Interface

All entity handlers must implement EntityHandlerInterface:

interface EntityHandlerInterface
{
    public function process(Import $import, array $mapped, array $raw, array $context = []): array;
    public function validate(array $mapped): array;
    public function getEntityClass(): string;
    public function resolve(array $mapped, array $context = []): mixed;
}

Handler Methods

• process(): Main processing method, returns result with action (inserted/updated/skipped) and entity
• validate(): Validates mapped data before processing
• getEntityClass(): Returns the model class name this handler manages
• resolve(): Resolves existing entity by key/reference

Creating a New Handler

1. Create handler class extending BaseEntityHandler:

<?php

namespace App\Services\Import\Handlers;

use App\Models\YourEntity;
use App\Models\Import;
use App\Services\Import\BaseEntityHandler;

class YourEntityHandler extends BaseEntityHandler
{
    public function getEntityClass(): string
    {
        return YourEntity::class;
    }

    public function resolve(array $mapped, array $context = []): mixed
    {
        // Implement entity resolution logic
        return YourEntity::where('key', $mapped['key'])->first();
    }

    public function process(Import $import, array $mapped, array $raw, array $context = []): array
    {
        $existing = $this->resolve($mapped, $context);

        if ($existing) {
            // Update logic
            $payload = $this->buildPayload($mapped, $existing);
            $appliedFields = $this->trackAppliedFields($existing, $payload);

            if (empty($appliedFields)) {
                return [
                    'action' => 'skipped',
                    'entity' => $existing,
                    'message' => 'No changes detected',
                ];
            }

            $existing->fill($payload);
            $existing->save();

            return [
                'action' => 'updated',
                'entity' => $existing,
                'applied_fields' => $appliedFields,
            ];
        }

        // Create logic
        $entity = new YourEntity;
        $payload = $this->buildPayload($mapped, $entity);
        $entity->fill($payload);
        $entity->save();

        return [
            'action' => 'inserted',
            'entity' => $entity,
            'applied_fields' => array_keys($payload),
        ];
    }

    protected function buildPayload(array $mapped, $model): array
    {
        // Map fields to model attributes
        return [
            'field1' => $mapped['field1'] ?? null,
            'field2' => $mapped['field2'] ?? null,
        ];
    }
}

2. Add configuration to import_entities table:

ImportEntity::create([
    'key' => 'your_entities',
    'canonical_root' => 'your_entity',
    'label' => 'Your Entities',
    'fields' => ['field1', 'field2'],
    'handler_class' => \App\Services\Import\Handlers\YourEntityHandler::class,
    'validation_rules' => [
        'field1' => 'required|string',
        'field2' => 'nullable|integer',
    ],
    'processing_options' => [
        'update_mode' => 'update',
    ],
    'is_active' => true,
    'priority' => 100,
]);

Usage

Synchronous Processing

use App\Services\Import\ImportServiceV2;

$service = app(ImportServiceV2::class);
$results = $service->process($import, $user);

Queue Processing (Large Imports)

use App\Jobs\ProcessLargeImportJob;

ProcessLargeImportJob::dispatch($import, $user->id);

Processing Options

Handler-specific options stored in processing_options JSON column:

Contract Handler

• update_mode: 'update' | 'skip' | 'error'
• create_missing: boolean

Account Handler

• update_mode: 'update' | 'skip'
• require_contract: boolean

Payment Handler (planned)

• deduplicate_by: array of fields
• create_booking: boolean
• create_activity: boolean

Migration Path

Phase 1: Setup (Current)

• ✅ Create directory structure
• ✅ Add v2 columns to import_entities
• ✅ Create base interfaces and classes
• ✅ Implement ContractHandler and AccountHandler
• ✅ Create ProcessLargeImportJob
• ✅ Create seeder for entity configurations

Phase 2: Implementation

• Implement remaining handlers (Payment, Activity, Person, Contacts)
• Add comprehensive tests
• Update controllers to use ImportServiceV2
• Add feature flag to toggle between v1 and v2

Phase 3: Migration

• Run both systems in parallel
• Compare results and fix discrepancies
• Migrate all imports to v2
• Remove ImportProcessor.php (v1)

Testing

# Run migrations
php artisan migrate

# Seed entity configurations
php artisan db:seed --class=ImportEntitiesV2Seeder

# Run tests
php artisan test --filter=ImportServiceV2

Benefits Over V1

1. Maintainability: Each entity has its own handler, easier to understand and modify
2. Testability: Handlers can be tested independently
3. Extensibility: New entities added without touching core service
4. Configuration: Business rules in database, no code deployment needed
5. Queue Support: Built-in queue support for large imports
6. Validation: Entity-level validation separate from processing logic
7. Priority Control: Process entities in configurable order
8. Reusability: Handlers can be reused across different import scenarios

Simulation Service

ImportSimulationServiceV2 provides a way to preview what an import would do without persisting any data to the database. This is useful for:

• Validating mappings before processing
• Previewing create/update actions
• Detecting errors before running actual import
• Testing handler logic

Usage

use App\Services\Import\ImportSimulationServiceV2;

$service = app(ImportSimulationServiceV2::class);

// Simulate first 100 rows (default)
$result = $service->simulate($import);

// Simulate 50 rows with verbose output
$result = $service->simulate($import, limit: 50, verbose: true);

// Result structure:
// [
//     'success' => true,
//     'total_simulated' => 50,
//     'limit' => 50,
//     'summaries' => [
//         'contract' => ['create' => 10, 'update' => 5, 'skip' => 0, 'invalid' => 1],
//         'account' => ['create' => 20, 'update' => 3, 'skip' => 0, 'invalid' => 0],
//     ],
//     'rows' => [
//         [
//             'row_number' => 2,
//             'entities' => [
//                 'contract' => [
//                     'action' => 'update',
//                     'reference' => 'CNT-001',
//                     'existing_id' => 123,
//                     'data' => ['reference', 'title', 'amount'],
//                     'changes' => ['title' => ['old' => 'Old', 'new' => 'New']],
//                 ],
//             ],
//             'warnings' => [],
//             'errors' => [],
//         ],
//     ],
//     'meta' => [
//         'has_header' => true,
//         'delimiter' => ',',
//         'mappings_count' => 8,
//     ],
// ]

CLI Command

# Simulate import with ID 123
php artisan import:simulate-v2 123

# Simulate with custom limit
php artisan import:simulate-v2 123 --limit=50

# Verbose mode shows field-level changes
php artisan import:simulate-v2 123 --verbose

Action Types

• create: Entity doesn't exist, would be created
• update: Entity exists, would be updated
• skip: Entity exists but update_mode is 'skip'
• invalid: Validation failed
• error: Processing error occurred

Comparison with V1 Simulation

Feature	ImportSimulationService (V1)	ImportSimulationServiceV2
Handler-based	❌ Hardcoded logic	✅ Uses V2 handlers
Configuration	❌ In code	✅ From database
Validation	❌ Manual	✅ Handler validation
Extensibility	❌ Modify service	✅ Add handlers
Change detection	✅ Yes	✅ Yes
Priority ordering	❌ Fixed	✅ Configurable
Error handling	✅ Basic	✅ Comprehensive

Original ImportProcessor.php

The original file remains at app/Services/ImportProcessor.php and can be used as reference for implementing remaining handlers.