banks2ff/specs/planning.md

# Implementation Plan: Bank2FF Refactoring

## 1. Objective
Refactor the `bank2ff` application from a prototype script into a robust, testable, and observable production-grade CLI tool. The application must synchronize bank transactions from GoCardless to Firefly III.

**Key Constraints:**
- **Multi-Crate Workspace**: Separate crates for the CLI application and the API clients.
- **Hand-Crafted Clients**: No autogenerated code. Custom, strongly-typed clients for better UX.
- **Clean Architecture**: Hexagonal architecture within the main application.
- **Multi-Currency**: Accurate handling of foreign amounts.
- **Test-Driven**: Every component must be testable from the start.
- **Observability**: Structured logging (tracing) throughout the stack.
- **Healer Strategy**: Detect and heal historical duplicates that lack external IDs.
- **Dry Run**: Safe mode to preview changes.
- **Rate Limit Handling**: Smart caching and graceful skipping to respect 4 requests/day limits.
- **Robust Agreement Handling**: Gracefully handle expired GoCardless EUAs without failing entire sync.

## 2. Architecture

### Workspace Structure
The project uses a Cargo Workspace with three members:

1.  `gocardless-client`: A reusable library crate wrapping the GoCardless Bank Account Data API v2.
2.  `firefly-client`: A reusable library crate wrapping the Firefly III API v6.4.4.
3.  `banks2ff`: The main CLI application containing the Domain Core and Adapters that use the client libraries.

### Directory Layout
```text
root/
├── Cargo.toml                  # Workspace definition
├── gocardless-client/          # Crate 1
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── client.rs           # Reqwest client logic
│       ├── models.rs           # Request/Response DTOs
│       └── tests/              # Unit/Integration tests with mocks
├── firefly-client/             # Crate 2
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs
│       ├── client.rs           # Reqwest client logic
│       ├── models.rs           # Request/Response DTOs
│       └── tests/              # Unit/Integration tests with mocks
└── banks2ff/                   # Crate 3 (Main App)
    ├── Cargo.toml
    └── src/
        ├── main.rs
        ├── core/               # Domain
        │   ├── models.rs
        │   ├── ports.rs        # Traits (Mockable)
        │   ├── sync.rs         # Logic (Tested with mocks)
        │   └── tests/          # Unit tests for logic
        └── adapters/           # Integration Layers
            ├── gocardless/
            │   ├── client.rs   # Uses gocardless-client & Cache
            │   ├── cache.rs    # JSON Cache for Account details
            │   └── mapper.rs
            └── firefly/
                └── client.rs   # Uses firefly-client
```

## 3. Observability Strategy
-   **Tracing**: All crates will use the `tracing` crate.
-   **Spans**:
    -   `client` crates: Create spans for every HTTP request (method, URL).
    -   `banks2ff` adapter: Create spans for "Fetching transactions".
    -   `banks2ff` sync: Create a span per account synchronization.
-   **Context**: IDs (account, transaction) must be attached to log events.

## 4. Testing Strategy
-   **Client Crates**:
    -   Use `wiremock` to mock the HTTP server.
    -   Test parsing of *real* JSON responses (saved in `tests/fixtures/`).
    -   Verify correct request construction (Headers, Auth, Body).
-   **Core (`banks2ff`)**:
    -   Use `mockall` to mock `TransactionSource` and `TransactionDestination` traits.
    -   Unit test `sync::run_sync` logic (filtering, flow control) without any I/O.
-   **Adapters (`banks2ff`)**:
    -   Test mapping logic (Client DTO -> Domain Model) using unit tests with fixtures.

## 5. Implementation Steps

### Phase 1: Infrastructure & Workspace
- [x] **Setup**: Initialize `gocardless-client` and `firefly-client` crates. Update root `Cargo.toml`.
- [x] **Dependencies**: Add `reqwest`, `serde`, `thiserror`, `url`, `tracing`.
- [x] **Test Deps**: Add `wiremock`, `tokio-test`, `serde_json` (dev-dependencies).

### Phase 2: Core (`banks2ff`)
- [x] **Definitions**: Implement `models.rs` and `ports.rs` in `banks2ff`.
- [x] **Mocks**: Add `mockall` attribute to ports for easier testing.

### Phase 3: GoCardless Client Crate
- [x] **Models**: Define DTOs in `gocardless-client/src/models.rs`.
- [x] **Fixtures**: Create `tests/fixtures/gc_transactions.json` (real example data).
- [x] **Client**: Implement `GoCardlessClient`.
- [x] **Tests**: Write `tests/client_test.rs` using `wiremock` to serve the fixture and verify the client parses it correctly.

### Phase 4: GoCardless Adapter (`banks2ff`)
- [x] **Implementation**: Implement `TransactionSource`.
- [x] **Logic**: Handle **Multi-Currency** (inspect `currencyExchange`).
- [x] **Tests**: Unit test the *mapping logic* specifically. Input: GC Client DTO. Output: Domain Model. Assert foreign amounts are correct.
- [x] **Optimization**: Implement "Firefly Leading" strategy (only fetch wanted accounts).
- [x] **Optimization**: Implement Account Cache & Rate Limit handling.

### Phase 5: Firefly Client Crate
- [x] **Models**: Define DTOs in `firefly-client/src/models.rs`.
- [x] **Fixtures**: Create `tests/fixtures/ff_store_transaction.json`.
- [x] **Client**: Implement `FireflyClient`.
- [x] **Tests**: Write `tests/client_test.rs` using `wiremock` to verify auth headers and body serialization.

### Phase 6: Firefly Adapter (`banks2ff`)
- [x] **Implementation**: Implement `TransactionDestination`.
- [x] **Logic**: Set `external_id`, handle Credit/Debit swap.
- [x] **Tests**: Unit test mapping logic. Verify `external_id` is populated.
- [x] **Update**: Refactor for "Healer" strategy (split `ingest` into `find`, `create`, `update`).

### Phase 7: Synchronization Engine
- [x] **Logic**: Implement `banks2ff::core::sync::run_sync` with "Healer" logic.
    -   Check Destination for existing transaction (Windowed Search).
    -   If found without ID: Heal (Update).
    -   If found with ID: Skip.
    -   If not found: Create.
- [x] **Smart Defaults**: Implement default start date (Last Firefly Date + 1) and end date (Yesterday).
- [x] **Tests**: Update unit tests for the new flow.

### Phase 8: Wiring & CLI
- [x] **CLI**: Add `-s/--start` and `-e/--end` arguments.
- [x] **CLI**: Add `--dry-run` argument.
- [x] **Wiring**: Pass these arguments to the sync engine.
- [x] **Observability**: Initialize `tracing_subscriber` with env filter.
- [x] **Config**: Load Env vars.

## 6. Multi-Currency Logic
- **GoCardless Adapter**:
  - `foreign_currency` = `currencyExchange[0].sourceCurrency`
  - `foreign_amount` = `amount` * `currencyExchange[0].exchangeRate`
  - Test this math explicitly in Phase 4 tests.