# Implementation Plan: Bank2FF Refactoring ## 1. Objective Refactor the `bank2ff` application from a prototype script into a robust, testable, and observable production-grade CLI tool. The application must synchronize bank transactions from GoCardless to Firefly III. **Key Constraints:** - **Multi-Crate Workspace**: Separate crates for the CLI application and the API clients. - **Hand-Crafted Clients**: No autogenerated code. Custom, strongly-typed clients for better UX. - **Clean Architecture**: Hexagonal architecture within the main application. - **Multi-Currency**: Accurate handling of foreign amounts. - **Test-Driven**: Every component must be testable from the start. - **Observability**: Structured logging (tracing) throughout the stack. - **Healer Strategy**: Detect and heal historical duplicates that lack external IDs. - **Dry Run**: Safe mode to preview changes. - **Rate Limit Handling**: Smart caching and graceful skipping to respect 4 requests/day limits. - **Robust Agreement Handling**: Gracefully handle expired GoCardless EUAs without failing entire sync. ## 2. Architecture ### Workspace Structure The project uses a Cargo Workspace with three members: 1. `gocardless-client`: A reusable library crate wrapping the GoCardless Bank Account Data API v2. 2. `firefly-client`: A reusable library crate wrapping the Firefly III API v6.4.4. 3. `banks2ff`: The main CLI application containing the Domain Core and Adapters that use the client libraries. ### Directory Layout ```text root/ ├── Cargo.toml # Workspace definition ├── gocardless-client/ # Crate 1 │ ├── Cargo.toml │ └── src/ │ ├── lib.rs │ ├── client.rs # Reqwest client logic │ ├── models.rs # Request/Response DTOs │ └── tests/ # Unit/Integration tests with mocks ├── firefly-client/ # Crate 2 │ ├── Cargo.toml │ └── src/ │ ├── lib.rs │ ├── client.rs # Reqwest client logic │ ├── models.rs # Request/Response DTOs │ └── tests/ # Unit/Integration tests with mocks └── banks2ff/ # Crate 3 (Main App) ├── Cargo.toml └── src/ ├── main.rs ├── core/ # Domain │ ├── models.rs │ ├── ports.rs # Traits (Mockable) │ ├── sync.rs # Logic (Tested with mocks) │ └── tests/ # Unit tests for logic └── adapters/ # Integration Layers ├── gocardless/ │ ├── client.rs # Uses gocardless-client & Cache │ ├── cache.rs # JSON Cache for Account details │ └── mapper.rs └── firefly/ └── client.rs # Uses firefly-client ``` ## 3. Observability Strategy - **Tracing**: All crates will use the `tracing` crate. - **Spans**: - `client` crates: Create spans for every HTTP request (method, URL). - `banks2ff` adapter: Create spans for "Fetching transactions". - `banks2ff` sync: Create a span per account synchronization. - **Context**: IDs (account, transaction) must be attached to log events. ## 4. Testing Strategy - **Client Crates**: - Use `wiremock` to mock the HTTP server. - Test parsing of *real* JSON responses (saved in `tests/fixtures/`). - Verify correct request construction (Headers, Auth, Body). - **Core (`banks2ff`)**: - Use `mockall` to mock `TransactionSource` and `TransactionDestination` traits. - Unit test `sync::run_sync` logic (filtering, flow control) without any I/O. - **Adapters (`banks2ff`)**: - Test mapping logic (Client DTO -> Domain Model) using unit tests with fixtures. ## 5. Implementation Steps ### Phase 1: Infrastructure & Workspace - [x] **Setup**: Initialize `gocardless-client` and `firefly-client` crates. Update root `Cargo.toml`. - [x] **Dependencies**: Add `reqwest`, `serde`, `thiserror`, `url`, `tracing`. - [x] **Test Deps**: Add `wiremock`, `tokio-test`, `serde_json` (dev-dependencies). ### Phase 2: Core (`banks2ff`) - [x] **Definitions**: Implement `models.rs` and `ports.rs` in `banks2ff`. - [x] **Mocks**: Add `mockall` attribute to ports for easier testing. ### Phase 3: GoCardless Client Crate - [x] **Models**: Define DTOs in `gocardless-client/src/models.rs`. - [x] **Fixtures**: Create `tests/fixtures/gc_transactions.json` (real example data). - [x] **Client**: Implement `GoCardlessClient`. - [x] **Tests**: Write `tests/client_test.rs` using `wiremock` to serve the fixture and verify the client parses it correctly. ### Phase 4: GoCardless Adapter (`banks2ff`) - [x] **Implementation**: Implement `TransactionSource`. - [x] **Logic**: Handle **Multi-Currency** (inspect `currencyExchange`). - [x] **Tests**: Unit test the *mapping logic* specifically. Input: GC Client DTO. Output: Domain Model. Assert foreign amounts are correct. - [x] **Optimization**: Implement "Firefly Leading" strategy (only fetch wanted accounts). - [x] **Optimization**: Implement Account Cache & Rate Limit handling. ### Phase 5: Firefly Client Crate - [x] **Models**: Define DTOs in `firefly-client/src/models.rs`. - [x] **Fixtures**: Create `tests/fixtures/ff_store_transaction.json`. - [x] **Client**: Implement `FireflyClient`. - [x] **Tests**: Write `tests/client_test.rs` using `wiremock` to verify auth headers and body serialization. ### Phase 6: Firefly Adapter (`banks2ff`) - [x] **Implementation**: Implement `TransactionDestination`. - [x] **Logic**: Set `external_id`, handle Credit/Debit swap. - [x] **Tests**: Unit test mapping logic. Verify `external_id` is populated. - [x] **Update**: Refactor for "Healer" strategy (split `ingest` into `find`, `create`, `update`). ### Phase 7: Synchronization Engine - [x] **Logic**: Implement `banks2ff::core::sync::run_sync` with "Healer" logic. - Check Destination for existing transaction (Windowed Search). - If found without ID: Heal (Update). - If found with ID: Skip. - If not found: Create. - [x] **Smart Defaults**: Implement default start date (Last Firefly Date + 1) and end date (Yesterday). - [x] **Tests**: Update unit tests for the new flow. ### Phase 8: Wiring & CLI - [x] **CLI**: Add `-s/--start` and `-e/--end` arguments. - [x] **CLI**: Add `--dry-run` argument. - [x] **Wiring**: Pass these arguments to the sync engine. - [x] **Observability**: Initialize `tracing_subscriber` with env filter. - [x] **Config**: Load Env vars. ## 6. Multi-Currency Logic - **GoCardless Adapter**: - `foreign_currency` = `currencyExchange[0].sourceCurrency` - `foreign_amount` = `amount` * `currencyExchange[0].exchangeRate` - Test this math explicitly in Phase 4 tests.