hledger-preprocessor
User Story DAG Explorer
Step 1a: Account Configuration
US-1a.1: Configure a single bank account with CSV import ▶
US-1a.2: Configure multiple bank accounts ▶
US-1a.3: Configure a cash wallet (no CSV) ▶
US-1a.4: Configure a cryptocurrency exchange account ▶
US-1a.5: Configure matching algorithm parameters ▶
US-1a.6: Configure base currency for reporting ▶
US-1a.7: Configure multi-row CSV merging for exchanges with split transactions ▶
Step 1b: Category Configuration
US-1b.1: Define hierarchical spending categories ▶
US-1b.2: Add a new category after initial setup
US-1b.3: Use categories for income as well as expenses ▶
Step 2a: Receipt Image Processing
US-2a.1: Rotate a receipt image
US-2a.2: Crop a receipt image
US-2a.3: Process a batch of receipt images
Step 2b: Receipt Labelling
US-2b.1: Label a simple same-currency card receipt ▶
US-2b.2: Label a cash receipt ▶
US-2b.3: Label a foreign-currency receipt ▶
US-2b.4: Label a split-payment receipt (card + cash) ▶
US-2b.5: Label a receipt with returned items ▶
US-2b.6: Use AI suggestions during manual labelling
US-2b.7: Edit an existing receipt label
US-2b.8: Fully automated AI receipt labelling
US-2b.9: Date-range error feedback during receipt labelling
US-2b.10: Inline matching CLI when no unique CSV match ▶
Step 3: Receipt-to-CSV Transaction Matching
US-3.1: Auto-match a simple same-currency receipt
US-3.2: Match a foreign-currency withdrawal receipt to a bank CSV in a different currency
US-3.3: Match when no candidates are found (widen date range)
US-3.4: Match when no candidates are found (widen amount range)
US-3.5: Match when DD-MM and MM-DD date formats are swapped
US-3.6: Disambiguate when multiple matches are found (2-14 candidates)
US-3.7: Reduce search scope when too many matches are found (15+)
US-3.8: Correct a receipt label during matching
US-3.9: Match a receipt for a direct asset purchase (e.g. gold)
US-3.10: Skip matching for cash-only receipts
US-3.11: Handle a receipt with withdrawal fees
US-3.12: Handle multiple transactions on the same account in one receipt
US-3.13: Handle a foreign-currency receipt with returned items
US-3.14: Prevent linking the same CSV transaction to two receipts
US-3.15: Verify transaction data is up to date
Step 4: Pipeline Execution
US-4.1: Run the full pipeline end-to-end
US-4.2: Run the pipeline with randomised/scrambled data for demos
US-4.3: Generate hledger rules files for a bank
US-4.4: Optional incremental pipeline runs
US-4.5: Include opening balances from a starting journal
US-4.6: Detect uncategorised transactions before interactive matching
US-4.7: Report unmatched transactions before interactive matching
US-4.8: Auto-check receipt matching after TUI labelling
Step 5: Visualisation
US-5.1: Generate a Sankey diagram of money flows
US-5.2: Generate a Treemap of spending by category
US-5.3: Launch an interactive Dash dashboard
US-5.4: Filter visualisations by time period
US-5.5: Generate monthly/quarterly/yearly reports
US-5.6: Calculate personal inflation rate
US-5.7: Show a correlation matrix of combined-account payments
US-5.8: Drill down from treemap to time-series chart
US-5.9: Sticky toolbar with keyboard shortcuts
US-5.10: Select period granularity (weekly / monthly / yearly)
US-5.11: Hierarchical legend with group toggle in drill-down
Transaction Classification
US-C.1: Classify transactions using rule-based logic
US-C.2: Classify transactions using a self-hosted AI
US-C.3: Train a classification model on my own categorised data
US-C.4: Classify receipt images by category using AI
Cross-cutting Concerns
US-X.1: Privacy: no data leaves my machine
US-X.2: Reproducible pipeline output
US-X.3: Multi-bank, multi-currency support
US-X.4: Unique transaction hashes prevent duplicates
US-X.5: GIF demos are auto-generated from integration tests
US-X.6: Enforce one receipt image per transaction
User Story DAG
Interactive explorer — use
→
to start cycling through stories.
Configuration
Receipt Labelling
Account Configuration
Bitvavo trading account — single-row atomic trades
Bitvavo exchange
(atomic CSV)
BTC digital wallet / exchange
BTC digital
wallet
EUR cash wallet — no CSV import
EUR physical
wallet
GBP cash wallet — foreign currency
GBP physical
wallet
GOLD physical asset wallet
GOLD physical
wallet
ING checking account — imports from CSV
ING checking
(EUR CSV)
Kraken trading account — split spend/receive rows linked by refid
Kraken exchange
(multi-row CSV)
SILVER physical asset wallet
SILVER physical
wallet
Triodos checking account — imports from CSV
Triodos checking
(EUR CSV)
Directory Paths
Standard directory layout: import/{holder}/{bank}/{type}/{year}/
Default dir paths
Filename Convention
Standard filename conventions for CSV, rules, and journal files
Default filenames
Categorisation Config
Standard categorisation settings from config.yaml
Default categorisation
Matching Algo Config
Standard matching algorithm settings from config.yaml
Default matching algo
Categories
groceries:{ekoplaza,supermarket}, repairs:{bike}, withdrawl:{euro:{pound}}
basic categories
basic + dining, transport, bankfees, investments:{gold,silver}
extended categories
basic + income:{salary,freelance}
categories + income
Matching Parameters
days:2, amount_range:0, days_month_swap:true
default (±2d, exact)
days:1, amount_range:0 — initially too tight
narrow (±1d, exact)
days:5, amount_range:0.05 — triggers 15+ matches
wide both (±5d, ±0.05)
days:5, amount_range:0
wide date (±5d)
Starting Journal
Opening balance: Assets:Triodos:Checking 1000 EUR
2024: 1000 EUR
Bank CSV Transactions
Foreign ATM withdrawal: 100 GBP * 1.175
Triodos -117.50 EUR
2025-01-15 ATM London
200 GBP * 1.17 + 3.50 EUR fee
Triodos -237.50 EUR
2025-01-15 ATM London
Posted 3 days late (receipt dated Jan 15)
Triodos -49.99 EUR
2025-01-18 ShopX
Same amount different day — ambiguous match candidate
Triodos -42.17 EUR
2025-01-14 Ekoplaza
Debit card purchase at Ekoplaza, exact date match
Triodos -42.17 EUR
2025-01-15 Ekoplaza
Same ekoplaza 42.17, bank posted 3 days late (receipt Jan 15)
Triodos -42.17 EUR
2025-01-18 Ekoplaza
Similar amount different day — ambiguous match candidate
Triodos -43.00 EUR
2025-01-16 Ekoplaza
10g gold purchase at 58 EUR/g
Triodos -580.00 EUR
2025-01-20 GoldDealer
Bank rounded 49.99 to 50.00
Triodos -50.00 EUR
2025-01-15 ShopX
Card portion of 50 EUR split dinner
Triodos -30.00 EUR
2025-02-01 Restaurant
Receipt entered as 01-05 but meant 05-01
Triodos -25.00 EUR
2025-05-01 BookShop
Receipt Images
ATM withdrawal slip, 100 GBP
atm_london.jpg
ATM withdrawal slip, 200 GBP + fee
atm_london_fee.jpg
Bike repair receipt, cash
bike_repair.jpg
Coffee receipt, paid cash
coffee_cash.jpg
Receipt dated Jan 15, bank posted Jan 18
delayed_shop.jpg
Ekoplaza receipt, paid by card
example_card.jpg
Gold purchase receipt, 10g
gold_dealer.jpg
Receipt with bought + returned items
return_item.jpg
Receipt 49.99, bank shows 50.00
rounded_shop.jpg
Restaurant receipt, split card+cash
split_dinner.jpg
Receipt with ambiguous DD-MM date
bookshop.jpg
No label exists yet
No receipt label JSON exists yet for ATM withdrawal receipt
atm_100gbp
(no label)
No receipt label JSON exists yet for ATM withdrawal with fee
atm_200gbp
(no label)
No receipt label JSON exists yet for bike repair
bike_repair
(no label)
No receipt label JSON exists yet for the coffee_cash receipt
coffee_cash
(no label)
No receipt label JSON exists yet for delayed shop
delayed_shop
(no label)
No receipt label JSON exists yet for dinner split
dinner_split
(no label)
No receipt label JSON exists yet for the example_card receipt
example_card
(no label)
No receipt label JSON exists yet for gold purchase
gold_10g
(no label)
No receipt label JSON exists yet for return
return_item
(no label)
No receipt label JSON exists yet for rounded shop
rounded_shop
(no label)
No receipt label JSON exists yet for swapped date
bookshop
(no label)
Label receipt in TUI
Label ATM withdrawal receipt via TUI
TUI label
atm_100gbp
Label ATM withdrawal with fee via TUI
TUI label
atm_200gbp
Label bike repair receipt via TUI
TUI label
bike_repair
Label coffee receipt via TUI
TUI label
coffee_cash
Label delayed shop receipt via TUI
TUI label
delayed_shop
Label dinner split receipt via TUI
TUI label
dinner_split
Label ekoplaza card receipt via TUI based on image
TUI label
ekoplaza_card
Label gold purchase receipt via TUI
TUI label
gold_10g
Label return receipt via TUI
TUI label
return_item
Label rounded shop receipt via TUI
TUI label
rounded_shop
Label swapped date receipt via TUI
TUI label
bookshop
Receipt label output JSON
Triodos account, GBP withdrawal receipt
atm_100gbp
GBP 100 withdrawal
Triodos account, GBP withdrawal with bank fee
atm_200gbp
GBP 200 + fee
EUR wallet, repairs:bike
bike_repair
EUR 14.50 cash
wallet wallet, 2025-02-10, food:coffee
coffee_cash
EUR 20 cash
Triodos account, receipt dated Jan 15
delayed_shop
EUR 49.99 card
Triodos 30 EUR + wallet 20 EUR
dinner_split
30 card + 20 cash
triodos account, 2025-01-15, groceries:ekoplaza
example_card
EUR 42.17 card
Triodos account, Currency.GOLD
gold_10g
10 GRAMS
Net EUR 50.00 (bought 75.00 - returned 25.00)
return_item
bought 3, returned 1
Triodos account, bank will show 50.00
rounded_shop
EUR 49.99 card
Triodos account, date entered as 01-05-2025
bookshop
EUR 25.00 card
Matching Outcome
Physical asset conversion (GRAMS -> EUR)
ASSET CONVERT
gold rate
Exact date+amount, no TUI needed
AUTO-LINK
1 match found
User fixes receipt label, matcher retries
CORRECT RECEIPT
edit inline
CSV transaction classified without receipt
CLASSIFY
rule/AI based
Receipt GBP != CSV EUR, user provides conversion ratio
CURRENCY CONVERT
user enters rate
Foreign withdrawal with bank fee split
CURRENCY + FEE
rate + fee amount
User selects from weighted-score-ranked list
DISAMBIGUATE
3 candidates ranked
CSV transaction already claimed by another receipt
BLOCKED
already linked
Cash receipt, no bank CSV to match against
SKIP
no CSV for wallet
Auto-retry with day/month swapped
SWAP DD/MM
01-05 -> 05-01
User tightens margins, retries
TOO MANY (15+)
user reduces scope
Mismatch in TUI, user widens date range via inline matching CLI
TUI WIDEN DATE
±2d -> ±5d inline
No match at exact, user widens to ±0.02, finds match
WIDEN AMOUNT
0 -> ±0.02
No match at ±2d, user widens to ±5d, finds match
WIDEN DATE
±2d -> ±5d
Journal Output
Gold asset purchased
Assets:Gold
10 GRAMS
ATM withdrawal fee posting
Expenses:BankFees
3.50 EUR
Debit food:coffee, credit wallet wallet
Expenses:Food:Coffee
20 EUR
Delayed posting matched after widening date
Expenses:Shopping
49.99 EUR
Card portion of split dinner
Expenses:Dining:
Restaurant 30 EUR (card)
Cash portion of split dinner
Expenses:Dining:
Restaurant 20 EUR (cash)
Debit groceries:ekoplaza, credit triodos checking
Expenses:Groceries:Ekoplaza
42.17 EUR
Debit repairs, credit EUR wallet
Expenses:Repairs:
Bike 14.50 EUR
Net after return: 42.50 bought - 14.00 returned
Expenses:Shopping
28.50 EUR (net)
Bank-rounded amount matched after widening amount
Expenses:Shopping
50.00 EUR
Date-swapped receipt matched after DD/MM fix
Expenses:Shopping
25.00 EUR
Bank debit for GBP withdrawal
Assets:Triodos:
Checking -117.50 EUR
Bank debit for gold purchase
Assets:Triodos:
Checking -580 EUR
Foreign currency asset from ATM withdrawal
Assets:Wallet:
GBP 100
Foreign currency asset (large withdrawal)
Assets:Wallet:
GBP 200
Visualisation
Interactive Plotly Sankey of all flows
Sankey Diagram
Interactive Plotly Treemap of spending
Treemap Plot