Credit Card Transaction Merchant Normalization

Credit Card Transaction Merchant Normalization

Messy transaction log descriptions must be mapped to canonical merchant categories using a character n-gram similarity metric.

Implement a function normalize_merchant(raw_transaction, canonical_merchants, threshold) that:

1. Removes all special characters, non-alphabetic symbols, and numbers from the transaction description (preserving spaces).

2. Converts the string to lowercase and splits it into character 3-grams (e.g., "starbucks" -> ['sta', 'tar', 'arb', 'rbu', 'buc', 'uck', 'cks']).

3. Calculates the Jaccard similarity coefficient between the 3-gram set of the cleaned transaction and the 3-gram set of each canonical merchant name.

   $\text{Jaccard}(A, B) = \frac{|A \cap B|}{|A \cup B|}$

   If the union is empty, Jaccard similarity is 0.0.

4. Returns the name of the canonical merchant with the highest similarity, provided it meets or exceeds the threshold. Returns "Unknown" if no match meets the threshold.

Example

raw = "STARBUCKS COFFEE #1234"
merchants = ["McDonalds", "Starbucks", "Walmart"]
normalize_merchant(raw, merchants, 0.3)
# Returns: "Starbucks"