Laboratory / GenAI / [001] BSA/AML Regulatory RAG
[GENAI-001] · Databricks · Live

BSA/AML Regulatory RAG

Area · GenAI Stack · LangChain · Gemini 2.0 · ChromaDB · BM25 Dataset · FFIEC BSA/AML Examination Manual Live
Overview
Architecture
GitHub

The Problem

"Our compliance team spends hours searching the FFIEC manual to answer internal policy questions. We need a faster and more reliable way to query regulatory documentation."

Every financial institution operating in the United States must comply with the Bank Secrecy Act and Anti Money Laundering regulations. The documentation is dense: FinCEN guidelines, FFIEC manuals, SAR filing requirements, and compliance teams spend significant time navigating it manually.

This project builds a retrieval augmented generation system that answers regulatory questions in plain English, with every response citing the exact source document, section, and page number. The system uses a hybrid retrieval approach and is configured to run deterministically at temperature 0.0.

What It Delivers

Query speed
Regulatory questions that take 20 to 40 minutes to answer manually are answered in under 10 seconds.
Source citation
Every answer references the exact document, section, and page. Compliance officers can verify the source without relying on the model alone.
Pluggable stack
The LLM, embedding model, and vector store are all configurable via a single .env file. No code changes required to swap any component.
Deterministic output
Temperature is fixed at 0.0. The same question asked twice returns the same answer, which is a requirement in any compliance context.

Pipeline Overview

BSA/AML Regulatory RAG Architecture
Fig. 1 · End to end pipeline. All layers are configurable via config.py and .env.

Key Decisions

Temperature 0.0
Regulatory answers must be deterministic. Any temperature above zero introduces variability, and the same question can return different answers, which is not acceptable in a compliance context.
Hybrid retrieval
Regulatory documents use precise legal terms such as "structuring" and "SAR". Pure semantic search does not reliably match these. BM25 keyword search covers what embeddings miss. The union of both returns better results than either alone.
Chunk size 500
Regulatory text is dense. Larger chunks mix multiple obligations into a single unit, reducing retrieval precision. 500 characters keeps each chunk semantically focused without losing context.
Mandatory citation
The generation prompt instructs the model to cite the source page for every claim. This is enforced at the prompt level, not trusted to model behavior alone.

Repository

genai-001-regulatory-rag-bsa-aml github.com →
bsa-aml-rag/
├── README.md
├── requirements.txt
├── .env.example
├── config.py
├── src/
│ ├── __init__.py
│ ├── ingestion.py
│ ├── processing.py
│ ├── embedding.py
│ ├── retrieval.py
│ └── generation.py
├── notebooks/
│ └── quickstart.ipynb
└── data/
└── BsaAmlManualSectionsPackage.pdf

Stack: LangChain, Gemini 2.0 Flash, Gemini Embedding 001, ChromaDB, BM25 (rank_bm25). Python 3.11 or higher. Requires a Google AI Studio API key, free at aistudio.google.com.

Laboratory / Risk / [002] Customer Churn with Explainability
[RISK-001] · Databricks · Live

Customer Churn Scoring

Area · Risk Stack · XGBoost · Optuna · SHAP Dataset · Bank Customer Churn (Kaggle) Live
Overview
Architecture
GitHub

The Problem

Banks lose customers every day without knowing who is at risk or why.

This is a production-grade pipeline for predicting customer churn in banking, with automated feature engineering, Bayesian hyperparameter optimization, and SHAP-based explainability. It scores every customer by churn probability, ranks them by risk, and explains the exact factors driving each prediction so retention teams know who to prioritize and what to address.

Built to work with any tabular churn dataset. Swap the CSV, adjust the target column, and the pipeline handles the rest from raw data to a scored, explainable model.

What It Delivers

Automatic feature engineering
PolynomialFeatures generates candidate interactions automatically, then RandomForest importance decides which ones matter — reusable across different churn datasets without code changes.
Bayesian optimization
Optuna finds strong hyperparameters in far fewer trials than exhaustive grid search. The search space lives in config/xgb_params.py, so tuning depth is a configuration change, not a code change.
Business-first metrics
KS statistic and LIFT by decile instead of raw accuracy, which is misleading on imbalanced churn data. LIFT tells the business exactly how much better than random the model performs at each decile.
Saved artifacts
Label encoders, scaler, polynomial transformer, and selected feature list are all saved during training, allowing new customers to be scored with the exact same transformations, without retraining.

Metrics Explained

AUC-ROC
Overall discrimination power. 0.87+ is strong for churn.
Gini
Derived from AUC. 0.70+ is considered very good in banking.
KS
Maximum separation between churner and non-churner distributions. 0.50+ is a strong model.
LIFT
How many times more churners are found in a decile compared to random selection. A decile 10 LIFT of 4x means targeting the top 10% of scores finds 4 times more churners than reaching out randomly.

Pipeline Overview

Customer Churn Scoring Architecture
Fig. 1 · Raw data to scored, explainable model. Every step saves its output to data/, models/, or outputs/, so the pipeline can be run in full or resumed from any step.

Key Decisions

01 · EDA
Summary report and initial cleaning, producing churn_clean.parquet.
02 · Target Analysis
Distribution and class balance check before any modeling decision is made.
03 · Preprocessing
Label encoding and standard scaling applied consistently across train and inference.
04 · Feature Engineering
PolynomialFeatures generates candidate interactions and powers automatically.
05 · Feature Selection
RandomForest importance selects the top N features from the generated candidates.
06 · Training
XGBoost combined with Optuna Bayesian optimization for hyperparameter tuning.
07 · Evaluation
AUC-ROC, Gini, KS statistic, and LIFT curve by decile.
08 · Explainability
SHAP values for global importance and per-customer drivers.

Repository

risk-001-customer-churn-scoring github.com →
churn-scoring-pipeline/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ └── xgb_params.py
├── src/
│ ├── __init__.py
│ ├── eda.py
│ ├── target_analysis.py
│ ├── preprocessing.py
│ ├── feature_engineering.py
│ ├── feature_selection.py
│ ├── training.py
│ ├── evaluation.py
│ └── explainability.py
├── data/
│ ├── bank_customer_churn.csv
│ ├── churn_clean.parquet
│ ├── churn_preprocessed.parquet
│ ├── churn_features.parquet
│ └── churn_selected.parquet
├── models/
│ ├── xgb_churn.pkl
│ ├── label_encoders.pkl
│ ├── scaler.pkl
│ ├── polynomial.pkl
│ └── selected_features.pkl
├── outputs/
│ ├── ks_curve.png
│ ├── lift_curve.png
│ ├── shap_summary.png
│ ├── shap_bar.png
│ └── shap_waterfall.png
└── notebooks/
├── quickstart.ipynb
└── test_predict.ipynb

Stack: XGBoost, Optuna, SHAP, scikit-learn. Python 3.13. Works with any tabular churn dataset with a binary target column and minimal configuration changes.

Laboratory / Causal Inference / [003] Credit Intervention Analysis
[CAUSAL-001] · Databricks · Live

Credit Limit Intervention Analysis

Area · Causal Inference Stack · DoWhy · EconML · Causal Forest Dataset · UCI Credit Card Default (Taiwan) Live
Overview
Architecture
GitHub

The Hypothesis

Risk teams commonly assume that reducing the credit limit of customers with a history of late payments lowers their risk of future default. Less exposure, less risk — the reasoning seems straightforward.

This project tests that assumption using causal inference, not just correlation. Built with DoWhy and EconML to separate correlation from causation — and to challenge a common assumption in credit risk management.

What the Data Shows

Controlling for 22 confounders — payment history, bill amounts, demographics — the analysis finds the opposite of the common assumption. A one standard deviation increase in credit limit (approximately NT$130,000) decreases the probability of default by 1.19 percentage points.

This is consistent with published research on the same dataset, which found a similar effect using a different estimation method. The likely mechanism is that a higher credit limit reduces utilization ratio, which reduces financial pressure and, in turn, default risk.

The effect is not uniform across customers. Segmenting by individual treatment effect reveals that customers with lower existing limits and higher baseline risk benefit the most from a limit increase, while a small subgroup shows the opposite pattern — for them, increasing the limit is associated with higher default risk.

Key Terms

ATE
Average Treatment Effect — the average causal impact of the treatment across the full population.
CATE
Conditional Average Treatment Effect — the causal impact estimated for a specific customer or segment, which can differ from the average.
Backdoor criterion
The condition under which controlling for a set of confounders is sufficient to identify a causal effect from observational data.
Refutation test
A robustness check that perturbs the model in a known way to confirm the estimated effect behaves as a real causal effect should.

Pipeline Overview

Credit Limit Intervention Architecture
Fig. 1 · Naive correlation → causal graph → identification → effect estimation → refutation → heterogeneous effects.

Key Decisions

Causal question is declared, not discovered
Unlike the churn and RAG projects in this hub, this pipeline cannot fully automate itself from a new dataset. Treatment, outcome, and confounders require domain knowledge, made explicit in causal_config.py. Automating causal assumptions would risk producing confident but wrong conclusions.
Naive correlation reported first
The EDA step deliberately calculates the raw correlation between treatment and outcome before any adjustment, making the value of causal inference visible.
Effect reported at business scale
A raw ATE per one-unit change in a currency variable is not interpretable when that variable ranges in the hundreds of thousands. The pipeline reports the effect per meaningful increment and per standard deviation, matching how a finding would be communicated to a risk committee.
Refutation is not optional
Every causal estimate goes through three robustness checks: adding a random confounder, replacing the treatment with a placebo, and re-estimating on a data subset. A causal claim without refutation testing is a hypothesis that has not been stressed.
Heterogeneity as the main output
An average effect can hide the fact that a subgroup responds in the opposite direction. The Causal Forest step exists specifically to surface that nuance, enabling a segmented recommendation instead of a blanket policy change.

Repository

causal-001-credit-limit-intervention github.com →
causal-001-credit-limit-intervention/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ └── causal_config.py
├── src/
│ ├── __init__.py
│ ├── causal_eda.py
│ ├── causal_graph.py
│ ├── identification.py
│ ├── effect_estimation.py
│ ├── refutation.py
│ └── heterogeneous_effects.py
├── data/
│ ├── UCI_Credit_Card.csv
│ └── causal_clean.parquet
├── outputs/
│ ├── causal_graph.png
│ ├── cate_distribution.png
│ └── segment_effects.png
└── notebooks/
└── quickstart.ipynb

Stack: DoWhy, EconML, pandas. Python 3.13. Uses the Default of Credit Card Clients dataset (Taiwan), publicly available on Kaggle / UCI Machine Learning Repository.