Explainable text model

Model lab

Try a lightweight local helper, then see how the real Python model performed on later complaints.

Use it forA shortlist, not a decision

Product (optional, but improves matching)Complaint description

103 / 1,200

Local instant check

Closest issue queues

01Managing an account100%

Matched: bankaccountcheckcheckingchecking account

02Written notification about debt0%

Matched: receivedreceive

03Other transaction problem0%

Matched: bankaccount

The selected product now removes issue queues that were not valid for that product in development data. Percentages still compare only these browser matches; they are not calibrated probabilities.

Untouched later-time test

What the score means.

Within every issue, the oldest 70% selected features, the next 15% selected the model, and the newest 15% was tested once. Exact repeated text and one dated CFPB label rename were harmonized first. A product constraint then removes issue queues never seen for that product in development data.

Exact accuracy71.8%

Correct on about 72 of every 100 later test rows.

Macro-F171.4%

Every issue class counts equally.

Top-three accuracy96.4%

The correct issue was in the shortlist.

95% range71.3–72.3%

Expected exact-accuracy range from this test.

Validation comparison

Six tested settings

Selected: Logistic SGD

ModelAccuracyMacro-F1Top three

Linear SVM · C 0.4573.3%73.1%95.5%

Linear SVM · C 0.7572.9%72.7%95.2%

Linear SVM · C 1.1072.5%72.3%95.1%

Logistic SGD74.5%74.3%96.2%

Modified-huber SGD72.5%72.3%94.2%

Complement Naive Bayes72.5%72.0%95.6%

These rows are validation results used to choose the model. The scorecard above is the separate final test.

Where it works

Results by issue

Some transaction and account issues are easier to separate. Debt and credit-reporting issues often use overlapping language.

Problem with a company's investigation into an existing problem46.8%

Written notification about debt47.9%

Improper use of your report49.1%

Attempts to collect debt not owed52.2%

False statements or representation53.5%

Incorrect information on your report55.2%

Took or threatened to take negative or legal action56.9%

Problem with a purchase shown on your statement97.3%

Dealing with your lender or servicer98.3%

Trouble during payment process99.3%

Managing an account99.8%

Logistic SGD

Eligible unique rows: 180,000
Train + validation: 153,000
Final test rows: 27,000
Issue classes: 12

Why it is not higher

Several debt and credit-reporting issue labels use nearly the same language.
The consumer-selected issue is a workflow label, not an adjudicated truth.
Using sub-issue to inflate accuracy would leak a field that already depends on issue.
Top-three performance is stronger, so the model remains a shortlist tool—not an automatic decision.