Classify-1
ScaleDown · scaledown/classify-1 · added 20 Oct 2026
Classify-1 by ScaleDown ranks #16 of 17 AI decision models on the DecisionBench Rulebook benchmark (following written rules), with accuracy 71.1%. View on OpenRouter.
- Rulebook rank
- #16 of 17
- Accuracy
- 71.1%
- Team
- 100.0%
- Refund
- 78.3%
- Urgency
- 35.0%
- Confidence gap
- 0.109
60 tickets · 0.0% of mistakes made while at least 90% sure · $0.0897 per 1,000 tickets · 430 ms per ticket
Every ticket
The model's three decisions on each ticket, with its confidence. Wrong answers show the right one. 45 of 60 tickets had at least one mistake.
| Ticket | Customer wrote | Team | Refund | Urgency |
|---|---|---|---|---|
| T-1000 | My invoice says $425 but checkout showed $405. Why was I charged more? | ✓ billing 98% | ✕ deny → approve 64% | ✓ 1 57% |
| T-1001 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping 93% | ✓ deny 79% | ✓ 1 51% |
| T-1002 | I changed my mind about order #468974 and want to return it. | ✓ returns 93% | ✓ deny 60% | ✓ 2 42% |
| T-1003 | Sync stops halfway through and shows error E-883. | ✓ technical 95% | ✓ deny 60% | ✕ 2 → 1 50% |
| T-1004 | Someone changed the email on my account without my permission. | ✓ fraud 95% | ✓ deny 73% | ✕ 1 → 0 43% |
| T-1005 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing 98% | ✓ deny 64% | ✓ 1 59% |
| T-1006 | My package for order #822804 hasn't arrived and tracking hasn't moved in 7 days. | ✓ shipping 95% | ✓ approve 50% | ✓ 2 48% |
| T-1007 | I changed my mind about order #797563 and want to return it. | ✓ returns 94% | ✓ approve 53% | ✕ 2 → 1 36% |
| T-1008 | The app crashes every time I open the settings page. | ✓ technical 97% | ✓ deny 59% | ✕ 2 → 1 47% |
| T-1009 | Someone changed the email on my account without my permission. | ✓ fraud 95% | ✓ deny 80% | ✓ 3 49% |
| T-1010 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing 99% | ✓ approve 51% | ✕ 2 → 1 47% |
| T-1011 | The courier sent order #441011 to my old address instead of the one I entered. | ✓ shipping 58% | ✕ deny → approve 53% | ✓ 2 45% |
| T-1012 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 85% | ✓ approve 56% | ✕ 3 → 2 36% |
| T-1013 | Export to PDF has been broken since the last update. | ✓ fraud 91% | ✓ deny 73% | ✕ 3 → 1 36% |
| T-1014 | Someone changed the email on my account without my permission. | ✓ fraud 96% | ✓ deny 51% | ✓ 1 61% |
| T-1015 | My invoice says $62 but checkout showed $42. Why was I charged more? | ✓ billing 98% | ✕ approve → deny 52% | ✕ 1 → 2 39% |
| T-1016 | My package for order #455490 hasn't arrived and tracking hasn't moved in 4 days. | ✓ fraud 90% | ✓ deny 64% | ✕ 2 → 1 29% |
| T-1017 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 91% | ✓ approve 50% | ✕ 2 → 1 43% |
| T-1018 | Export to PDF has been broken since the last update. | ✓ fraud 86% | ✓ deny 60% | ✕ 2 → 1 39% |
| T-1019 | Someone changed the email on my account without my permission. | ✓ fraud 91% | ✓ deny 54% | ✕ 2 → 1 31% |
| T-1020 | I was charged twice for order #227551. Please fix the duplicate charge. | ✓ fraud 93% | ✓ deny 77% | ✕ 3 → 2 37% |
| T-1021 | My package for order #718743 hasn't arrived and tracking hasn't moved in 8 days. | ✓ shipping 92% | ✓ approve 59% | ✕ 2 → 3 36% |
| T-1022 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 84% | ✓ deny 61% | ✕ 3 → 4 41% |
| T-1023 | Export to PDF has been broken since the last update. | ✓ technical 95% | ✓ deny 53% | ✕ 1 → 2 46% |
| T-1024 | I got a login alert from a country I've never been to. | ✓ fraud 92% | ✓ deny 74% | ✓ 2 38% |
| T-1025 | My invoice says $261 but checkout showed $241. Why was I charged more? | ✓ billing 99% | ✕ deny → approve 70% | ✕ 2 → 1 57% |
| T-1026 | My package for order #821430 hasn't arrived and tracking hasn't moved in 10 days. | ✓ shipping 96% | ✕ deny → approve 70% | ✕ 1 → 0 68% |
| T-1027 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 90% | ✕ deny → approve 58% | ✕ 1 → 0 45% |
| T-1028 | The app crashes every time I open the settings page. | ✓ technical 96% | ✓ deny 71% | ✓ 3 42% |
| T-1029 | Someone changed the email on my account without my permission. | ✓ fraud 95% | ✓ deny 53% | ✓ 2 36% |
| T-1030 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing 98% | ✕ deny → approve 53% | ✕ 2 → 1 34% |
| T-1031 | The courier sent order #259717 to my old address instead of the one I entered. | ✓ shipping 94% | ✓ approve 51% | ✕ 2 → 1 32% |
| T-1032 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 88% | ✕ deny → approve 63% | ✓ 2 46% |
| T-1033 | Export to PDF has been broken since the last update. | ✓ technical 93% | ✓ deny 56% | ✕ 2 → 3 42% |
| T-1034 | There's an order #135622 on my account that I did not place. | ✓ fraud 92% | ✓ deny 60% | ✕ 3 → 4 40% |
| T-1035 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing 99% | ✓ approve 55% | ✕ 2 → 1 45% |
| T-1036 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping 97% | ✕ deny → approve 61% | ✕ 1 → 0 50% |
| T-1037 | You sent me the wrong colour. How do I return it? | ✓ returns 92% | ✓ deny 54% | ✓ 2 43% |
| T-1038 | I can't log in. It keeps saying "session expired" even after resetting my password. | ✓ technical 96% | ✓ deny 71% | ✕ 2 → 1 43% |
| T-1039 | There's an order #919845 on my account that I did not place. | ✓ fraud 95% | ✓ deny 57% | ✕ 1 → 0 43% |
| T-1040 | My invoice says $72 but checkout showed $52. Why was I charged more? | ✓ billing 99% | ✓ deny 51% | ✓ 2 45% |
| T-1041 | My package for order #643805 hasn't arrived and tracking hasn't moved in 9 days. | ✓ shipping 97% | ✓ deny 54% | ✕ 1 → 0 54% |
| T-1042 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 82% | ✓ deny 73% | ✕ 2 → 1 34% |
| T-1043 | Sync stops halfway through and shows error E-132. | ✓ technical 95% | ✓ deny 80% | ✕ 1 → 0 39% |
| T-1044 | I got a login alert from a country I've never been to. | ✓ fraud 92% | ✓ deny 60% | ✕ 2 → 1 35% |
| T-1045 | My invoice says $377 but checkout showed $357. Why was I charged more? | ✓ billing 99% | ✓ approve 53% | ✓ 2 44% |
| T-1046 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ fraud 87% | ✓ deny 59% | ✓ 3 38% |
| T-1047 | You sent me the wrong colour. How do I return it? | ✓ returns 95% | ✓ deny 57% | ✕ 2 → 1 41% |
| T-1048 | Export to PDF has been broken since the last update. | ✓ fraud 40% | ✓ deny 75% | ✕ 2 → 1 38% |
| T-1049 | There's an order #917499 on my account that I did not place. | ✓ fraud 91% | ✕ approve → deny 53% | ✓ 1 64% |
| T-1050 | I was charged twice for order #334980. Please fix the duplicate charge. | ✓ billing 95% | ✓ approve 57% | ✕ 3 → 4 35% |
| T-1051 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping 95% | ✕ deny → approve 64% | ✓ 3 48% |
| T-1052 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 89% | ✓ deny 63% | ✕ 3 → 2 42% |
| T-1053 | Sync stops halfway through and shows error E-508. | ✓ technical 94% | ✓ deny 56% | ✕ 1 → 0 49% |
| T-1054 | Someone changed the email on my account without my permission. | ✓ fraud 94% | ✓ deny 77% | ✕ 1 → 0 45% |
| T-1055 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing 99% | ✕ deny → approve 60% | ✓ 3 39% |
| T-1056 | The courier sent order #983082 to my old address instead of the one I entered. | ✓ shipping 90% | ✕ deny → approve 56% | ✕ 2 → 1 58% |
| T-1057 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns 86% | ✓ approve 68% | ✓ 2 50% |
| T-1058 | The app crashes every time I open the settings page. | ✓ technical 96% | ✓ deny 59% | ✕ 1 → 0 43% |
| T-1059 | There's an order #230179 on my account that I did not place. | ✓ fraud 94% | ✓ deny 53% | ✓ 2 37% |
Percentages are the model's own confidence. Every rule is on the Rulebook page.