Decision-1
Microsoft · microsoft/microsoft-decision-1 · added 20 Oct 2026
Decision-1 by Microsoft ranks #2 of 12 AI decision models on the DecisionBench Rulebook benchmark (following written rules), with accuracy 100.0%. View on OpenRouter.
- Rulebook rank
- #2 of 12
- Accuracy
- 100.0%
- Team
- 100.0%
- Refund
- 100.0%
- Urgency
- 100.0%
- Confidence gap
- 0.076
60 tickets · 0.0% of mistakes made while at least 90% sure · $0.0260 per 1,000 tickets · 485 ms per ticket
Every ticket
The model's three decisions on each ticket, with its confidence. Wrong answers show the right one. No mistakes.
| Ticket | Customer wrote | Team | Refund | Urgency |
|---|---|---|---|---|
| T-1000 | My invoice says $425 but checkout showed $405. Why was I charged more? | ✓ billing100% | ✓ approve96% | ✓ 185% |
| T-1001 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping99% | ✓ deny99% | ✓ 185% |
| T-1002 | I changed my mind about order #468974 and want to return it. | ✓ returns100% | ✓ deny100% | ✓ 279% |
| T-1003 | Sync stops halfway through and shows error E-883. | ✓ technical100% | ✓ deny98% | ✓ 186% |
| T-1004 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny99% | ✓ 096% |
| T-1005 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing99% | ✓ deny80% | ✓ 180% |
| T-1006 | My package for order #822804 hasn't arrived and tracking hasn't moved in 7 days. | ✓ shipping100% | ✓ approve92% | ✓ 284% |
| T-1007 | I changed my mind about order #797563 and want to return it. | ✓ returns100% | ✓ approve96% | ✓ 182% |
| T-1008 | The app crashes every time I open the settings page. | ✓ technical100% | ✓ deny100% | ✓ 185% |
| T-1009 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny99% | ✓ 375% |
| T-1010 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing99% | ✓ approve87% | ✓ 190% |
| T-1011 | The courier sent order #441011 to my old address instead of the one I entered. | ✓ shipping100% | ✓ approve97% | ✓ 280% |
| T-1012 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ approve90% | ✓ 285% |
| T-1013 | Export to PDF has been broken since the last update. | ✓ fraud100% | ✓ deny100% | ✓ 180% |
| T-1014 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny97% | ✓ 186% |
| T-1015 | My invoice says $62 but checkout showed $42. Why was I charged more? | ✓ billing100% | ✓ deny99% | ✓ 277% |
| T-1016 | My package for order #455490 hasn't arrived and tracking hasn't moved in 4 days. | ✓ fraud100% | ✓ deny100% | ✓ 176% |
| T-1017 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns100% | ✓ approve71% | ✓ 185% |
| T-1018 | Export to PDF has been broken since the last update. | ✓ fraud100% | ✓ deny100% | ✓ 177% |
| T-1019 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny100% | ✓ 175% |
| T-1020 | I was charged twice for order #227551. Please fix the duplicate charge. | ✓ fraud100% | ✓ deny100% | ✓ 278% |
| T-1021 | My package for order #718743 hasn't arrived and tracking hasn't moved in 8 days. | ✓ shipping100% | ✓ approve92% | ✓ 383% |
| T-1022 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ deny98% | ✓ 471% |
| T-1023 | Export to PDF has been broken since the last update. | ✓ technical99% | ✓ deny99% | ✓ 278% |
| T-1024 | I got a login alert from a country I've never been to. | ✓ fraud97% | ✓ deny99% | ✓ 274% |
| T-1025 | My invoice says $261 but checkout showed $241. Why was I charged more? | ✓ billing100% | ✓ approve98% | ✓ 185% |
| T-1026 | My package for order #821430 hasn't arrived and tracking hasn't moved in 10 days. | ✓ shipping100% | ✓ approve91% | ✓ 094% |
| T-1027 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ approve82% | ✓ 096% |
| T-1028 | The app crashes every time I open the settings page. | ✓ technical100% | ✓ deny100% | ✓ 380% |
| T-1029 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny99% | ✓ 270% |
| T-1030 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing99% | ✓ approve85% | ✓ 184% |
| T-1031 | The courier sent order #259717 to my old address instead of the one I entered. | ✓ shipping100% | ✓ approve96% | ✓ 187% |
| T-1032 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns100% | ✓ approve92% | ✓ 279% |
| T-1033 | Export to PDF has been broken since the last update. | ✓ technical100% | ✓ deny100% | ✓ 374% |
| T-1034 | There's an order #135622 on my account that I did not place. | ✓ fraud100% | ✓ deny100% | ✓ 470% |
| T-1035 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing99% | ✓ approve92% | ✓ 183% |
| T-1036 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping100% | ✓ approve96% | ✓ 097% |
| T-1037 | You sent me the wrong colour. How do I return it? | ✓ returns99% | ✓ deny96% | ✓ 278% |
| T-1038 | I can't log in. It keeps saying "session expired" even after resetting my password. | ✓ technical100% | ✓ deny100% | ✓ 184% |
| T-1039 | There's an order #919845 on my account that I did not place. | ✓ fraud100% | ✓ deny97% | ✓ 094% |
| T-1040 | My invoice says $72 but checkout showed $52. Why was I charged more? | ✓ billing100% | ✓ deny99% | ✓ 277% |
| T-1041 | My package for order #643805 hasn't arrived and tracking hasn't moved in 9 days. | ✓ shipping100% | ✓ deny100% | ✓ 090% |
| T-1042 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ deny99% | ✓ 178% |
| T-1043 | Sync stops halfway through and shows error E-132. | ✓ technical100% | ✓ deny99% | ✓ 097% |
| T-1044 | I got a login alert from a country I've never been to. | ✓ fraud94% | ✓ deny100% | ✓ 176% |
| T-1045 | My invoice says $377 but checkout showed $357. Why was I charged more? | ✓ billing100% | ✓ approve97% | ✓ 277% |
| T-1046 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ fraud100% | ✓ deny100% | ✓ 376% |
| T-1047 | You sent me the wrong colour. How do I return it? | ✓ returns99% | ✓ deny99% | ✓ 189% |
| T-1048 | Export to PDF has been broken since the last update. | ✓ fraud100% | ✓ deny100% | ✓ 184% |
| T-1049 | There's an order #917499 on my account that I did not place. | ✓ fraud100% | ✓ deny99% | ✓ 180% |
| T-1050 | I was charged twice for order #334980. Please fix the duplicate charge. | ✓ billing100% | ✓ approve80% | ✓ 475% |
| T-1051 | Tracking says delivered but there is nothing at my door or with my neighbours. | ✓ shipping100% | ✓ approve87% | ✓ 384% |
| T-1052 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ deny99% | ✓ 279% |
| T-1053 | Sync stops halfway through and shows error E-508. | ✓ technical100% | ✓ deny99% | ✓ 094% |
| T-1054 | Someone changed the email on my account without my permission. | ✓ fraud100% | ✓ deny98% | ✓ 092% |
| T-1055 | My card was billed again after I cancelled. Can you look into this charge? | ✓ billing98% | ✓ approve87% | ✓ 379% |
| T-1056 | The courier sent order #983082 to my old address instead of the one I entered. | ✓ shipping100% | ✓ approve78% | ✓ 188% |
| T-1057 | I bought the wrong plan by mistake and want to undo the purchase. | ✓ returns99% | ✓ approve88% | ✓ 282% |
| T-1058 | The app crashes every time I open the settings page. | ✓ technical100% | ✓ deny100% | ✓ 091% |
| T-1059 | There's an order #230179 on my account that I did not place. | ✓ fraud100% | ✓ deny96% | ✓ 279% |
Percentages are the model's own confidence. Every rule is on the Rulebook page.