classification

5bb0f366deed4974a8864ec6c33e224c

CPU

Intel(R) Xeon(R) Processor

family 6 · model 207 · stepping 2 · 4 logical CPUs

Memory

15.63 GiB

3.45 GiB available when this file was written

Server

Ollama 0.40.5

llamacpp · libggml-cpu-sapphirerapids.so

Recorded

2026-10-16 05:09:30.501 UTC

Linux 6.12.94+ x86_64

qwen2.5:0.5b

Q4_K_M · n 8 · exact 4/8 · schema 8/8

double-chargeexact 0.000 · schema 1.000 · judge not run

I was charged twice on my card for the same order.

1324.7 ms · 61 in / 10 out · $0

Field	Expected	Predicted	Match
label	billing	account	differs

  {
-   "label": "billing"
+   "label": "account"
  }

refund-statusexact 0.000 · schema 1.000 · judge not run

Where is the refund you said was sent last week?

177.8 ms · 60 in / 10 out · $0

Field	Expected	Predicted	Match
label	billing	account	differs

  {
-   "label": "billing"
+   "label": "account"
  }

not-deliveredexact 0.000 · schema 1.000 · judge not run

The tracking page says delivered but the package is not here.

178.7 ms · 61 in / 10 out · $0

Field	Expected	Predicted	Match
label	shipping	product	differs

  {
-   "label": "shipping"
+   "label": "product"
  }

late-shipexact 0.000 · schema 1.000 · judge not run

My order still has not shipped and the estimate was yesterday.

172.2 ms · 61 in / 10 out · $0

Field	Expected	Predicted	Match
label	shipping	account	differs

  {
-   "label": "shipping"
+   "label": "account"
  }

broken-unitexact 1.000 · schema 1.000 · judge not run

The blender I received does not turn on.

165.8 ms · 58 in / 10 out · $0

Field	Expected	Predicted	Match
label	product	product	match

  {
    "label": "product"
  }

wrong-colorexact 1.000 · schema 1.000 · judge not run

You sent a blue jacket and I ordered the black one.

167.8 ms · 61 in / 10 out · $0

Field	Expected	Predicted	Match
label	product	product	match

  {
    "label": "product"
  }

cannot-loginexact 1.000 · schema 1.000 · judge not run

I cannot sign in after the password reset email.

156.4 ms · 59 in / 10 out · $0

Field	Expected	Predicted	Match
label	account	account	match

  {
    "label": "account"
  }

close-accountexact 1.000 · schema 1.000 · judge not run

Please close my account and delete the profile.

152.8 ms · 58 in / 10 out · $0

Field	Expected	Predicted	Match
label	account	account	match

  {
    "label": "account"
  }

smollm2:360m

F16 · n 8 · exact 0/8 · schema 0/8

double-chargeexact 0.000 · schema 0.000 · judge not run

I was charged twice on my card for the same order.

1903.9 ms · 71 in / 41 out · $0

Field	Expected	Predicted	Match
label	billing	billing	match
status	null	pending	differs
message	null	You were charged twice on your card for the same order. Please try again later.	differs

status: Extra inputs are not permitted
message: Extra inputs are not permitted

  {
-   "label": "billing"
+   "label": "billing",
+   "status": "pending",
+   "message": "You were charged twice on your card for the same order. Please try again later."
  }

refund-statusexact 0.000 · schema 0.000 · judge not run

Where is the refund you said was sent last week?

859.4 ms · 70 in / 43 out · $0

Field	Expected	Predicted	Match
label	billing	billing	match
status	null	pending	differs
message	null	Refund request for refund was not sent last week. Please check your inbox for the confirmation.	differs

status: Extra inputs are not permitted
message: Extra inputs are not permitted

  {
-   "label": "billing"
+   "label": "billing",
+   "status": "pending",
+   "message": "Refund request for refund was not sent last week. Please check your inbox for the confirmation."
  }

not-deliveredexact 0.000 · schema 0.000 · judge not run

The tracking page says delivered but the package is not here.

603.7 ms · 71 in / 29 out · $0

Field	Expected	Predicted	Match
label	shipping	billing	differs
message	null	The tracking page says delivered but the package is not here.	differs

message: Extra inputs are not permitted

  {
-   "label": "shipping"
+   "label": "billing",
+   "message": "The tracking page says delivered but the package is not here."
  }

late-shipexact 0.000 · schema 0.000 · judge not run

My order still has not shipped and the estimate was yesterday.

1348.8 ms · 71 in / 65 out · $0

Field	Expected	Predicted	Match
label	shipping	billing	differs
status	null	pending	differs
message	null	My order still has not shipped and the estimate was yesterday.	differs
status_description	null	Pending	differs
status_reason	null	Pending	differs
status_type	null	pending	differs

status: Extra inputs are not permitted
message: Extra inputs are not permitted
status_description: Extra inputs are not permitted
status_reason: Extra inputs are not permitted
status_type: Extra inputs are not permitted

  {
-   "label": "shipping"
+   "label": "billing",
+   "status": "pending",
+   "message": "My order still has not shipped and the estimate was yesterday.",
+   "status_description": "Pending",
+   "status_reason": "Pending",
+   "status_type": "pending"
  }

broken-unitexact 0.000 · schema 0.000 · judge not run

The blender I received does not turn on.

677.1 ms · 68 in / 33 out · $0

Field	Expected	Predicted	Match
label	product	billing	differs
status	null	failed	differs
message	null	The blender I received does not turn on.	differs

status: Extra inputs are not permitted
message: Extra inputs are not permitted

  {
-   "label": "product"
+   "label": "billing",
+   "status": "failed",
+   "message": "The blender I received does not turn on."
  }

wrong-colorexact 0.000 · schema 0.000 · judge not run

You sent a blue jacket and I ordered the black one.

388.9 ms · 71 in / 17 out · $0

Field	Expected	Predicted	Match
label	product	—	no JSON

output is not valid JSON

- {
-   "label": "product"
- }
+ Ticket:
+ You sent a blue jacket and I ordered the black one.

cannot-loginexact 0.000 · schema 0.000 · judge not run

I cannot sign in after the password reset email.

677.0 ms · 69 in / 33 out · $0

Field	Expected	Predicted	Match
label	account	billing	differs
status	null	failed	differs
message	null	Failed to sign in after password reset email	differs

status: Extra inputs are not permitted
message: Extra inputs are not permitted

  {
-   "label": "account"
+   "label": "billing",
+   "status": "failed",
+   "message": "Failed to sign in after password reset email"
  }

close-accountexact 0.000 · schema 0.000 · judge not run

Please close my account and delete the profile.

540.7 ms · 68 in / 26 out · $0

Field	Expected	Predicted	Match
label	account	billing	differs
message	null	Please close your account and delete your profile.	differs

message: Extra inputs are not permitted

  {
-   "label": "account"
+   "label": "billing",
+   "message": "Please close your account and delete your profile."
  }

llama3.2:1b

Q8_0 · n 8 · exact 2/8 · schema 4/8

double-chargeexact 1.000 · schema 1.000 · judge not run

I was charged twice on my card for the same order.

2786.4 ms · 73 in / 10 out · $0

Field	Expected	Predicted	Match
label	billing	billing	match

  {
    "label": "billing"
  }

refund-statusexact 1.000 · schema 1.000 · judge not run

Where is the refund you said was sent last week?

275.0 ms · 72 in / 10 out · $0

Field	Expected	Predicted	Match
label	billing	billing	match

  {
    "label": "billing"
  }

not-deliveredexact 0.000 · schema 1.000 · judge not run

The tracking page says delivered but the package is not here.

294.1 ms · 73 in / 10 out · $0

Field	Expected	Predicted	Match
label	shipping	billing	differs

  {
-   "label": "shipping"
+   "label": "billing"
  }

late-shipexact 0.000 · schema 0.000 · judge not run

My order still has not shipped and the estimate was yesterday.

519.9 ms · 73 in / 20 out · $0

Field	Expected	Predicted	Match
label	shipping	My order still has not shipped and the estimate was yesterday.	differs

label: Input should be 'billing', 'shipping', 'product' or 'account'

  {
-   "label": "shipping"
+   "label": "My order still has not shipped and the estimate was yesterday."
  }

broken-unitexact 0.000 · schema 0.000 · judge not run

The blender I received does not turn on.

435.1 ms · 70 in / 17 out · $0

Field	Expected	Predicted	Match
label	product	The blender I received does not turn on.	differs

label: Input should be 'billing', 'shipping', 'product' or 'account'

  {
-   "label": "product"
+   "label": "The blender I received does not turn on."
  }

wrong-colorexact 0.000 · schema 0.000 · judge not run

You sent a blue jacket and I ordered the black one.

511.1 ms · 73 in / 20 out · $0

Field	Expected	Predicted	Match
label	product	You sent a blue jacket and I ordered the black one.	differs

label: Input should be 'billing', 'shipping', 'product' or 'account'

  {
-   "label": "product"
+   "label": "You sent a blue jacket and I ordered the black one."
  }

cannot-loginexact 0.000 · schema 0.000 · judge not run

I cannot sign in after the password reset email.

429.6 ms · 71 in / 17 out · $0

Field	Expected	Predicted	Match
label	account	I cannot sign in after password reset email	differs

label: Input should be 'billing', 'shipping', 'product' or 'account'

  {
-   "label": "account"
+   "label": "I cannot sign in after password reset email"
  }

close-accountexact 0.000 · schema 1.000 · judge not run

Please close my account and delete the profile.

269.0 ms · 70 in / 10 out · $0

Field	Expected	Predicted	Match
label	account	billing	differs

  {
-   "label": "account"
+   "label": "billing"
  }