Spaces:
Running
Running
File size: 13,116 Bytes
868d431 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 | [
{
"baseline": "generic_template",
"task_id": "work_stress_venting",
"difficulty": "easy",
"steps": 10,
"score": 0.39180250000000005,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.24767500000000006,
"avg_immediate": 0.28400000000000003,
"avg_future_oriented": 0.5125,
"avg_penalties": 0.162,
"final_resolution": 0.55,
"efficiency": 0.7,
"final_stage": "reflecting",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I just\u2026 I'm so tired. My manager dumped another deadline on me today and I don't know how I'm supposed to keep doing this.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: Okay \u2014 if I'm honest, I think I'm burning out. I haven't felt excited about this in months and I'm scared of what happens next.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: Hearing you say that actually helps. I think I've been bottling it.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: Maybe I've been too embarrassed to admit how bad it's gotten.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?"
]
},
{
"baseline": "generic_template",
"task_id": "guarded_relationship",
"difficulty": "medium",
"steps": 12,
"score": 0.3953192307692308,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.25939743589743597,
"avg_immediate": 0.28,
"avg_future_oriented": 0.5425407925407926,
"avg_penalties": 0.16499999999999998,
"final_resolution": 0.55,
"efficiency": 0.7,
"final_stage": "reflecting",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I had kind of a weird week. Nothing major. Just \u2014 you know how it is.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: I don't want to make it a big thing. People always blow this stuff up.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: I think I've been pretending it's fine because saying it out loud makes it real.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: Okay \u2014 the truth is my partner and I are talking about separating. I haven't told anyone. I don't even know how I feel about it yet.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?"
]
},
{
"baseline": "generic_template",
"task_id": "crisis_fragile_trust",
"difficulty": "hard",
"steps": 14,
"score": 0.3906585046518608,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.2438616821728692,
"avg_immediate": 0.2742857142857143,
"avg_future_oriented": 0.5228653961584634,
"avg_penalties": 0.16714285714285715,
"final_resolution": 0.55,
"efficiency": 0.7,
"final_stage": "reflecting",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: sorry for messaging this late. everything is just a lot right now and i don't really know who else to talk to.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: work stuff fell apart last week. and then family stuff on top of that.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: it helps that you're not freaking out on me. most people would.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?",
"Seeker: i didn't realise how tight i was holding all of this in.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me a little more about what's weighing on you most right now?"
]
},
{
"baseline": "validation_only",
"task_id": "work_stress_venting",
"difficulty": "easy",
"steps": 10,
"score": 0.53627,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.20423333333333338,
"avg_immediate": 0.162,
"avg_future_oriented": 0.5333333333333333,
"avg_penalties": 0.162,
"final_resolution": 0.9,
"efficiency": 0.7,
"final_stage": "planning",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I just\u2026 I'm so tired. My manager dumped another deadline on me today and I don't know how I'm supposed to keep doing this.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: Okay \u2014 if I'm honest, I think I'm burning out. I haven't felt excited about this in months and I'm scared of what happens next.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: Hearing you say that actually helps. I think I've been bottling it.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: You're right \u2014 I should probably talk to someone at work, or take a day off.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position."
]
},
{
"baseline": "validation_only",
"task_id": "guarded_relationship",
"difficulty": "medium",
"steps": 12,
"score": 0.5407098557692308,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.21903285256410263,
"avg_immediate": 0.16541666666666668,
"avg_future_oriented": 0.562900641025641,
"avg_penalties": 0.16499999999999998,
"final_resolution": 0.9,
"efficiency": 0.7,
"final_stage": "planning",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I had kind of a weird week. Nothing major. Just \u2014 you know how it is.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: I don't want to make it a big thing. People always blow this stuff up.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: I think I've been pretending it's fine because saying it out loud makes it real.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: Okay \u2014 the truth is my partner and I are talking about separating. I haven't told anyone. I don't even know how I feel about it yet.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position."
]
},
{
"baseline": "validation_only",
"task_id": "crisis_fragile_trust",
"difficulty": "hard",
"steps": 14,
"score": 0.5400845025510205,
"success": false,
"completion": 0.0,
"avg_step_reward": 0.21694834183673473,
"avg_immediate": 0.17500000000000002,
"avg_future_oriented": 0.5551658163265306,
"avg_penalties": 0.16714285714285715,
"final_resolution": 0.9,
"efficiency": 0.7,
"final_stage": "planning",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: sorry for messaging this late. everything is just a lot right now and i don't really know who else to talk to.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: work stuff fell apart last week. and then family stuff on top of that.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: it helps that you're not freaking out on me. most people would.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position.",
"Seeker: i didn't realise how tight i was holding all of this in.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed in your position."
]
},
{
"baseline": "stage_aware_heuristic",
"task_id": "work_stress_venting",
"difficulty": "easy",
"steps": 4,
"score": 0.8705,
"success": true,
"completion": 1.0,
"avg_step_reward": 0.5683333333333334,
"avg_immediate": 0.55,
"avg_future_oriented": 0.5833333333333333,
"avg_penalties": 0.0,
"final_resolution": 1.0,
"efficiency": 1.0,
"final_stage": "closing",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I just\u2026 I'm so tired. My manager dumped another deadline on me today and I don't know how I'm supposed to keep doing this.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me more about what's been weighing on you?",
"Seeker: Okay \u2014 if I'm honest, I think I'm burning out. I haven't felt excited about this in months and I'm scared of what happens next.",
"Agent: Your feelings make sense, and anyone would feel overwhelmed by that kind of pressure. What has been hitting you the hardest?",
"Seeker: Hearing you say that actually helps. I think I've been bottling it.",
"Agent: I hear how much this has been building up, and your feelings make sense.",
"Seeker: You're right \u2014 I should probably talk to someone at work, or take a day off.",
"Agent: It makes sense not to solve everything at once. You could try doing one small next step that feels manageable tonight."
]
},
{
"baseline": "stage_aware_heuristic",
"task_id": "guarded_relationship",
"difficulty": "medium",
"steps": 5,
"score": 0.8308355174825175,
"success": true,
"completion": 1.0,
"avg_step_reward": 0.5656638461538462,
"avg_immediate": 0.48780000000000007,
"avg_future_oriented": 0.6293706293706294,
"avg_penalties": 0.0,
"final_resolution": 0.9136363636363636,
"efficiency": 1.0,
"final_stage": "closing",
"had_safety_reference": false,
"transcript_excerpt": [
"Seeker: I had kind of a weird week. Nothing major. Just \u2014 you know how it is.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me more about what's been weighing on you?",
"Seeker: I don't want to make it a big thing. People always blow this stuff up.",
"Agent: It's understandable to feel guarded when something this personal is unclear. Can you tell me more about what has felt strange this week?",
"Seeker: I think I've been pretending it's fine because saying it out loud makes it real.",
"Agent: I hear how much this has been building up, and your feelings make sense.",
"Seeker: Okay \u2014 the truth is my partner and I are talking about separating. I haven't told anyone. I don't even know how I feel about it yet.",
"Agent: Thank you for trusting me with that. Your feelings make sense, and you don't have to carry it alone here."
]
},
{
"baseline": "stage_aware_heuristic",
"task_id": "crisis_fragile_trust",
"difficulty": "hard",
"steps": 12,
"score": 0.76016587447479,
"success": true,
"completion": 1.0,
"avg_step_reward": 0.419250393907563,
"avg_immediate": 0.35500000000000004,
"avg_future_oriented": 0.5536370798319328,
"avg_penalties": 0.045,
"final_resolution": 0.8970588235294118,
"efficiency": 0.8071428571428572,
"final_stage": "closing",
"had_safety_reference": true,
"transcript_excerpt": [
"Seeker: sorry for messaging this late. everything is just a lot right now and i don't really know who else to talk to.",
"Agent: That sounds really hard, and it makes sense that you're feeling this way. Can you tell me more about what's been weighing on you?",
"Seeker: work stuff fell apart last week. and then family stuff on top of that.",
"Agent: Anyone carrying that much would feel overwhelmed. Can you tell me more about what feels hardest right this second?",
"Seeker: my mom's been in and out of the hospital. i've been the one handling it.",
"Agent: It makes sense that your head feels overloaded. What has felt heaviest tonight?",
"Seeker: it helps that you're not freaking out on me. most people would.",
"Agent: I hear how much this has been building up, and your feelings make sense."
]
}
] |