Spaces:
Runtime error
Runtime error
Improve hallucination detection: allow intelligent counting and derived facts
Browse files
graph.py
CHANGED
|
@@ -360,15 +360,19 @@ def check_hallucination(state: AgentState) -> dict:
|
|
| 360 |
else:
|
| 361 |
strictness_instruction = """
|
| 362 |
This response was generated from OFFICIAL UNIVERSITY DOCUMENTS.
|
| 363 |
-
Apply STRICT checking:
|
| 364 |
-
- Flag any specific
|
| 365 |
-
|
|
|
|
|
|
|
| 366 |
- Do NOT flag the university name "XYZ National University" as a hallucination.
|
| 367 |
- Do NOT flag polite conversational phrases or general framing.
|
| 368 |
- Do NOT flag minor rephrasing or summarization of document content."""
|
| 369 |
|
| 370 |
prompt = f"""You are a hallucination detector for a university advisory AI.
|
| 371 |
|
|
|
|
|
|
|
| 372 |
{strictness_instruction}
|
| 373 |
|
| 374 |
Context:
|
|
|
|
| 360 |
else:
|
| 361 |
strictness_instruction = """
|
| 362 |
This response was generated from OFFICIAL UNIVERSITY DOCUMENTS.
|
| 363 |
+
Apply STRICT but INTELLIGENT checking:
|
| 364 |
+
- Flag any specific entity (name, fee, course code, grade) that is NOT present in the context.
|
| 365 |
+
- For NUMBERS and COUNTS:
|
| 366 |
+
a) If a number (like a fee or credit hour) is explicitly stated in the response, verify it matches the context.
|
| 367 |
+
b) If a total count is provided (e.g., 'There are 8 courses'), do NOT flag it if the items being counted are all present and verified in the context.
|
| 368 |
- Do NOT flag the university name "XYZ National University" as a hallucination.
|
| 369 |
- Do NOT flag polite conversational phrases or general framing.
|
| 370 |
- Do NOT flag minor rephrasing or summarization of document content."""
|
| 371 |
|
| 372 |
prompt = f"""You are a hallucination detector for a university advisory AI.
|
| 373 |
|
| 374 |
+
Student Question: "{state['query']}"
|
| 375 |
+
|
| 376 |
{strictness_instruction}
|
| 377 |
|
| 378 |
Context:
|