Asad999 commited on
Commit
e97fb77
·
1 Parent(s): cccfee0

Improve hallucination detection: allow intelligent counting and derived facts

Browse files
Files changed (1) hide show
  1. graph.py +7 -3
graph.py CHANGED
@@ -360,15 +360,19 @@ def check_hallucination(state: AgentState) -> dict:
360
  else:
361
  strictness_instruction = """
362
  This response was generated from OFFICIAL UNIVERSITY DOCUMENTS.
363
- Apply STRICT checking:
364
- - Flag any specific fact (number, name, policy, fee, grade, course code)
365
- that is NOT present in the context.
 
 
366
  - Do NOT flag the university name "XYZ National University" as a hallucination.
367
  - Do NOT flag polite conversational phrases or general framing.
368
  - Do NOT flag minor rephrasing or summarization of document content."""
369
 
370
  prompt = f"""You are a hallucination detector for a university advisory AI.
371
 
 
 
372
  {strictness_instruction}
373
 
374
  Context:
 
360
  else:
361
  strictness_instruction = """
362
  This response was generated from OFFICIAL UNIVERSITY DOCUMENTS.
363
+ Apply STRICT but INTELLIGENT checking:
364
+ - Flag any specific entity (name, fee, course code, grade) that is NOT present in the context.
365
+ - For NUMBERS and COUNTS:
366
+ a) If a number (like a fee or credit hour) is explicitly stated in the response, verify it matches the context.
367
+ b) If a total count is provided (e.g., 'There are 8 courses'), do NOT flag it if the items being counted are all present and verified in the context.
368
  - Do NOT flag the university name "XYZ National University" as a hallucination.
369
  - Do NOT flag polite conversational phrases or general framing.
370
  - Do NOT flag minor rephrasing or summarization of document content."""
371
 
372
  prompt = f"""You are a hallucination detector for a university advisory AI.
373
 
374
+ Student Question: "{state['query']}"
375
+
376
  {strictness_instruction}
377
 
378
  Context: