[
  {
    "objectID": "about.html",
    "href": "about.html",
    "title": "About",
    "section": "",
    "text": "AI Research @Redis explores and prototypes retrieval and language technologies that can be shipped into Redis products and used by developers.\nThe team’s work spans both research and applied systems. Recent work has covered semantic caching, embedding-based retrieval, hybrid search, reranking, model evaluation, local LLM inference, and agent-oriented workflows. Alongside research experiments, the team also develops the infrastructure, datasets, and evaluation practices needed to turn promising ideas into reliable product capabilities."
  },
  {
    "objectID": "about.html#main-themes",
    "href": "about.html#main-themes",
    "title": "About",
    "section": "Main themes",
    "text": "Main themes\n\nRetrieval, search, and semantic caching\nLLM systems and evaluation\nAgentic workflows and research AI infrastructure"
  },
  {
    "objectID": "about.html#links",
    "href": "about.html#links",
    "title": "About",
    "section": "Links",
    "text": "Links\n\nHugging Face: https://huggingface.co/redis"
  },
  {
    "objectID": "posts/hello.html",
    "href": "posts/hello.html",
    "title": "Hello",
    "section": "",
    "text": "Write here."
  },
  {
    "objectID": "posts/hello.html#links",
    "href": "posts/hello.html#links",
    "title": "Hello",
    "section": "Links",
    "text": "Links\n\nSpace HTML: this page\nHub page: https://huggingface.co/spaces/srijithrajamohan/ai-research-redis\nMedium: fill medium_url after publishing"
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "",
    "text": "The examples below illustrate a hybrid search retrieval of a query from a corpus of documents by combining semantic search using ModernBERT embeddings and the BM25 algorithm.\nThe scores from each are normalized between 0 and 1 in order to create a weighted average hybrid score. The raw embedding score is also shown in the columns for diagnostics. The ranks in each measure are then combined using RRF to create another score that can be used for retrieval.\nThe most correct match for each query is highlighted in each example."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#example-1",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#example-1",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "Example 1",
    "text": "Example 1\n\n\n\nHybrid retrieval example 1 (BM25 vs embedding vs hybrid)."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#example-2",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#example-2",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "Example 2",
    "text": "Example 2\n\n\n\nHybrid retrieval example 2 (BM25 vs embedding vs hybrid)."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#failure-mode-1---negationsdirectionality",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#failure-mode-1---negationsdirectionality",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "Failure mode 1 - Negations/directionality",
    "text": "Failure mode 1 - Negations/directionality\nBM25 is not keyword search; it up-weights rare terms and not just keywords.\nIn example 1, the key term of interest or the attribute/identifier is #124. However, the other terms in the query semantic, search, for, doc are all relatively rare and contribute to higher BM25 scores as evidenced in the first two records in the table index 8 and 11. These have negated intents however and therefore incorrect matches. BM25 has no ability to identify negations.\nThe similarity scores fortunately are not the highest but will most likely exceed most commonly used thresholds. Semantic search can fail to identify negated intent."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#failure-mode-2---attributesidentifiers",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#failure-mode-2---attributesidentifiers",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "Failure mode 2 - Attributes/Identifiers",
    "text": "Failure mode 2 - Attributes/Identifiers\n“The next three docs given by ids 5, 7 and 6 have the same phrasing but with the wrong identifier of #123, #1243 and #1233 respectively which lowers their BM25 score a bit but certainly higher than all the other documents. BM25 can detect a different attribute/identifier but the actual magnitude of this will depend on what the rest of the sentence looks like.”\n\nIf there is strong lexical overlap but only the identifier is different, there will be a small drop in the BM25 score.\nIf the rest of the sentence is NULL then the BM25 score is 0.\nIf there is very little overlap in the rest of the query and the doc.\n\nBM25 can identify attribute/identifier changes but is hard to put a threshold on this score.\nW.r.t embeddings two things are notable:\n\nThere are two forces at play here: the global semantic intent match and the identifier match.\nIn 8 and 11, there is negated intent whereas there is an identifier match.\nIn 5, 6 and 7 the general intent is correct but the identifier is incorrect.\nThe resulting score is a complex interaction of the two and the answer to “Can we identify negations or attribute changes with a threshold?” is heavily data-dependent.\n\nNote that the embedding scores for 5, 7 and 6 are some of the highest because attribute/identifier differences are something that embeddings struggle to detect."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#how-does-the-hybrid-score-and-rrf-fare",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#how-does-the-hybrid-score-and-rrf-fare",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "How does the hybrid score and RRF fare?",
    "text": "How does the hybrid score and RRF fare?\nAveraging out with a weighted hybrid score or RRF has no impact here. In example 1, record id 9 which is the closest match still does not rank highly in either of these scenarios.\nTo summarize the above for both approaches for the failure modes, it seems that embeddings seem to have the slight edge here.\n\n\n\nMethod\nNegation\nDirectionality\nAttribute/Identifier\n\n\n\n\nBM25\nN\nN\nN (hard to put a threshold)\n\n\nEmbedding\nSometimes\nN\nSometimes\n\n\n\nWe can’t eliminate all the issues, but we can hope to mitigate based on our understanding of the data."
  },
  {
    "objectID": "posts/hybrid-search-retrieval-bm25-modernbert.html#but-what-did-we-want-to-achieve-with-bm25",
    "href": "posts/hybrid-search-retrieval-bm25-modernbert.html#but-what-did-we-want-to-achieve-with-bm25",
    "title": "Hybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples",
    "section": "But what did we want to achieve with BM25?",
    "text": "But what did we want to achieve with BM25?\nWe are really looking to filter out matches that do not have the right attributes/identifiers we seek. BM25 looks like it could provide that, but does more than attribute matching.\n\nExtract identifiers from query.\nPerform embedding based retrieval of query + docs to retrieve set M.\nFilter out docs where identifiers do not match in set M.\nUnfortunately, this does nothing to address the negation/directionality limitation."
  },
  {
    "objectID": "posts/index.html",
    "href": "posts/index.html",
    "title": "Posts",
    "section": "",
    "text": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling\n\n\nA simpler yes/no classification setup improves determinism, scales better on larger batches, and preserves evaluation quality.\n\n\n\n\n\nMar 24, 2026\n\n\nRadoslav Ralev\n\n\n\n\n\n\n\nHybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples\n\n\nHow to combine ModernBERT embedding similarity with BM25, plus failure modes (negation and identifiers).\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\n\n\nEquations in Quarto (LaTeX Math)\n\n\nInline math, display math, aligned equations, and a few practical tips.\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\n\n\nHello\n\n\nFirst Quarto post (template).\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\nNo matching items"
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "",
    "text": "This post summarizes an internal evaluation of a revised local LLM-as-a-judge pipeline for cache-hit classification.\nThe main change is simple: instead of asking the model to generate a structured JSON object for each example, the pipeline now asks for a binary decision, yes or no, and compares the token probabilities p(yes) and p(no) at inference time.\nThat small interface change has a large systems impact. It removes fragile output parsing, reduces run failures, improves determinism across batch orderings, and makes larger-batch execution much more practical."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#why-the-earlier-approach-was-brittle",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#why-the-earlier-approach-was-brittle",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "Why the earlier approach was brittle",
    "text": "Why the earlier approach was brittle\nThe older implementation had three main weaknesses:\n\nIt relied on the model to emit valid JSON, so small formatting errors could break a run.\nIt showed non-deterministic behavior across batched inference, where identical sentence pairs could receive different labels depending on batch composition.\nIt was effectively zero-shot, which limited performance compared with a prompt design that better framed the classification task.\n\nIn practice, this meant that the evaluation pipeline was doing more work than necessary. The model was spending capacity on output formatting instead of the classification decision itself."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#the-new-classification-setup",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#the-new-classification-setup",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "The new classification setup",
    "text": "The new classification setup\nThe revised pipeline narrows the task to a direct binary choice:\n\nPredict yes or no\nCompare p(yes) against p(no)\nMark the pair as a cache hit when p(yes) > p(no)\n\nThis design makes the system easier to reason about and easier to scale. The surrounding code owns the output structure, while the model focuses on the decision boundary."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#what-improved",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#what-improved",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "What improved",
    "text": "What improved\nThe updated pipeline delivers four practical gains:\n\nImproved stability through fewer failed runs\nBetter batch-size scaling on L40S GPUs\nDeterministic outputs for repeated sentence pairs\nLower latency from a simpler forward-pass pattern\n\nThese are not just implementation conveniences. They matter directly for evaluation throughput, reproducibility, and confidence in benchmark results."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#prompt-comparison-and-implementation-behavior",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#prompt-comparison-and-implementation-behavior",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "Prompt comparison and implementation behavior",
    "text": "Prompt comparison and implementation behavior\nThe writeup compares the old and new setups across different batch sizes and prompt choices.\nOne key takeaway is that the new implementation produces more stable metrics with less variance. The goal is not to claim a dramatic quality jump from prompt engineering alone, but to show that the revised setup behaves more consistently under scale.\nThe experiments also compare Hugging Face execution with vLLM. Metric quality stays comparable, while vLLM provides a meaningful speed advantage in most runs."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#sub-10b-model-benchmark-on-quora-question-pairs",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#sub-10b-model-benchmark-on-quora-question-pairs",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "Sub-10B model benchmark on Quora Question Pairs",
    "text": "Sub-10B model benchmark on Quora Question Pairs\nThe second part of the evaluation benchmarks local models under 10B parameters on Quora Question Pairs.\nTwo experimental choices make the results more trustworthy:\n\nThe evaluation sample size increases from 1024 to 4096\nEach model is run 5 times on separate 4096-sample batches\n\nThat setup makes it possible to report means and standard deviations for precision, recall, F1, and runtime, rather than relying on a single noisy run."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#benchmark-observations",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#benchmark-observations",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "Benchmark observations",
    "text": "Benchmark observations\nThe main observations from the benchmark are:\n\nPrecision, recall, and F1 remain stable across repeated runs\nTop-performing small models show low variance, suggesting the results are not driven by one favorable sample\nvLLM is often about 2x faster than the Hugging Face path, though some smaller models can occasionally be faster with HF because of overhead effects\n\nThe overall message is that local LLM judging can be both practical and reproducible when the task formulation is kept narrow and the serving path is optimized."
  },
  {
    "objectID": "posts/alternative-llm-as-a-judge-local-pipeline.html#takeaway",
    "href": "posts/alternative-llm-as-a-judge-local-pipeline.html#takeaway",
    "title": "An Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling",
    "section": "Takeaway",
    "text": "Takeaway\nThe strongest result here is not a single benchmark number. It is the systems lesson.\nWhen an evaluation pipeline asks a model to do only the minimum necessary work, the entire stack becomes easier to scale and more reliable. In this case, replacing structured generation with probability-based binary classification improves stability, preserves evaluation quality, and makes local LLM judging a stronger option for large batch workloads."
  },
  {
    "objectID": "posts/latex-equations.html",
    "href": "posts/latex-equations.html",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "",
    "text": "This post is a quick reference for writing LaTeX math in Quarto."
  },
  {
    "objectID": "posts/latex-equations.html#inline-math",
    "href": "posts/latex-equations.html#inline-math",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "Inline math",
    "text": "Inline math\nUse single dollar signs for inline equations: \\(E = mc^2\\).\nAnother example: the normalized score is \\(\\hat{s} = \\frac{s - \\min(s)}{\\max(s) - \\min(s)}\\)."
  },
  {
    "objectID": "posts/latex-equations.html#display-math",
    "href": "posts/latex-equations.html#display-math",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "Display math",
    "text": "Display math\nUse double dollar signs for centered display equations:\n\\[\n\\mathrm{score}(q, d) = \\alpha \\cdot \\mathrm{BM25}(q,d) + (1-\\alpha)\\cdot \\mathrm{sim}(q,d)\n\\]"
  },
  {
    "objectID": "posts/latex-equations.html#multi-line-and-aligned-equations",
    "href": "posts/latex-equations.html#multi-line-and-aligned-equations",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "Multi-line and aligned equations",
    "text": "Multi-line and aligned equations\nUse an aligned block inside display math:\n\\[\n\\begin{aligned}\n\\mathrm{RRF}(d) &= \\sum_{m \\in M} \\frac{1}{k + \\mathrm{rank}_m(d)} \\\\\n\\hat{s}(d) &= \\frac{s(d) - \\min(s)}{\\max(s) - \\min(s)}\n\\end{aligned}\n\\]"
  },
  {
    "objectID": "posts/latex-equations.html#common-symbols",
    "href": "posts/latex-equations.html#common-symbols",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "Common symbols",
    "text": "Common symbols\n\nGreek letters: \\(\\alpha, \\beta, \\gamma, \\lambda\\)\nVectors: \\(\\mathbf{v}\\), norms: \\(\\lVert \\mathbf{v} \\rVert_2\\)\nSets: \\(\\{x \\mid x > 0\\}\\)"
  },
  {
    "objectID": "posts/latex-equations.html#tips",
    "href": "posts/latex-equations.html#tips",
    "title": "Equations in Quarto (LaTeX Math)",
    "section": "Tips",
    "text": "Tips\n\nInline: $...$ and display: $$...$$\nUse \\\\ for line breaks inside aligned\nIf math ever renders as raw text, confirm you are rendering HTML and that MathJax/KaTeX is enabled."
  },
  {
    "objectID": "index.html",
    "href": "index.html",
    "title": "AI Research @Redis",
    "section": "",
    "text": "Homepage of the AI Research team at Redis.\n\n\n\n\n\n\n\n\n\nAn Alternative LLM-as-a-Judge Local Pipeline for Better Stability and Batch Scaling\n\n\nA simpler yes/no classification setup improves determinism, scales better on larger batches, and preserves evaluation quality.\n\n\n\n\n\nMar 24, 2026\n\n\nRadoslav Ralev\n\n\n\n\n\n\n\nHybrid Search Retrieval: Combining Semantic Search and BM25 Algorithm with Examples\n\n\nHow to combine ModernBERT embedding similarity with BM25, plus failure modes (negation and identifiers).\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\n\n\nEquations in Quarto (LaTeX Math)\n\n\nInline math, display math, aligned equations, and a few practical tips.\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\n\n\nHello\n\n\nFirst Quarto post (template).\n\n\n\n\n\nMar 18, 2026\n\n\nRedis AI Research\n\n\n\n\n\nNo matching items"
  }
]