Search

A GRC platform is document-heavy: findings, evidence and uploaded documents (PDFs/images/CSVs, in Hebrew and English), policies, questionnaires and responses, standards clauses, systems, processes, vendors, and audit logs all need to be searchable. Two hard constraints shape the design: multilingual full-text quality (Hebrew especially) and tenant isolation — a cross-tenant leak in a compliance product is catastrophic. This page locks the search approach and its graduation path.

Decision — ✅ Locked (staged)

Start Postgres-first, single tier. Use native full-text search (tsvector/GIN) + pg_trgm + unaccent for keyword search, and pgvector for the semantic/RAG layer, all inside each tenant's schema. Fuse keyword + vector with Reciprocal Rank Fusion (RRF) for both product search and the BNAI assistant. Graduate to Amazon OpenSearch Service (native Hebrew analyzer) only on documented triggers.

Why Postgres-first

• Fewest moving parts. PG 16 with pgvector and TimescaleDB is already the store; adding a search tier at launch is premature engineering (and premature Elasticsearch is a widely-cited SaaS regret).
• Isolation rides on the same mechanism as all tenant data. Search columns live inside each tenant schema; a query executes in the tenant's search_path, so there is no shared index and no runtime tenant filter to forget — strictly safer than any shared-index-plus-filter engine. For a compliance product, this alone is decisive.
• Hybrid keyword+vector is the 2025–2026 best practice and is available natively: run FTS and pgvector in parallel, fuse with RRF (1/(k+rank)). Reported lift ≈ 62% → 84% precision with near-perfect exact match — and the vector side captures Hebrew meaning even when the keyword side under-stems.
• Covers ~80–85% of search use cases without a second tier.

The Hebrew problem (handled per stage)

Postgres FTS Hebrew is genuinely weak — no bundled Hebrew stemmer/dictionary, so inflected/prefixed forms miss. This is real but manageable and does not by itself justify a second tier at launch:

Stage	Hebrew handling
Stage 0 — launch	pg_trgm n-gram / fuzzy matching (script-agnostic, gives usable Hebrew partial match without a stemmer) + unaccent, plus multilingual embeddings on the vector side for Hebrew semantics.
Stage 1 — weak lexical ranking	Add ParadeDB pg_search (BM25 inside Postgres) — still one tier, better relevance than ts_rank.
Stage 2 — graduate	AWS OpenSearch Service native Hebrew analyzer (GA April 2024) or HebMorph — a true morphological analyzer.

Comparison (why not the others, at launch)

Option	Hebrew / multilingual	Ops cost	Multi-tenant fit	Verdict
Native PG FTS + pg_trgm	Weak stemming, usable via n-grams	Lowest (in-DB)	Best — per-schema, automatic	✅ Launch
pgvector hybrid (FTS+vector+RRF)	Vector side language-agnostic	Medium, all in PG	Per-schema, automatic	✅ Launch (also RAG)
AWS OpenSearch	Strongest — native Hebrew analyzer + HebMorph	Medium-high (cluster, sync)	Index-per-tenant (silo) for parity	⏭ Stage 2 target
Elasticsearch (self-managed)	Good via HebMorph	Medium-high; licensing/ops friction	Same patterns	❌ No edge over AWS-native OpenSearch
Meilisearch	Claims He/Ar — vendor self-reported, unverified	Low-med, extra tier	Tenant tokens (shared index)	❌ Weaker isolation, unproven Hebrew
Typesense	Weak (Unicode-only tokenization)	Low-med, extra tier	Scoped API keys (shared index)	❌ Little gain
Algolia (SaaS)	Good	Lowest ops, highest $	Off-AWS	❌ GRC data leaves the VPC — non-starter
Dedicated vector DB (Pinecone/Qdrant/Weaviate)	n/a (semantic)	Extra tier	—	❌ Unjustified below ~5–10M vectors

Multi-tenant isolation

Keep search in each tenant's Postgres schema (tsvector + vector columns per schema). No shared index, no filter to forget. When Stage 2 arrives, mirror schema-per-tenant with index-per-tenant (silo) in OpenSearch — never a single pooled index with a runtime tenant filter for compliance data (one missing filter = cross-tenant leak).

RAG / semantic layer (BNAI)

• Vectors stay in Postgres (pgvector) — the right default under ~5M (comfortably ~10M) vectors; eliminates a separate tier and keeps embeddings per-schema (same automatic isolation). Only cross to a dedicated DB (Qdrant fastest at scale) beyond that.
• Hybrid FTS + vector with RRF is the retrieval pattern; cache fused scores in materialized views for hot endpoints.
• Ranking upgrade path: ParadeDB pg_search (BM25) or Timescale's pg_textsearch — both keep it single-tier.

Indexing pipeline (upload → index)

Reuses BullMQ + S3 — no new infra: S3 upload event → BullMQ job → text extraction → chunk → embed → write tsvector + vector into the tenant schema.

• Text extraction / OCR: born-digital PDFs extract directly; scanned images/PDFs need OCR. AWS Textract (native to the stack) outperforms Tesseract on noisy docs and RTL scripts.
• Hebrew OCR caveat: no neutral 2025 benchmark isolates Hebrew accuracy for Textract vs Tesseract — validate on a real Hebrew sample before committing, and consider a Hebrew-tuned OCR if accuracy is insufficient.

Graduation triggers (document-and-monitor)

Move to Stage 2 (OpenSearch) when any one fires:

1. Hebrew search-relevance complaints that pg_trgm + custom config + BM25 can't fix;
2. a tenant's searchable corpus > ~2M rows, or global > ~10M documents;
3. the product needs faceting / highlighting / advanced relevance tuning at scale;
4. hybrid-RRF p95 search latency breaches its SLO under load.

At Stage 2, Postgres stays the source of truth; OpenSearch is a projection fed by the same BullMQ pipeline, large tenants migrated first.

Field signal

PG-first hybrid (FTS + pgvector + RRF) is the dominant 2025–2026 pattern for teams already on Postgres building their first search/RAG — multiple independent write-ups converge on "pgvector is enough for most production RAG." AWS's investment in a native OpenSearch Hebrew analyzer (Apr 2024) marks the sanctioned AWS path for Hebrew search and the natural Stage-2 target. Multi-tenant SaaS at scale trends toward hybrid silo/pool indexing, but schema-per-tenant products get silo-grade isolation for free by keeping search in-schema.

Flagged conflicts: Meilisearch's Hebrew/Typesense comparisons are self-published (treat as marketing); the independently corroborated Hebrew options are HebMorph and the AWS OpenSearch analyzer. FTS→dedicated-engine thresholds vary by source (500K–2M rows vs 10M docs) — treat as order-of-magnitude. No neutral Hebrew-specific OCR benchmark exists — validate Textract on real Hebrew documents.