Functional requirementsNon-functional requirements (NFRs)Scale estimatesData storageAPI designSearch results endpointBackgroundBlueprint high-level designOffline query processing pipelineLog aggregationParallelized enrichmentQuery segmentation and classification serviceLinking and query expansion serviceRanking and indexing workerMaintaining freshnessMemory efficiencyReal-time search flowAPI gatewaySearch serviceGenAI service (LLM gateway)LLM providersProduct discovery flowOffline flowOnline flowData modellingProduct catalogproduct_catalogSmart search cachesmart_query cacheProduct search index (for keyword search and filtering)product_search_indexVector database (for semantic search)semantic_search_indexProduct discovery cacheproduct_discoveryAnalytic logs store (datalake)User profileuser_profileSystem design deep diveHow often should the offline pipeline run?Near-real-time pathHandling cachingCache tiers: hybrid strategy for speed and scaleL1 and L2 caches (K-V lookups for hot queries)Warm tier (semantic cache for the long tail)Cache warmingCache invalidationLatency and the P99 trade-offUtility vs.latencyOptimized vector search: ANNHNSW using graph-based partitioningInverted partitioningEfficient rankingOffline static ranking (global relevance)Online personalized re-rankingEnsuring robustness: availability, scalability, and resilienceScalabilityManaged shardingManaged throughputVector database scalingResilience and fault toleranceLoad balancingDatabase failoverTurbopuffer (serverless vector database)Circuit breakersMonitoringAlertingTesting and evaluationLLM as a judgeImplementationGolden set regression suiteA/B testingImplementationModel routing and fallback logicRouting strategyImplementationHuman evaluationHuman-in-the-loop for NRT trendsSummaryReferences