Service Overview & API Reference
| Module | Artifact | Default port | Responsibilities |
|---|---|---|---|
| base | sandi-base.jar | — | Shared models, services, Solr client, utilities — library only |
| search | sandi.war | 8081 | Search API: hybrid search, spell check, query expansion, reranking, RAG answers |
| index | sandi.war | 8082 | Index API, Delete API, Schedule API, Admin API; document parsing and embedding pipeline |
| Service | Default port | Technology | Used by |
|---|---|---|---|
| Embedding (emb) | 8085 | Qwen3 Embeddings / OpenAI | Index: text→vector; Search: query→vector |
| NLP (nlp) | 8086 | SpaCy | Search: entity extraction, POS, lemmatization |
| LLM (llm) | 8087 | Qwen3-4B / OpenAI | Search: spell check, query expansion, RAG answers |
| Reranker (rer) | 8088 | Qwen3-Reranker / OpenAI | Search: reorder results by cross-encoder relevance |
| Client search hook (cls) | configurable | Custom Python | Search: optional per-client pre/post processing |
| Client index hook (cli) | configurable | Custom Python | Index: optional per-client document transformation |
| Field | Type | Description |
|---|---|---|
requestId | String | Caller-supplied correlation ID. Returned in every response unchanged. Used in all server-side log entries to trace a request end-to-end. |
clientId | String | Identifies the tenant. Must match an existing, active client record. Determines which Solr collection, field mappings, synonyms, and AI service endpoints are used. |
| Field | Value | Meaning |
|---|---|---|
status | "SUCCESS" | Request completed without errors |
status | "ERROR" | Request failed — see message for detail |
status | "MISSING" | A required field was absent; requestId in the response is "MISSING" |
status | "SCHEDULED" | Job accepted by the scheduler (Schedule API only) |
status | "CANCELLED" | Job cancelled successfully (Schedule API only) |
| Code | When returned |
|---|---|
200 OK | Request succeeded |
400 Bad Request | Missing required field, invalid parameter value, or inactive/unknown client |
404 Not Found | Addressed resource (client, collection, job) does not exist |
500 Internal Server Error | Unexpected server-side failure; details in message |
The Search API exposes one logical operation — execute a search — available as both
POST (JSON body) and GET (query parameters).
Accepts a JSON body. Preferred for complex queries with many optional flags.
Same parameters as POST but passed as query string. Suitable for simple queries and browser testing.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
requestId | String | Yes | — | Correlation ID |
clientId | String | Yes | — | Tenant identifier |
searchQuery | String | No | — | Natural-language query text. Processed by NLP and embedding services when AI features are enabled. |
filterQuery | String | No | — | Solr filter query (fq). Applied after main search to narrow results without affecting relevance scores. |
query | String | No | — | Raw Solr query string. Bypasses NLP processing. Use when you need exact Solr syntax control. |
pageNumber | Integer | No | 1 | 1-based page number |
pageSize | Integer | No | 10 | Results per page. Maximum 10 000. |
resultFields | String | No | all fields | Comma-separated list of fields to return in each result document |
sortFields | String | No | score desc | Solr sort expression, e.g. date desc,score desc |
groupFields | String | No | — | Comma-separated fields to group results by (Solr field collapsing) |
facetFields | String | No | — | Comma-separated fields to compute facet counts for |
highlightFields | String | No | — | Comma-separated fields to apply snippet highlighting to |
highlightTags | String | No | <em>,</em> | Open/close highlight tags separated by comma |
searchClients | String | No | — | Comma-separated additional client IDs whose collections are also searched (cross-client search) |
searchCollections | String | No | client collection | Override the Solr collections to search |
precision | String | No | "medium" | Search precision level: high, medium, or low. Affects vector score thresholds and query strategy. |
exact | Boolean | No | false | When true, only exact phrase matches are returned; vector/semantic path is skipped |
legacy | Boolean | No | false | Force BM25-only keyword search; disables all vector search |
synonyms | Boolean | No | false | Expand the query using synonym sets configured on the client |
dym | Boolean | No | false | "Did You Mean" — send query to LLM for spelling correction; returns dymQuery in response |
expand | Boolean | No | false | Query expansion — LLM generates alternative phrasings and adds them to the search |
rerank | Boolean | No | false | Pass the result page through the reranking service to reorder by cross-encoder relevance |
rag | Boolean | No | false | Generate a natural-language answer from the top results using the LLM; returned in ragAnswer |
collapse | Boolean | No | false | Collapse duplicate documents (POST only) |
debug | Boolean | No | false | Include internal score and query detail in the debug response field (POST only) |
| Field | Type | Description |
|---|---|---|
requestId | String | Echoed from the request |
status | String | SUCCESS or ERROR |
message | String | Error detail when status is ERROR, otherwise null |
foundResults | Long | Total matching documents in the index (not just the current page) |
start | Long | Zero-based offset of the first returned document |
took | Integer | Server-side processing time in milliseconds |
dymQuery | String | Spell-corrected query suggested by the LLM when dym=true. Null if no correction was needed. |
ragAnswer | String | Natural-language answer generated from the top results when rag=true |
results | Array | Array of Solr document objects. Field set determined by resultFields. |
debug | Object | Internal scoring and query detail. Present only when debug=true. |
| Value | Vector threshold | Behaviour |
|---|---|---|
high | Strict | Only results with high vector similarity are returned. Best for technical or factual queries where precision matters more than recall. |
medium | Moderate | Default. Balances precision and recall. Suitable for most general-purpose searches. |
low | Relaxed | Returns more results, including weakly related documents. Useful for exploratory search or thin corpora. |
Accepts an array of JSON documents and immediately indexes them into the client's Solr collection. Each document is chunked, embedded, and written to Solr in a single synchronous call. For large bulk imports use the Schedule API instead.
| Field | Type | Required | Description |
|---|---|---|---|
requestId | String | Yes | Correlation ID |
clientId | String | Yes | Tenant identifier |
data | Array of objects | Yes | One or more JSON documents to index. Each object can contain any fields — field importance mapping is determined by the client configuration (fieldsHigh, fieldsLow, fieldsContent). |
| Field | Type | Description |
|---|---|---|
requestId | String | Echoed from the request |
status | String | SUCCESS or ERROR |
message | String | Human-readable result or error detail |
indexed | Integer | Number of documents successfully indexed |
SANDI resolves a unique document ID in this order:
id field in the document, if presenturi field, if presentid nor uri is providedforceReindexing set to
false.
Deletes one or more documents from the client's Solr collection. Three deletion modes can be used simultaneously in a single request.
| Field | Type | Required | Description |
|---|---|---|---|
requestId | String | Yes | Correlation ID |
clientId | String | Yes | Tenant identifier |
id | String | One of three | Delete a single document by its ID |
ids | Array of String | One of three | Delete multiple documents by their IDs |
query | String | One of three | Delete all documents matching a Solr query string |
id, ids, or query must be
provided. If none are present the request returns an error with the message
"Nothing to delete".
| Field | Type | Description |
|---|---|---|
requestId | String | Echoed from the request |
status | String | SUCCESS or ERROR |
message | String | Result detail or error description |
deleted | Integer | Count field (currently always 0 — deletion count is not returned by Solr commit) |
The Schedule API manages background indexing jobs. Jobs are stored in memory and
checked every second by IndexingJobScheduler. When a job's
scheduledTime is reached it is executed asynchronously.
Create and schedule a new indexing job.
| Field | Type | Required | Description |
|---|---|---|---|
requestId | String | Yes | Correlation ID |
clientId | String | Yes | Tenant identifier |
jobType | String | Yes | See job type table below |
directory | String | Yes | File system path or URL to process |
jobId | String | No | Explicit job ID. If omitted, requestId is used. |
cron | String | No | Cron expression for recurring jobs. If omitted the job runs once. |
fileExtensions | String | No | Comma-separated file extensions to include, e.g. "pdf,docx". Used with file-based job types. |
forceReindexing | Boolean | No | If true, re-index documents that already exist. Default false — existing documents are skipped. |
scheduledTime | DateTime | No | ISO-8601 datetime when the job should run. Must be in the future. If omitted, the job is scheduled 5 seconds from now. |
createdTime | DateTime | No | Job creation timestamp. Informational. |
| Field | Type | Description |
|---|---|---|
requestId | String | Echoed from the request |
status | String | SCHEDULED on success, ERROR on failure |
message | String | Confirmation or error detail |
jobId | String | Assigned job ID — use for status polling and cancellation |
scheduledTime | DateTime | Effective scheduled execution time |
createdTime | DateTime | Job creation time |
| jobType | Description | directory value |
|---|---|---|
JSONL | Process a directory of JSONL files (one JSON object per line) | File system path |
JSON | Process a directory of JSON files (each file is a single document or array) | File system path |
TXT | Process a directory of plain text files | File system path |
TXTL | Process a directory of text files, one document per line | File system path |
EXCEL | Process Excel spreadsheets (.xlsx); each row becomes a document | File system path |
SITE | Crawl and index a website; uses Apache Tika for HTML extraction | Root URL to crawl |
SITEMAP | Index URLs listed in an XML sitemap | Sitemap XML URL |
JSONMAP | JSONL with an explicit field mapping definition | File system path |
Returns all jobs (scheduled, running, completed, failed, cancelled) as an array of IndexingJob objects.
| Field | Type | Description |
|---|---|---|
jobId | String | Unique job identifier |
clientId | String | Associated tenant |
jobType | String | Job type constant |
status | Enum | Current status: SCHEDULED / RUNNING / COMPLETED / FAILED / CANCELLED |
cron | String | Cron expression (null for one-time jobs) |
directory | String | Source file path or URL |
fileExtensions | String | Included file extensions |
forceReindexing | Boolean | Whether existing documents are overwritten |
scheduledTime | DateTime | When the job is/was scheduled to run |
createdTime | DateTime | When the job was submitted |
startedTime | DateTime | Actual execution start time |
completedTime | DateTime | Execution completion time |
message | String | Status message or error detail |
totalFiles | int | Number of files found |
totalDocuments | int | Total document records found across all files |
successfulDocuments | int | Documents indexed successfully |
failedDocuments | int | Documents that failed to index |
Returns a single IndexingJob object. Returns 404 if the job does not exist.
Returns all jobs with the given status. Valid values: SCHEDULED, RUNNING, COMPLETED, FAILED, CANCELLED (case-insensitive). Returns 400 for an unrecognised status value.
Cancels a scheduled job. Returns 400 if the job is not found or is not in SCHEDULED state.
Returns a simple counts object.
The Admin API manages tenants (clients), Solr collections, Solr configurations, and
synonym sets. All endpoints require a requestId query parameter.
Returns an array of all client objects.
Returns a single client object. Returns 404 if not found.
Creates or updates a client. If createCollection is true, a new Solr
collection is created using the named configuration set before the client record
is saved.
| Field | Type | Required | Description | |
|---|---|---|---|---|
clientId | String | Yes | Unique tenant identifier. Only word characters allowed (no spaces or special chars). | |
name | String | Yes | Human-readable display name | |
collection | String | No | Solr collection name. Required if createCollection=true. | |
createCollection | Boolean | No | When true, creates the Solr collection before saving the client record | |
configuration | String | No | Name of the Solr configuration set to use when creating a collection | |
| Field mapping — at least one array must be non-empty | ||||
fieldsHigh | String[] | * | Fields treated as high-importance for search (titles, headings). Highest BM25 boost. | |
fieldsLow | String[] | * | Fields treated as low-importance metadata (tags, categories). Lower boost. | |
fieldsContent | String[] | * | Full-text content fields (body, description). Used for chunking and embedding. | |
| AI service endpoints (override global defaults) | ||||
embServiceUrl | String | No | Embedding service URL for this client. Overrides the global sandi.service.emb.url. | |
llmServiceUrl | String | No | LLM service URL for this client | |
nlpServiceUrl | String | No | NLP service URL for this client | |
rerServiceUrl | String | No | Reranker service URL for this client | |
cliServiceUrl | String | No | Client index hook URL — called before indexing to transform each document | |
clsServiceUrl | String | No | Client search hook URL — called before and/or after search for custom processing | |
| Behaviour flags | ||||
synonyms | String[] | No | Names of synonym sets to load for this client's searches | |
legacy | Boolean | No | Default to legacy BM25-only search for this client | |
nested | Boolean | No | Enable nested document support (skips document flattening during indexing) | |
active | Boolean | No | Inactive clients are rejected by all APIs with a 400 error | |
clientSearchProcessing | Boolean | No | Enable the client search hook (clsServiceUrl) | |
clientIndexProcessing | Boolean | No | Enable the client index hook (cliServiceUrl) | |
Deletes the client record. Does not delete the associated Solr collection. Returns 404 if not found.
Returns a list of all Solr collection names.
Returns a Collection object: { "name": "acme_docs", "configuration": "sandi_config" }.
Creates a new Solr collection. Request body: { "name": "my_collection", "configuration": "sandi_config" }
Updates the configuration set linked to an existing collection. The name in the path and body must match.
Deletes the Solr collection and all its data. Returns 404 if not found.
Solr configurations (config sets) are stored in the directory specified by
sandi.collections.config. These endpoints upload them to ZooKeeper
so SolrCloud can use them when creating collections.
Returns a list of all available configuration set names.
Uploads the named configuration set from the local file system to ZooKeeper.
Deletes the named configuration set from ZooKeeper. Returns 404 if not found.
Synonym sets are uploaded to SolrCloud via ZooKeeper and referenced by name in
the client's synonyms array.
Returns a list of all synonym set names available in ZooKeeper.
Uploads a synonym file to ZooKeeper.
| Query param | Required | Description |
|---|---|---|
name | Yes (path) | Name to register the synonym set under |
fileName | Yes | Source file name on the server file system |
language | Yes | Language code, e.g. en |
requestId | Yes | Correlation ID |
Deletes the named synonym set from ZooKeeper. Returns 404 if not found.
Every document submitted to POST /index or processed by a scheduled
job passes through the following steps:
clientIndexProcessing=true on the client, the raw document JSON is POSTed to cliServiceUrl. The hook response replaces the original document. Use this to normalise, enrich, or filter documents using custom logic.id field, then uri field, then a generated UUID.forceReindexing=false, the resolved UUID is checked against existing Solr records. Documents already indexed are skipped.client.nested=true, nested JSON objects are flattened into dot-notation fields (e.g. address.city).POST /embed). The service returns a dense float vector. Chunks are batched — default batch size is 50._embs field; chunks in _chunks.When a search request is received the following steps are executed, each step conditional on the corresponding request flag:
requestId, clientId, pageSize (≤ 10 000), and pageNumber (≥ 1) are validated. The client record is fetched and verified as active.clientSearchProcessing=true, the request is sent to clsServiceUrl for pre-processing before any AI calls.synonyms=true). The query is expanded with terms from the client's synonym sets stored in Solr.dym=true). The query is sent to the LLM service with a spell-checking prompt. If the LLM returns a corrected query, it is stored in dymQuery and used for the remainder of the pipeline.expand=true). The LLM generates 2–3 alternative phrasings of the query. These are merged into the Solr boolean query as optional clauses.fieldsHigh/fieldsLow/fieldsContent, fused with kNN vector search results. Fusion weights are configurable (weightLegacyQuery default 0.4, weightVectorQuery default 0.8).rerank=true). The current result page is sent to the reranker service with the original query. The service returns relevance scores and the result list is reordered.rag=true). The top-N document excerpts are concatenated into a context window and sent to the LLM with a question-answering prompt. The generated answer is returned in ragAnswer.Each AI service is a lightweight Python/Flask microservice. SANDI calls them over HTTP; the interface contract below is what SANDI expects, regardless of whether the service is backed by a local GPU model or a cloud API.
Converts one or more text strings into dense float vectors. Called during indexing (per chunk) and during search (per query).
Computes cosine similarity between two texts. Used internally for relevance checks.
Runs SpaCy NLP analysis on the input text. Returns entities, part-of-speech tags, and lemmatized tokens. Used by the search pipeline to understand query intent.
Sends a prompt to the language model and returns generated text. Used for spell check, query expansion, and RAG answer generation. The caller constructs the full prompt; the service returns the raw model output.
/run endpoint is used for all three LLM tasks: spell
checking, query expansion, and RAG. SANDI builds a task-specific prompt and
parses the plain-text response accordingly.
Accepts a query and a list of document text snippets. Returns a relevance score for each document computed by a cross-encoder model. SANDI uses these scores to reorder the search result page.
Scores are in the same order as the input docs array. SANDI sorts the
result page by descending score after receiving the response.
The client record is the central configuration object for a tenant. It is stored
as a Solr document in the internal _clients collection and loaded at
request time for every search and index operation.
| Field | Type | Description |
|---|---|---|
clientId | String | Unique identifier. Word characters only. Used in all API requests as the tenant key. |
name | String | Human-readable display name. Used in admin UIs and log messages. |
collection | String | Name of the Solr collection that stores this client's documents. Multiple clients can share a collection. |
active | Boolean | When false, all API calls for this client are rejected with 400. |
fieldsHigh | String[] | Document fields treated as high-relevance (titles, headings). Receive the highest BM25 boost factor. Also used for high-weight text in the embedding chunk. |
fieldsLow | String[] | Document fields treated as low-relevance metadata (author, category, tags). Lower BM25 boost. |
fieldsContent | String[] | Full-text content fields (body, description, abstract). Used for chunking, embedding, and content-level BM25. |
synonyms | String[] | Names of synonym sets to apply when synonyms=true is passed on a search request. |
legacy | Boolean | When true, this client always uses BM25-only search regardless of the request flag. |
nested | Boolean | When true, document flattening is skipped and nested Solr documents are used. Requires a Solr schema with _nest_path_. |
embServiceUrl | String | Override the global embedding service URL for this client. Useful when different clients use different embedding models. |
llmServiceUrl | String | Override the global LLM service URL for this client. |
nlpServiceUrl | String | Override the global NLP service URL for this client. |
rerServiceUrl | String | Override the global reranker service URL for this client. |
cliServiceUrl | String | URL of the client index hook service. Called before indexing each document when clientIndexProcessing=true. |
clsServiceUrl | String | URL of the client search hook service. Called around the search pipeline when clientSearchProcessing=true. |
clientIndexProcessing | Boolean | Enables the client index hook. Requires cliServiceUrl to be set. |
clientSearchProcessing | Boolean | Enables the client search hook. Requires clsServiceUrl to be set. |