SANDI Solr

Installation Guide

Overview

SANDI Solr is a hybrid search system combining Apache Solr with dense vector embeddings, LLM-based query enhancement, and cross-encoder reranking. This guide covers single-machine installation for learning and testing purposes. For production deployments, each service should be hosted on dedicated machines to maximize performance and reliability.

Installation modes

SANDI Solr supports two installation modes depending on your available hardware and preferences:

Local AI Services (GPU)

  • All AI models run locally on your machine
  • Requires an NVIDIA GPU with at least 24 GB VRAM
  • Requires at least 32 GB RAM
  • No external API subscriptions needed
  • No ongoing API usage costs

OpenAI Services (No GPU)

  • AI work is offloaded to OpenAI's cloud
  • No GPU required — runs on a laptop with 16 GB RAM
  • Requires an active OpenAI subscription
  • Incurs API usage costs per volume of data

Installation with Local AI Services (GPU)

In this mode all embedding, language, and reranking models run locally on your machine. This requires a powerful machine with an NVIDIA GPU. No external API subscriptions are needed.

Prerequisites

RequirementRecommendedNotes
Operating systemUbuntu 24.04.4 LTSOther Linux distributions may work
RAM32 GB or moreRequired for local LLM and embedding services
GPUNVIDIA 24 GB VRAM cardNVIDIA RTX 3090 or equivalent; NVIDIA drivers must be installed
Disk100 GB freeFor Docker images and model weights
Container runtimeDocker with NVIDIA Container ToolkitSee step 1
PythonPython 3Required for setup utilities
1

Install Docker and verify NVIDIA drivers

Ubuntu 24.04.4 LTS is the recommended operating system with Python 3 pre-installed. Follow the official Docker installation guide for Ubuntu:

https://docs.docker.com/engine/install/ubuntu/

After installing Docker, verify that your NVIDIA drivers are installed and the GPU is visible:

nvidia-smi

You should see your GPU model, driver version, and CUDA version listed in the output. If the command is not found, install the NVIDIA drivers for your GPU before continuing.

2

Download the SANDI installation package

Download the SANDI Solr installation package from the SoftCorporation website:

https://softcorporation.com/products/sandi/downloads/sandi-solr-0.4.1.zip

Note: The current version is still in active development and is not yet published on GitHub. Check the download page for the latest available version.

You can download directly from the terminal:

wget https://softcorporation.com/products/sandi/downloads/sandi-solr-X.X.X.zip
3

Unpack the installation package

Extract the ZIP archive to a directory where you have full write permissions. The home directory is recommended:

cd ~ unzip sandi-solr-X.X.X.zip

This will create a sandi-solr-X.X.X directory containing all configuration files, Docker Compose definitions, and Solr collection configs.

4

Start SANDI Solr with Docker Compose

Navigate into the extracted directory and start all services using Docker Compose:

cd ~/sandi-solr-X.X.X docker compose up -d

Docker will pull the required images and start all containers in detached mode. This may take long time on first run as images are downloaded.

5

Verify all services are running

When startup completes successfully, you should see output similar to the following:

Network sandi-solr_sandi_net Created 0.1s Container sandi_rer1 Started 1.0s Container sandi_emb4 Started 1.0s Container sandi_zoo1 Started 0.7s Container sandi_llm2 Started 1.0s Container sandi_cls1 Started 1.0s Container sandi_zoo3 Started 0.9s Container sandi_nlp1 Started 0.8s Container sandi_zoo2 Started 0.9s Container sandi_cli1 Started 0.8s Container sandi_solr1 Started 1.1s Container sandi_solr2 Started 1.1s Container sandi_search1 Started 1.6s Container sandi_index1 Started 1.6s

You can check the status of all running containers at any time with:

docker compose ps
6

Open the SANDI web interfaces

Once all containers are running, open the following URLs in your browser:

InterfaceURL
Search API http://localhost:8081/sandi/en/sandi-search.html
Index API http://localhost:8082/sandi/en/sandi-index.html
Admin API http://localhost:8082/sandi/en/sandi-admin.html

Installation with OpenAI (No GPU Required)

In this mode SANDI Solr uses OpenAI API services for embedding, language processing, and reranking instead of running local GPU-based models. You do not need a powerful machine with a GPU — SANDI will run even on a laptop with 16 GB RAM. The AI-intensive work is offloaded to OpenAI's cloud infrastructure, so local hardware requirements are minimal. However, this requires an active subscription to OpenAI services and incurs API usage costs depending on the volume of documents indexed and queries processed.
Requirement: You need an active OpenAI subscription and a valid API key. Obtain your API key at https://platform.openai.com/api-keys.

Prerequisites

RequirementRecommendedNotes
Operating systemUbuntu 24.04.4 LTSOther Linux distributions may work
RAM16 GB or moreNo GPU memory needed — AI services run in OpenAI cloud
GPUNot requiredLocal GPU services are disabled in this mode
Container runtimeDockerNVIDIA Container Toolkit is not required
OpenAI API keyActive subscriptionUsed by embedding, LLM, and reranking services
PythonPython 3Required for setup utilities
1

Install Docker

Ubuntu 24.04.4 LTS is the recommended operating system with Python 3 pre-installed. Follow the official Docker installation guide for Ubuntu:

https://docs.docker.com/engine/install/ubuntu/

No NVIDIA drivers or GPU toolkit are required for this installation mode.

2

Download the SANDI installation package

Download the SANDI Solr installation package from the SoftCorporation website:

https://softcorporation.com/products/sandi/downloads/sandi-solr-0.3.0.zip

Note: The current version is still in active development and is not yet published on GitHub. Check the download page for the latest available version.

You can download directly from the terminal:

wget https://softcorporation.com/products/sandi/downloads/sandi-solr-0.3.0.zip
3

Unpack the installation package

Extract the ZIP archive to a directory where you have full write permissions. The home directory is recommended:

cd ~ unzip sandi-solr-X.X.X.zip

This will create a sandi-solr-X.X.X directory containing all configuration files, Docker Compose definitions, and Solr collection configs.

4

Set your OpenAI API key

Open the .env file in the extracted directory and set your OpenAI API key:

OPENAI_API_KEY=sk-...

Docker Compose automatically loads this file at startup and passes the key to the OpenAI services.

5

Activate OpenAI services in docker-compose.yml

Open docker-compose.yml in a text editor. Services that contain a profiles: [manual_start] block are excluded from the default startup. Commenting those lines activates the service. You need to activate the three OpenAI-backed services and deactivate their GPU-dependent counterparts.

Embedding — activate sandi_emb5, deactivate sandi_emb4:

sandi_emb5: # OpenAI embedding — comment out profiles to activate ... # profiles: # - manual_start sandi_emb4: # GPU embedding — uncomment profiles to deactivate ... profiles: - manual_start

LLM — activate sandi_llm4, deactivate sandi_llm3:

sandi_llm4: # OpenAI LLM — comment out profiles to activate ... # profiles: # - manual_start sandi_llm3: # GPU LLM — uncomment profiles to deactivate ... profiles: - manual_start

Reranking — activate sandi_rer2, deactivate sandi_rer1:

sandi_rer2: # OpenAI reranking — comment out profiles to activate ... # profiles: # - manual_start sandi_rer1: # GPU reranking — uncomment profiles to deactivate ... profiles: - manual_start
6

Start SANDI Solr with Docker Compose

Save docker-compose.yml, then navigate into the extracted directory and start all services:

cd ~/sandi-solr-X.X.X docker compose up -d

Docker will pull the required images and start all containers in detached mode. This may take long time on first run as images are downloaded.

7

Verify all services are running

When startup completes successfully, you should see output similar to the following. Note that sandi_emb5, sandi_llm4, and sandi_rer2 appear instead of their GPU counterparts:

Network sandi-solr_sandi_net Created 0.1s Container sandi_rer2 Started 1.0s Container sandi_emb5 Started 1.0s Container sandi_zoo1 Started 0.7s Container sandi_llm4 Started 1.0s Container sandi_cls1 Started 1.0s Container sandi_zoo3 Started 0.9s Container sandi_nlp1 Started 0.8s Container sandi_zoo2 Started 0.9s Container sandi_cli1 Started 0.8s Container sandi_solr1 Started 1.1s Container sandi_solr2 Started 1.1s Container sandi_search1 Started 1.6s Container sandi_index1 Started 1.6s

You can check the status of all running containers at any time with:

docker compose ps
8

Open the SANDI web interfaces

Once all containers are running, open the following URLs in your browser:

InterfaceURL
Search API http://localhost:8081/sandi/en/sandi-search.html
Index API http://localhost:8082/sandi/en/sandi-index.html
Admin API http://localhost:8082/sandi/en/sandi-admin.html

Services Overview

The Docker Compose stack starts the following services. The AI services differ between installation modes:

ContainerRoleMode
sandi_zoo1, sandi_zoo2, sandi_zoo3Apache ZooKeeper cluster — coordinates SolrCloudBoth
sandi_solr1, sandi_solr2Apache Solr 9.8.1 nodes — storage and retrievalBoth
sandi_nlp1NLP service — entity extraction, POS tagging, lemmatizationBoth
sandi_cls1Client search processing serviceBoth
sandi_cli1Client index processing serviceBoth
sandi_search1SANDI Search REST API (port 8081)Both
sandi_index1SANDI Index (and Admin) REST API (port 8082)Both
sandi_emb4Embedding service — GTE-Large model (local GPU)Local AI
sandi_llm3LLM service — spell check, query expansion, RAG answers (local GPU)Local AI
sandi_rer1Reranking service — Qwen3-Reranker-0.6B cross-encoder (local GPU)Local AI
sandi_emb5Embedding service — OpenAI APIOpenAI
sandi_llm4LLM service — OpenAI APIOpenAI
sandi_rer2Reranking service — OpenAI APIOpenAI

Production Deployment

The single-machine setup described in this guide is intended for learning, evaluation, and development. For production use, each service group should run on dedicated hardware:

Troubleshooting

Containers exit immediately after start

Check logs for the affected container:

docker compose logs sandi_solr1

GPU not detected by containers (local AI services mode only)

Ensure the NVIDIA Container Toolkit is installed and Docker is configured to use it. Verify GPU access inside a container:

docker run --rm --gpus all nvidia/cuda:12.0-base-ubuntu22.04 nvidia-smi

Services not reachable in the browser

Allow a minute after startup for all services to initialize, particularly the Solr nodes and ZooKeeper cluster. Check that no other process is using ports 8081 or 8082:

ss -tlnp | grep -E '8081|8082'

Stopping SANDI Solr

To stop all running containers:

docker compose down