Vector Store Configuration

Vector store settings determine where your document embeddings are stored and how they're accessed.

Select the project first, then choose a provider from the card grid. The selected card shows its supported embedding dimensions and distance metrics. Configure that provider below the grid, enter its credential, and click Save or Save Key. Either button saves the connection and the credential, then tests the connection automatically. If the index or table is missing, its setup option (for example Create Index for Pinecone) appears straight away. Test Connection remains available to re-check later. Changing the provider does not move existing vectors; re-sync documents to populate the new destination.

Available Providers

Provider	Description	Supported Dimensions	Best For
Supabase	PostgreSQL with pgvector extension	1–2,000	Integrated solution, SQL queries
PostgreSQL / pgvector	Any public PostgreSQL database with pgvector	1–2,000	Existing managed PostgreSQL accounts
Pinecone	Managed vector database	1–4,096	High scale, low latency
Cloudflare Vectorize	Globally distributed managed vector database	1–1,536	Cloudflare applications and edge retrieval
Qdrant	Open-source vector database	1–4,096	Self-hosted, flexible deployment
Milvus	Open-source distributed vector database	1–4,096	Self-hosted or Zilliz Cloud

These ranges describe the configurations currently supported and tested by Vector Data Loader. The Supabase and generic PostgreSQL adapters create pgvector HNSW indexes over the vector type, so their current limit is 2,000 dimensions.

Provider compatibility is checked against the embedding model saved for the selected project. An existing index must use the same vector dimension; if it does not, choose a compatible model or initialize a separate index. Credentials and connection settings are scoped to the selected project.

Supabase Configuration

When using Supabase (the default):

1. Service URL: Your Supabase project URL
2. Service Key: Service role key for server-side operations
3. Table Name: Name of the table storing vectors (default: document_chunks)

Setup Status: Shows whether the required database table exists

Use the Project URL, not the MCP server URL

The Service URL must be your project's Data API endpoint — an origin with no path:

https://your-project-ref.supabase.co

Find it in Supabase under Project Settings → Data API → Project URL.

Other Supabase URLs will not work, even though they look similar. The most common mistake is pasting the MCP server URL:

https://mcp.supabase.com/project_ref=your-project-ref   ❌ not a Data API endpoint

That address serves the Model Context Protocol, not PostgREST, so every read and write returns 404. Saving a URL with a path, query string, or the mcp.supabase.com host is now rejected with an explanatory error.

Initialize Database: Creates the necessary table and indexes if not present

Auto-Setup Limitation for External Supabase Projects

The Initialize Database button uses automatic setup, which requires the exec_sql helper function to be present on your Supabase database. This is a technical limitation of Supabase's REST API (PostgREST), which cannot execute DDL statements like CREATE TABLE directly.

For external Supabase projects (not the app's built-in database):

1. Click Initialize Database - it will attempt auto-setup
2. If it fails, a Manual Setup SQL box will appear
3. Copy the SQL and run it in your Supabase project's SQL Editor
4. Click Test Connection to verify the setup succeeded

This is a one-time setup. Once the table and functions are created, everything works automatically.

Setup update available: Supabase stores set up before this update work as before, but Test Connection shows a Setup update available notice. The update adds:

• a replace_document_chunks function, so a re-sync replaces a document's chunks in one transaction: if the re-sync fails part-way, the previous version stays searchable;
• the image_ids column, needed to store chunks linked to extracted images;
• a small vdl_statement_timeout_probe function that checks your project applies the function's 60-second time limit.

Choose Apply update (needs the exec_sql helper, as above), or copy the update SQL from the Manual Setup SQL box and run it in your project's SQL Editor, then choose Test Connection again. The update only adds these items; it does not change your table's data, your search functions or your access policies, and it is safe to run more than once. Each table configured on the same Supabase project needs its own update run. Custom API sources require it, together with PostgREST 12.2 or later (all current Supabase projects).

PostgreSQL / pgvector Configuration

Use this provider to connect a PostgreSQL database that is independent of Supabase. Configure:

1. Host and Port: A public PostgreSQL endpoint and its TCP port. Enter the hostname only, without a URL scheme or credentials.
2. Database and Username: The target database and a dedicated database role.
3. Database Password: Saved separately, encrypted at rest, and never returned to the browser.
4. Table Name: The table used for chunks (default: document_chunks).
5. TLS Certificate Validation: Use Verify certificate whenever the provider has a publicly trusted certificate. Use Require encryption only when the provider requires TLS but does not present a verifiable certificate chain.

Save the connection and password, then choose Test Connection. If pgvector, the table, or the cosine HNSW index is missing, choose Initialize Database. The database role must be allowed to create the vector extension, table, and indexes. If it lacks those privileges, copy the generated SQL and have a database administrator run it. Re-syncs replace a document's chunks in one transaction, so a failed re-sync keeps the previous version searchable.

Public endpoints and tenant filters

The current release rejects localhost, private IP ranges, and internal hostnames. PostgreSQL traffic always uses TLS. Vector Data Loader filters every data operation by organization_id and project_id; external applications must apply both filters as well.

Auto-Setup Comparison

Provider	Auto-Setup	Notes
Pinecone	✅ Works	Native SDK supports index creation
Cloudflare Vectorize	✅ Works	Creates the index and required tenant metadata indexes through the Cloudflare API
Qdrant	✅ Works	REST API supports collection creation
Milvus	✅ Works	REST API v2 supports collection creation
PostgreSQL / pgvector	✅ Works	Requires a role that can create the extension, table, and indexes
Supabase	⚠️ Manual	Requires SQL execution in Supabase Dashboard

Pinecone Configuration

For Pinecone:

1. API Key: Your Pinecone API key
2. Index Name: Name of your Pinecone dense-vector index
3. Namespace: Set automatically. Each project writes to its own namespace, named after the project ID, so projects can share one index.

The index dimension must match the embedding model configured for the project. Saving tests the connection; if the index does not exist yet, choose Create Index to create a serverless index with the project's embedding dimensions. Use Test Connection to re-check the key and index before syncing documents.

Cloudflare Vectorize Configuration

Configure the following values from the Cloudflare dashboard:

1. Account ID: The 32-character identifier for the Cloudflare account that owns the index.
2. Index Name: An existing Vectorize V2 index, or the name to create during initialization.
3. API Token: A scoped token with Vectorize Read and Vectorize Write permissions. The token is encrypted at rest and never returned to the browser.

Choose Test Connection to inspect the index and tenant metadata indexes. Choose Initialize Vectorize to create a missing index using the current project's embedding dimensions and cosine metric, then create string metadata indexes for organization_id and project_id.

Tenant isolation and asynchronous writes

Vector Data Loader derives a namespace from the authenticated organization and selected project. Searches also require matching organization_id and project_id metadata. Vectorize mutations are asynchronous, so newly synced or deleted vectors can take a few seconds to appear in queries.

Vectorize currently supports up to 1,536 dimensions. Select an embedding model at or below that limit before saving or initializing this provider. External clients must reproduce the application's namespace derivation and apply both metadata filters; Connection Details does not expose the namespace hash or an API token.

Qdrant Configuration

For Qdrant:

1. API Key: Authentication key required by the current initial-sync flow
2. Server URL: Qdrant server endpoint
3. Collection Name: Target collection for vectors

Each chunk is stored as a point whose ID is a UUID derived from its organization, document and chunk index, so re-syncing a document overwrites its points instead of adding new ones. Searches and deletes filter on the organization_id, project_id and document_id payload fields, which Initialize indexes. Test Connection lists any of those indexes the collection is missing (for a collection created outside the app, or after an index failed during setup); choose Create Collection to create them. An existing collection and its points are left in place.

If syncs to Qdrant failed with an error saying a value "is not a valid point ID", re-sync those documents: earlier versions sent chunk IDs that Qdrant rejects.

Milvus Configuration

Milvus is an open-source distributed vector database designed for high-performance similarity search. It supports both self-hosted deployment and Zilliz Cloud, the fully managed Milvus service.

Configuration fields:

1. Address: Milvus server URL. For self-hosted Milvus, use an authenticated endpoint reachable from the hosted app. Zilliz Cloud: your cluster's Public Endpoint (e.g. https://in03-xxxx.serverless.aws-us-east-1.cloud.zilliz.com).
2. Collection Name: Name of the collection to store vectors (default: document_chunks). The collection is created automatically when you click Initialize.
3. Token: Authentication token. Required by the current initial-sync flow for both Zilliz Cloud and self-hosted deployments.

Getting Started with Milvus

Self-hosted (Docker):

1. Start Milvus standalone:

   docker run -d --name milvus-standalone -p 19530:19530 milvusdb/milvus:latest standalone

2. Enter the authenticated server endpoint reachable from Vector Data Loader as the Address. localhost on your computer is not reachable from the hosted app.
3. Enter the authentication Token. Connection testing alone does not confirm that initial sync will accept an unauthenticated configuration.
4. Click Initialize — the collection is created automatically.

Zilliz Cloud (managed Milvus):

1. Create a cluster at cloud.zilliz.com.
2. Copy the Public Endpoint URL and paste it as the Address.
3. Copy the API Key and paste it as the Token.
4. Click Initialize — the collection is created automatically.

Initialize Database

Unlike Supabase, Milvus auto-setup works out of the box. The collection is created automatically via the Milvus REST API v2 when you click Initialize. No manual SQL execution is required.

Troubleshooting Milvus

• Connection refused: Verify Milvus is running and port 19530 is accessible from the server.
• Unauthorized: Check that your token is correct. Zilliz Cloud always requires a token; self-hosted without authentication should have a blank token.
• Collection not found: Click Initialize to create the collection automatically.

Testing Connection

Click Test Connection to verify your configuration:

• Checks connectivity to the provider
• Validates credentials
• Confirms collection/table existence

Embedding Compatibility

Vector Data Loader checks the vector-store provider against the embedding model selected for the current project. It blocks settings or initialization when the embedding dimensions exceed the adapter's supported range. When the provider exposes index metadata, it also compares the live index dimension with the embedding model dimension.

The compatibility panel reports one of these states:

• Compatible: The live index and embedding model use the same dimensions.
• Incompatible: The provider limit or live index dimension conflicts with the embedding model. Choose a compatible embedding dimension or create a matching index before syncing.
• Unverified: The provider cannot report a dimension yet, which is common for an empty Supabase table. Compatibility is checked again after vectors exist.

Initialization always uses the saved embedding configuration for the current project. A browser request cannot override the dimension.

Index Information

After a successful connection test, an Index Information card appears showing:

Field	Description
Dimension	Vector dimension (e.g., 1536 for OpenAI text-embedding-3-small)
Distance Metric	Similarity algorithm used (Cosine, Euclidean, or Dot Product)
Status	Current index status (Ready, Initializing, Error, or Unknown)
Vectors	Number of vector embeddings stored. Shows “—” if the vector store couldn't count them, rather than 0

Project-Scoped Configuration

Vector store settings are configured per-project. This allows different projects to use different vector stores or configurations, enabling:

• Separation of data by client or use case
• Different performance configurations
• Testing environments