How to Choose a Vector Database: Five Decisions That Are Hard to Reverse Later 

If you are working out how to choose a vector database, five decisions matter most: where the vectors live (inside Postgres or in a separate service), which embedding model and vector size you commit to, how much data and how many tenants you plan for, what speed and accuracy targets you set, and which index type you build. Some of these take an afternoon to change. Others mean moving every record you have. Any useful vector database comparison should weigh them in that order. 

This guide to how to choose a vector database orders the five decisions from easiest to hardest to undo, so your team spends its review time where a wrong call costs the most. 

Key takeaways 

• Index type is the easiest decision to change. The storage location of your vectors is the hardest. 

• pgvector runs an exact search by default and supports HNSW and IVFFlat approximate indexes, each with documented trade-offs. 

• pgvector stores vectors up to 16,000 dimensions but indexes standard vectors only up to 2,000, which matters for large embedding models. 

• Changing the embedding model later means re-embedding your whole corpus, so test the model before you load production data. 

• In healthcare, embeddings built from clinical notes carry patient context and need the same protection as the source records. 

Why a vector database choice tends to stick 

A vector database stores embeddings: lists of numbers that represent the meaning of a piece of text, an image or a record. A retrieval-augmented generation (RAG) system searches those embeddings to find the passages an AI model should read before it answers. Our guide to private LLM development explains how this retrieval step fits into a wider AI system. 

Once an application depends on that search, the database sits under every query. Changing it later touches ingestion pipelines, access controls, backups and the application code that sends queries. 

The space is also growing fast, which adds pressure to decide quickly. MarketsandMarkets estimates the vector database market at USD 2.65 billion in 2025, rising to USD 8.95 billion by 2030, a 27.5% annual growth rate. More options mean more vendor claims to test, and a vector database comparison based on marketing pages alone tends to miss the costs that appear a year later. 

Decision 1: Which index type will you build first? 

This is the easiest decision to reverse, because an index can be dropped and rebuilt without moving the data. It still deserves care, since a rebuild on a large table can take hours and use a lot of memory. Figuring out how to choose a vector database usually comes down to five decisions, ranked here from easiest to reverse to hardest.

pgvector’s documentation states that by default it “performs exact nearest neighbor search, which provides perfect recall”. Recall means the share of the true closest matches that a search returns, and it sits at the center of every hnsw vs ivfflat decision. Exact search is accurate but slows down as the table grows, so most teams add an approximate index. 

The hnsw vs ivfflat choice comes down to a trade-off the pgvector docs spell out. HNSW “has better query performance than IVFFlat” but “has slower build times and uses more memory.” IVFFlat has “faster build times and uses less memory than HNSW.” HNSW itself comes from a 2016 research paper by Malkov and Yashunin that layers several graphs so a search can start broad and narrow down quickly. 

Cost to undo later: low. Plan to rebuild time and memory and test both index types on a copy of real data. 

Decision 2: What speed and accuracy will users accept? 

Set a target for response time and a target for recall before choosing settings. Both targets shape every later tuning step, and changing them after launch usually means retuning indexes and retesting the whole search flow. 

Filtering is where many teams meet a surprise. pgvector notes that with approximate indexes, “queries with filtering can return less results since filtering is applied after the index is scanned.” A search for “notes from the cardiology department in the last 30 days” may return fewer results than requested. pgvector offers iterative index scans to address this, and dedicated databases handle filtered search in their own ways. 

Cost to undo later: medium. Write down the latency and recall targets with real queries, including filtered ones, before you tune. 

Decision 3: How much data and how many tenants will it hold? 

Plan for the data volume you expect in two years, which is usually far larger than the pilot. The number of vectors, how fast new ones arrive, and how data is split between customers or departments all decide whether one database server is enough. Getting this right is as much a part of how to choose a vector database as picking the index type itself.

Splitting data later is hard. If you store all tenants in one table and later need separate storage per hospital or per client for compliance, every record must move. pgvector’s docs also note that indexes “build significantly faster” when the graph fits in memory, so growth affects both query speed and rebuild time. 

Cost to undo later: high. Decide how data is partitioned between tenants on day one, even if you start small. 

Decision 4: Which embedding model and vector size will you commit to? 

The embedding model turns your content into vectors, and every stored vector depends on it. Switching models later means re-embedding the entire corpus, because vectors from two different models cannot be compared with each other. 

Vector size, measured in dimensions, interacts with your database limits. OpenAI’s documentation lists 3,072 dimensions for text-embedding-3-large and 1,536 for text-embedding-3-small, and lets you request shorter vectors at some cost to accuracy. 

pgvector stores vectors “up to 16,000 dimensions,” but its HNSW and IVFFlat indexes support standard vectors only “up to 2,000 dimensions,” with half-precision vectors up to 4,000. A 3,072-dimension model therefore needs half-precision storage or shorter vectors before it can be indexed in pgvector. This limit is easy to miss in a prototype and hard to fix after millions of rows are loaded. 

Cost to undo later: high. Test two or three models on your own documents and confirm the vector size fits your index limits before loading production data. 

Decision 5: Do the vectors live inside Postgres or in a separate service? 

This is the hardest decision to reverse, because it shapes your data pipelines, backups, security models, and team skills. Moving later means migrating every vector, rewriting query code, and running two systems in parallel during the switch. 

Keeping vectors in Postgres with pgvector means one database to secure, back up and monitor. Your metadata and vectors sit in the same transaction, which makes joins and access rules simpler. A dedicated vector database is built for vector search only and often scales it further but adds a second system to keep in sync. 

Cost to undo later: very high. Base the choice on how to choose a vector database using the criteria in the next section, and on a short test with your real data.

Pgvector vs dedicated vector database: five criteria side by side 

The table below turns the five decisions into vector database selection criteria you can score. It reflects documented pgvector behavior and the general design of dedicated services; check any specific product’s current documentation before you commit. 

Criterion	pgvector (Postgres extension)	Dedicated vector database
Scale	Fits many applications on one well-sized server; very large or fast-growing collections need careful planning	Built to spread vector search across machines as collections grow
Latency	Fast with HNSW; filtered queries may need iterative scans	Tuned for low-latency vector search, with filtering built into the engine
Index type	HNSW and IVFFlat, rebuildable with standard SQL	Varies by product; often HNSW-based plus vendor-specific options
Dimension limits	Stores up to 16,000; indexes up to 2,000 (vector) or 4,000 (halfvec)	Varies by product; check the limit against your embedding model
Operational overhead	One database for data, metadata and vectors; existing Postgres skills apply	A second system to secure, back up, monitor and keep in sync

A practical reading of this pgvector vs dedicated vector database table: if your team already runs Postgres well and the collection fits on one server, start there. If vector search is the core of the product at a very large scale, test a dedicated service early. 

How to Choose a Vector Database: A Healthcare Search Feature, Decided with the Ladder

This walkthrough shows how to choose a vector database in practice, working down the same five-decision ladder end to end. Here is an illustrative example. The platform team at a regional hospital network is building a vector database for RAG, so clinicians can search for policy documents and discharge summaries in plain language. The team already runs Postgres for its scheduling and records systems. 

Working down the ladder, the team chose pgvector, because the collection of a few million passages fits on one server, and the security team already audits Postgres. Because this vector database for RAG would answer clinical staff, it tested two embedding models on real discharge summaries and picked one that fits the 2,000-dimension index limit. It splits data by facility from the start, because some facilities need separate retention rules. 

Patient privacy shaped the setup. Embeddings built from clinical notes carry patient context, so the team protected them with the same access controls and audit logging as the source records. Where data had to be de-identified first, it followed the HHS guidance on the Expert Determination and Safe Harbor methods. 

For more on the data work behind features like this, see our guide to data engineering in healthcare and the healthcare solutions ViitorCloud builds. 

A one-week test before you commit 

A short, structured test answers most of these questions with your own data. Use these steps as a simple process your team can repeat: 

1. Collect 200 to 500 real queries, including filtered ones, with the documents a person would expect for each. 

2. Embed a realistic sample of your corpus with two or three candidate models and confirm each vector size fits your index limits. 

3. Load the sample into each candidate store, for example pgvector and one dedicated database. 

4. Build HNSW and IVFFlat indexes where available, and record build time and memory use. 

5. Measure recall and response time for every query against the targets you wrote down in Decision 2. 

6. Estimate the full-scale cost, including storage, compute, backups and the team time to run each option. 

7. Record the decision with the numbers, so a future team can see why the choice was made. 

How to choose a vector database your team will keep 

Knowing how to choose a vector database comes down to spending most of your effort on the decisions at the bottom of the ladder: storage location, embedding model and data layout. ViitorCloud’s data analytics team helps engineering teams run this test, choose the store and build the retrieval pipeline around it. To discuss your retrieval setup, talk to our team.