The Vector as an AI Primitive

Is the vector where meaning itself is encoded?

Hi, I’m Matt Hartman. In this Substack I share observations and analysis on frontier technology. Earlier this year, I started Factorial, which has a focus on AI (more info at the bottom of this note).

When there are new tech paradigms interfaces or technologies, new products often need to be designed natively. I came across this piece by the team at USV that uses the notion of “primitives” in mobile (e.g., GPS, touch, constant Internet connection —all of which led to mobile native products from Instagram to Uber), and asks what AI primitives might be.

Entrepreneurs building mobile first products experimented with these primitives while building natively for the iPhone. AI native startups will similarly discover and experiment, right now both developers and investors are trying to discern what the Primitives of AI are, and what they unlock.

One possible AI primitive is the vector itself — it’s where the meaning lives. 

Vector == Meaning

In What GPT Actually Means, I noted that the original word2vec paper on embeddings showed how you could point a vector from Paris to France and when you placed the same vector starting at Rome, it would point to Italy. The vector itself means the idea of a “capital city” — the vector is where the meaning itself is encoded.

(Diagram from the original word2vec paper)

Why primitives matter

When we build natively for AI, we can’t just ask what AI can do, we should be asking what AI can do 10x better and that wasn’t possible before its existence.

Thinking of the vector as a primitive could mean that “fuzzy search” becomes the default interface instead of a fallback, or it could mean entire products can be designed with the assumption that computers can intuit how two concepts are related.

An Example

One idea we’re experimenting with at Factorial is to take each of our contacts and embed them, placing them in a vector database. Then we can take the “asks” founders send us (e.g., “can you introduce us to a customer in a certain industry”) and overlay them as vectors as well.

This acknowledges the vector as a primitive but is just the starting point. Quickly we learn that asks and people won’t show up in the same place as vectors what we actually have to do is take an ask and hallucinate a theoretical person who would be a great resource for the ask. We can compare that performance to doing the inverse — generating theoretical asks for each person we actually know and then mapping the asks out, not the people.

It also turns out this approach not only shows us who in our network can answer some of the founder asks, but also shows us when we can’t help. When the distances are too far, we’ll learn that we don’t know anyone who can meaningfully help. This level of understanding comes from vector distances and is fundamentally different from simply outputting a set of people ina particular order of relevance.

What this means for AI-First Teams

At the beginning of a new technology adoption curve as we are in with AI, using these primitives in a way that is helpful to end users requires iteration.

We’re already seeing this play out at OpenAI: its “playground” interface for GPT3 became its “chat” interface for GPT3.5 and suddenly made sense to end users. This seems likely to be the product development motion of AI native startups this decade.

Matt Hartman is Managing Partner at Factorial Capital. Factorial is the first of its kind multi-manager early stage firm. We partner with a handful of successfully exited founders (and some who are still operating their late stage companies), each with a specific area of expertise.

AI is our first practice area given my experience in the category as both a coder and an investor in AI since 2016. We’ve started to be a bit more public about Factorial: Hugging Face CEO teams up with newly launched venture firm Factorial to invest in AI startups and we’ll share more of our teammates soon.