A concept not unique to RAG, query rewriting is the act of adjusting the user’s input to enhance results. In a RAG application, this can mean providing additional context beyond the query itself.

A common example:

User: How many zip codes are there in Texas?

AI: There are approximately 1,900–1,935 zip codes.

User: And how about Colorado?

Sending the last message from the user raw would result in no context for what was being asked.

For conversations with few turns and short messages, it may be enough to send the message history with context instructing the model to answer the last user message based on conversational context.

To narrow the focus, though, it would be beneficial to add a step where another model call rephrases the query.

AI Refined query: How many zip codes are in Colorado?

Moving query rewriting into a separate process allows for greater complexity. Say that this is an AI application integration within a specialized business. For a more specific question, such as "How many zip codes are in our third market?", the query rewriting process may include a step that involves retrieving the markets served and identifying which one we haven’t already addressed. This could be surfaced through a suite of read-only tools that connect to your data.

Another dimension of this includes multi-query messages:

const initialQuery = "Which state has the most zip codes? Which state has the fewest? Which state has the zip code with the highest numerical value?"

// invocation to break up sub-queries...

// resulting query
const query = [
  "Which state has the most zip codes?",
  "Which state has the fewest?",
  "Which state has the zip code with the highest numerical value?"
]

These can then be handled individually, later to be merged into a cohesive response by a process that composes the final message.

Getting hands-on! 👋

Part of Proko's Anatomy of the Human Body course. Big, thorough, and rich with material.

🤵

This one was a bit of a tongue twister!

🚠

Never knew that this is quite the famous melody before studying it!

Studying the gun show 💪

Part of Proko's Anatomy of the Human Body course. Big, thorough, and rich with material.

🏔️

One of the first Python lines I learned was opening files using the with keyword:

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

# Automatically close file after the above block
print("File closed")

The neat thing is that you can use the file within with and it will automatically close down once exiting the block. A simple way to teach an approach that's just as useful with db connections and other clients.

Today I happened upon @contextmanager, an elegent way to implement this type of behavior with your own classes and methods:

@contextmanager
def datastore():
    try:
        client = ds.connect()
        # Implement logic in the while block
        yield
        
    # proceed after the while block concludes
    finally:
        client.disconnect()

This simple decorator then allows usage in a with block:

with datastore() as dsc:
    dsc.get(...)

# Closed

That encapsulates up what otherwise would be a great deal of setup.

The elegance is, quite simply, very Pythonic!

RIP Annie Dillard. My personal memories with her work are with her collection The Abundance. Rich with sparkling verbal illustrations of a life lived with eyes wide open, it was a warm companion during my masters, read in between bike rides and practicing musical repertoire. Her essays were a struck match that lit the fire in my work and attention at that time, as undoubtedly they have for countless others.

Why are we reading, if not in hope of beauty laid bare, life heightened, and its deepest mystery probed? Can the writer isolate and vivify all in experience that most deeply engages his intellect and heart—and our own? Can the writer renew our hope for literary forms? Why are we reading if not in hope that the writer will magnify and dramatize our days, will illuminate and inspire us with wisdom, courage, and the possibility of meaning, and will press upon our minds the deepest mysteries, so we may feel again their majesty and power?

Taking a moment to express sincere appreciation for forums of all shapes and sizes!

It's easy to feel isolated in solitary practices — software, art, music, writing, etc. Something thats feels admittedly unnatural for me, having been involved in highly communal music making for ages. But, rejoice! We're all only a quick jump away from a portal to rooms where our kind of people are.

I just said that I’d enjoyed the book very much. And he said thank you. For a moment, it achieved the most basic and treasured of human communications: you are real, and therefore so am I.

-- Terry Pratchett on writing fan mail to J.R.R. Tolkien

I'll never forget my first forum experiences as a kid. The internet already seemed like a great resource, but being able to talk shop over Super Mario 64 was a game changer! At some point I threw the baby out with the bathwater, sticking strictly to lurking. But joining the conversation has helped buoy my creative efforts, connecting with and contributing to folks at a similar junction on the learning journey.

For anyone looking to make the best use of these channels, my humble advice: give small gifts.

Why gifts? When you center around a sense of contribution, there is bound to be someone on the other end who appreciates your efforts. From that frame of mind, you seek out people. Instead of waiting for people to come to you, you go out in search of someone you can impact. A brief, 2-3 turn exchange can be the most atomic version of that essential feeling of connection. Our identities are shaped in part by others, and you have cast a vote for an emerging identity through connection.

Why small? The internet is full of vast resources. It can be discouraging, feeling like you have to have a massive contribution to make any difference. Blockers can include limited experience, lack of audience, or the feeling that your perspective is not unique. So don't get bogged down. Look to make something just a little better.

"Attention is the rarest and purest form of generosity."

-- Simone Weil

Even a simple acknowledgement is enough. "I just started learning to draw, and this website has really helped", "I struggled with the same chord changes, you can do it", and even "Congrats, I see loads of progress from where you started to now!" You'd be surprised how a small interaction can be all it takes to get a positive feeling. And even if they aren't connected, all of those tiny efforts add up to feeling a part of something bigger.

The ocean is vast, but at that point — with your own droplet, however small — you have joined the body of water. After ages of taking in each tidal wave, you are now, for someone else, part of that massive resource of information, inspiration, and connection.

When initially digging into developing a RAG app, I was introduced to vector databases. It ultimately wasn't the tool for the job at the time, but I've since grown curious about their use and setup.

It's clear to see why these are useful in the context of AI integration (and beyond!) These are specifically designed to store highly dimensional data, where relationships can span many different attributes across entries.

Visualization is the fun part. Here is a simplified visual of points on a two dimensional graph. The points have embedded such that "France" and "Paris" are positioned closely together due to their relationship. They are closer than "France" is to "Berlin." Along another dimension, "Paris" is closer to "Berlin" than "France" because they share a relationship of being capitals of their countries.

That's two dimensions. We could easily leap to three dimensions. BUT! Vector databases have anywhere between 128 to over 4,000!! Unless you are a quantum space creature, that gets tricker to visualize. But, the example is similar. Items that have a similar relationship based on their point in this impossible graph are identified as highly related.

An example of a vector:

[
  0.0432, -0.1285, 0.8931, 0.1102, -0.5541, 0.0023, 0.4120, -0.0984,
  0.2215, -0.3342, 0.7612, 0.0543, -0.1120, 0.4901, -0.2234, 0.1089,
  ... 
  /* MANY more floating point values in between */
  ...
  -0.0124, 0.3312, -0.1543, 0.0891, 0.4412, -0.2210, 0.0045, -0.7712
]

Vector DB's work thanks to specialized models that are able to convert human recognizable data into vectors. Deciding on the embedding model is just as important as selecting your database, as they can be specialized for certain mediums and datasets.

When adding data, you pass an entry through the embedding model. That gets stored in the DB. When you query your db, that query itself is ran through the embedding model, compared, then returns a number of entries returned based on their proximity.

An additional consideration is what distance calculation algorithm you chose. This is something that's configured at collection creation, so it's worth some thought ahead of time. Options include:

• Euclidean: a common choice, the straight line distance between two points
• Cosine: Measure the cosine (angle of divergence of points)
• Inner Product: Factors in length and direction (distance from 0,0 and angle)

Usefulness of a vector database go beyond RAG. Semantic search is a maturation of keyword search that allows for finding results based on relevance of simple string matching. Multimodal search also allows queries to span across text, images, and audio. Plenty of flexible use cases.

Why it ultimately wasn't the fit for the project at the time: maintenance. It takes experimentation to find the right embedding approach, adjusting size of embeds and meta-data, and gathering user feedback on search effectiveness. Full Text Search in less complex cases may get you most of the way there without all of this. Still, contextual search can be combines with FTS and for the best of both for rich searching features.

🎠

🌹

Gabriel Zaid introduces Socrates' criticism of books in So Many Books:

Thanks to books, we know that Socrates distrusted books... Conversation depends on those who take part in it... In contrast, books are unfeeling monologues... In sum, intelligence, experience, and the creative life develop and are propagated through live speech, not dead letters.

Zaid argues against this push back on technological progress:

Intelligent conversations like those engaged in by Socrates and Phaedrus, who meet on the street, begin discussing a clever passage by Lycias on love, and go for a walk outside of Athens to debate its meaning, are only possible in an underdeveloped world, where productivity is low and there is plenty of free time. In the modern world, with everyone traveling by car, and leaving just enough time to get where they are going, Socrates and Phaedrus would never meet.

...[W]e no longer have the leisure of free afternoons in Athens. And the simulacrum of inspired life that exists in the great books seems more than a simulacrum: it seems like life, like latent inspiration waiting to be reclaimed. The dead text of Plato's Dialogues preserves the germ of [Socrates'] contagious freedom...

In his sign off to Socrates back in time, Zaid writes:

[I]n making us doubt the progress represented by writing, your criticisms have helped us to identify the true role of books, which is to continue our conversation by other means.

Studying abs! 💪⚾️

Part of Proko's Anatomy of the Human Body course. Big, thorough, and rich with material.