Meta-Prompt for Better API Integration

• Experiment 1: Verifying Statements Using Meta-Prompt in ChatGPT
• Experiment 2: Visualizing Hacker News from the CLI
• Experiment 3: Building a Simple RAG System with JSON Storage
• Experiment 4: Building an App Factory with Meta-Prompt
• General Observations
• Insights from Using Different LLMs

URL Source: https://jina.ai/news/meta-prompt-for-better-jina-api-integration-and-codegen/

We recently published Meta-Prompt, a single text file that outlines all of our API specifications. You can think of it as documentation for LLMs, and use it to automatically generate integrations of our APIs including Reader, Embeddings, Reranker, and more.

0:00

/1:44

It’s as simple as copying and pasting our prompt into ChatGPT/Claude, or piping it into the llm command as a system prompt, then adding your own prompt to specify what you want to build (which we do below). It’s great if you want to use LLMs to quickly build apps that scrape the web, work with embeddings, or even full-blown RAG systems. All that with minimal hallucinations.

Let’s say you want to use an LLM to generate code that uses Jina’s APIs. Let’s ask GPT-4o to do just that:

0:00

/0:27

Looks good, right? It’s got the from jina import Client and everything.

One small problem: The Jina package is in maintenance mode, and it is not the way to access our APIs. Even if you do install the Jina package, the generated program will crash when you try to run it:

0:00

/0:21

So what? We can just ask GPT to search the web for Jina’s APIs, right? Here’s what we get:

0:00

/1:14

However, if you look at the code it doesn’t use all of the relevant Jina APIs. It very clearly didn’t find out that Reader is a thing, instead making us install BeautifulSoup to do the scraping. And, even when it could (supposedly) do the scraping with BeautifulSoup, it didn’t accurately parse the response format for Jina Embeddings, leading to a crash:

0:00

/0:16

Yet, even if ChatGPT could do it properly by searching, many other LLMs (like Claude) don’t currently support web search, severely limiting your options.

This is where Meta-Prompt shines. With Meta-Prompt, you can load all the context and specifications of Jina’s APIs into the LLM. This means the LLM can generate code that leverages Jina’s APIs directly, without hallucinations or unnecessary workarounds, giving you code that works the first time.

💡

Okay, usually the first time. LLMs can be unpredictable, but as you can see below, things went well in our experiments.

To put the Meta-Prompt through its paces, we ran a few experiments and evaluated the results. Unless otherwise specified, we used Claude-3.5-Sonnet as the LLM.

For all experiments, we specified relevant API keys (like JINA_API_KEY and ANTHROPIC_API_KEY) as environment variables before running the generated code.

#Experiment 1: Verifying Statements Using Meta-Prompt in ChatGPT

We’re writing this just after the US elections, where more disinformation than ever was flying around. How can we separate the signal from the noise in our feeds, and get just the good stuff with none of the lies?

Let’s say we want to check whether a new UK law is accurately reported on BBC.com, specifically the claim:

“The UK government has announced a new law that will require social media companies to verify the age of their users.”

We can copy-paste the Meta-Prompt into ChatGPT, then type our own prompt to generate the code to do that, like:

0:00

/0:35

We can then run that with node grounding.js (after installing any prerequisite packages like axios). We get output like this, showing that the claim is true, along with sources:

0:00

/0:04

#Experiment 2: Visualizing Hacker News from the CLI

If you’re more of a command line warrior, you can use Meta-Prompt from the CLI via cURL. First, you’ll need to install the llm Python package:

And then the Claude-3 plugin:

For the last stage of setup, specify your Anthropic API key:

Now, let’s write a prompt to visualize every sentence from the Hacker News front page:

We can pipe this into the llm command with:

0:00

/0:24

If we extract and and run the generated code, we get something like this:

0:00

/0:38

💡

One current limitation (though I’m sure with some extra coding from the user there’s a way around it) is that you’ll need to install requirements manually. No requirements.txt is generated. In this case we needed UMAP and Matplotlib, though your mileage may vary.

#Experiment 3: Building a Simple RAG System with JSON Storage

To push things even farther, let’s create a simple RAG system. In my spare time I’m learning SolidPython so we’ll use the repo and wiki as a knowledge base. To keep things simple, we won’t use a database, but rather just store the data in a JSON file.

Here’s the prompt, stored in the file prompt.txt:

As you can see, we can give the LLM additional tools by specifying them in the prompt. Without this, Claude often hallucinates a less optimal (or even broken) way to add the LLM to the RAG system.

Since this is a very long prompt (with plenty of punctuation that may break any pipe we run it in), we’ll use the text $(cat prompt.txt) rather than the prompt itself when we run our command:

0:00

/0:34

Phew! That’s a lot of output. But (like with the Hacker News example) it’s a pain in the neck to extract and run the code from that big blob of text. Of course, there’s no problem that can’t be solved by just throwing more LLM at it, right? So let’s add another prompt to “de-blob” the original output:

Now we add that to our command pipeline and run it:

💡

Since we’re using > app.py at the end of our command to direct all output into a file, there’s nothing to show in a video.

We can then run that app with python app.py and we get our RAG program. As you can see, it can answer questions and maintain a working memory:

0:00

/0:34

💡

The first run of this took a little longer, since it had to segment and encode all the data. For subsequent runs it loaded that from a JSON file to save time and cost.

#Experiment 4: Building an App Factory with Meta-Prompt

Now that we can generate scripts and apps non-interactively, we can easily automate an “app factory” - a script that iterates over prompts and produces Python scripts as output. You can get the app factory script in a GitHub gist for now:

App Factory with Jina AI Meta-Prompt App Factory with Jina AI Meta-Prompt. GitHub Gist: instantly share code, notes, and snippets. 262588213843476

What it does, in short, is:

• Iterate through the prompts directory which contains (you guessed it) prompt files.
• Pass the Meta-Prompt and each prompt text to Claude-3.5-Sonnet (via llm).
• Take the output and pass that to Claude again, this time with the prompt telling it to just leave the code.
• Write that to a file in the apps directory.

We’d show a demo, but there’s not much to see. It just logs which prompt filename it’s working on, and otherwise operates silently with no interesting output to the screen.

💡

Testing the apps it generates is another matter, one that I can’t solve off the top of my head. In our experience, we often specify the data we want to use in our prompts, usually by passing an external URL to download with Reader. Yet sometimes the LLM hallucinates mock data, and the script runs without obvious issues — it just “lies” about what it’s doing.

To take the app factory to the next level, you could go full Factorio and write another script to generate app ideas and from there generate prompts to feed into the factory. We haven’t done that yet, but we leave it as an exercise for you, the reader.

We learned a lot from using Meta-Prompt, both about what to put in our own prompts and how different LLMs generate different output.

#General Observations

• API Specialization: Using task-specific APIs (e.g., Google Books for book-related queries) ensures more consistent results than general-purpose search APIs, which can reduce token usage and improve reliability.
• Custom Prompts for Reusability: For non-interactive setups, saving prompts as .txt files and piping them into the CLI enables efficient code-only outputs without extra explanatory text cluttering things up.
• Structured Output: Storing outputs (usually in JSON format) and reloading them as needed saves tokens and streamlines tasks like generating embeddings, where token usage can be expensive.

#Insights from Using Different LLMs

GPT

• Prompt Retention Issues: GPT-4o sometimes loses details with lengthy instructions, leading to issues when it “forgets” key elements mid-discussion. This leads to a lot of frustration when you have to remind it of simple things.
• API Integration Challenges: In cases like integrating Milvus Lite with jina-embeddings-v3, even when we provide the Milvus Lite API instructions, GPT-4o fails completely and repeatedly, generating code that creates databases that lack the embeddings that the code just generated, making semantic search applications impossible.

Claude

• Code Output Limitations: Claude-3.5 often produces scripts that appear complete but contain silent issues, like missing error handling or failing to account for missing API keys. Additionally, it sometimes falls back on pre-set examples rather than generating responses tailored to specific instructions.
• Silent Output: With LLM-generated code it really helps to have some logging of what’s happening behind the scenes when you run the program, just to make sure the model didn’t mess things up. Unless you directly specify to do so, apps created with Claude will often run silently, leaving you with no clue what’s happening.
• Interaction with CLI: You need to clearly specify that CLI commands are CLI commands. If you tell Claude it can use the llm command, often it will try to call a Python llm() function which doesn’t exist.
• Claude 3.5-Sonnet Is the Way to Go: Claude-3.5-Haiku also seemed to work okay in initial tests, but Opus and Sonnet-3 just summarize the Jina API instructions, without taking into account the user prompt.

#Conclusion

Using Meta-Prompt provides new ways to integrate Jina’s APIs with LLMs, allowing you to run experiments and build apps that work on the first try. No more crashes, missed API connections, or hallucinated functions — Meta-Prompt ensures the code generated is accurate and functional right out of the gate. Whether you’re verifying statements, generating embeddings, building a lightweight RAG system, or automating app creation, Meta-Prompt transforms natural language instructions into actionable, correct code, bypassing the typical back and forth with an LLM to get things that actually work.

Whether you’re copying Meta-Prompt into ChatGPT or using it with a custom LLM command, it offers a straightforward, reliable way to leverage Jina’s capabilities. Our experiments and insights show Meta-Prompt as a solid tool for robust integration into your projects.

If you’re ready to explore what Meta-Prompt can do, head to docs.jina.ai for the latest documentation and resources.