Six AIs. One Prompt. Who Nailed It?
Benchmarking AI-Generated Investment Proposals for Lovable.dev
šĀ Hi, Iām Andre and welcome to my newsletter Data-Driven VC which is all about becoming a better investor with Data & AI. Join 33,220 thought leaders from VCs like a16z, Accel, Index, Sequoia, and more to understand how startup investing becomes more data-driven, why it matters, and what it means for you.
Brought to you by Harmonic ā The Startup Discovery Engine
Did a former portfolio employee just start something new? With the new alert flow in Harmonic, youāll never miss an opportunity to engage with the best startups at the right time. Join the likes of NEA, Bessemer, and Redpoint, and get the hottest startups straight to your inbox or Slack!
Hi there!
On Sunday, we published our top 10 prompts for startup sourcing here. Few weeks before, we published our most powerful prompt for startup evaluation here. And last year, we published a prompt for extracting names from startup landscapes here.
Yes, we love prompts and will publish many more in the future. Interestingly, every time we share our newest prompts, we receive a flood of responsesāall asking the same thing: Which AI model or chat provider are you using? Is it ChatGPT, Gemini, Claude, Perplexity, or another one?
Isnāt it interesting how much we focus on creating new and refining existing prompts, yet donāt talk about their performance across models and providers all that much?
If we think of prompts as a recipe, spelling out what you want to cook, and the model/chat provider as the chef who reads the recipe, selects the ingredients, adds his own style and taste, and eventually delivers the dish - shouldnāt we focus as much on the chef as we do on the recipe?
In todayās episode, Iāll take our startup evaluation prompt to generate investment memos for an early-stage company everyone talks about these days - lovable.dev - and put it into the six most frequently used chat providers / models among investors. I measure and compare criteria like the time to generate the results, words, number of sources included, coverage and accuracy of information, adherence to structural requirements, and more, and put all findings into a benchmarking table at the end of this episode, drawing a clear conclusion for investor usage.
Letās jump in!
The Prompt
Can you compile a deep dive on [company]?
I'm interested in a report covering the background and history of the company, including how it got started, what was its initial product-market fit, and how it has expanded over time. What are its products? Who uses their products? What roles, verticals, industries, or functions does it target? What's their ICP? What's the average contract value? What's their business model? What's its go-to-market strategy?
What are the company's opportunities? Where is it growing? What is it doing around AI? How are these opportunities meaningfully unique or different from competitors? What are the risks in its business? Where are there threats? How is it responding to them?
What do users think of the company's products? Do they like it or do they not? Why is it sticky? What attracts customers to it?
What competitors does the company have? Focus on close competitors, not everyone. Include comparative metrics if available. Who are its threats and who is it threatening?
Does the company have traction? Can you surface any key metrics or KPIs? What can you tell me about the company's financials? Is it generating revenue? Can you ballpark how much? How many customers does it have? Are there notable customers that they reference? What's the company's ARR? The company is private, so whatever data is available, whether it's publicly available or rumors or estimates.
What's the company's current headcount? How has that grown in the past few years? Where are the bulk of employees? Are they in office or remote? What can you tell me about the company's funding? How much has it raised? What Series funding are they at? When did they last raise money? Who did they raise money from? What valuation did they last raise at? What's the history of their valuations? What are their stated next plans, if any? Are they planning for an IPO?
I'm also interested in knowing about the founders and their backgrounds, as well as information on the current senior leadership, particularly if the current CEO is not the founder.
Ideally, there are some good profiles I can read about the company.
Make the report structured with headings and sections. The structure could be something like follows:
Executive Summary
Key Insights
Key Risks
Team Info
Problem & Market
Solution & Product
Competition
Business Model
Traction
Funding and Investors
Conclusion
The Competitors
Spoiler alert: Weāve surveyed hundreds of investors for our upcoming Data Driven VC Landscape 2025 report and asked which model and chat providers they use, among many other interesting aspects like budgets, tool stacks, and more.
Below is an extract from the unpublished report that serves as the foundation for our benchmarking. It shows that 75% of investors use ChatGPT, Perplexity, Claude, and Gemini.
While most chat providers offer access to different models such as GPT-4o, GPT 4.5, o3, o4 mini, and Deep Research in the case of OpenAI ChatGPT, majority of investors tend to use the āgreat for most questionsā suggestion and only few use the Deep Research functionality.
Therefore, weāll take above prompt and benchmark the outcomes across the four most frequently used chat providers, and where suitable across their most used model and the deep research functionality.
Please note that I reset memory to test all providers with the same test conditions.