Why You Shouldn't Let AI Write Your Wave or Magic Quadrant Responses

(Even Though I Know You're Going To Try It Anyway)

AI can fill out forms. It just can't do the strategic thinking that moves you up and to the right.

Listen. I’m a big AI promoter. I vibecode and prompt engineer with the best of them. I wrote a custom GPT to play with evaluation response drafting while GPTs were still in beta!

But here's what I've learned after literally years of experimenting with every AI tool on the market for analyst evaluations: the results are mediocre. And mediocre simply isn't good enough for a major evaluation that matters to your business like a Gartner Magic Quadrant or a Forrester Wave.

Let me explain why, and then I'll tell you exactly how to use AI productively for these high-stakes evaluations.

It’s a Brain Science Problem

Here's something most people don't realize: when you ask someone to edit AI-generated text versus write from scratch, the human brain engages completely differently.

You might have seen the 2025 study from MIT looking at how brains engage when using AI to do research based writing, as compared to using search engines or just the human brain. The researchers used EEGs to understand which parts of the brain lit up under each circumstance, and when AI was used to do the writing, the writers showed lower overall mental engagement, with fewer parts of their brain engaged in the work, and lower recall of the final product. Yikes.

We also know that this is true when our brains are editing versus writing from scratch. This piece goes into great detail on the neuroscience, if you’re into that, but the TLDR is that our brains have a Generator mode, which uses more parts of the brain to make connections, retrieve ideas, and synthesize new meaning, and an Editor mode, which essentially shuts off the generator in order to evaluate the quality of the specific ideas in front of it. In other words, your editor brain is good at figuring out to improve the sentence that’s right there in front of it, but it’s not so great at spotting the idea that isn’t there at all. Double yikes.

This is very dangerous for Major Evaluation Questionnaires! When you feed a questionnaire into an LLM and use that draft as a starting point, your brain is in editor mode, not generator mode. You're checking if the sentences sound right, you’re not pulling from the deep well of customer stories, proof points, and strategic insights that actually differentiate your company.

You miss what only you know—because your brain never activated the retrieval systems that would surface those critical connections.

It’s also an LLM Problem

Beyond the brain science, there are fundamental technical limitations with how Large Language Models work that make them poorly suited for high-stakes differentiation.

The Statistical Average Problem. LLMs are trained on vast amounts of text and learn to predict the most statistically likely next word. When you ask an LLM to describe software capabilities, it's essentially generating "the average description of enterprise software that it has seen in its training data."

This is why every AI-generated RFP response sounds vaguely the same. The model has learned that enterprise software descriptions typically include phrases like "industry-leading," "cutting-edge," "empower teams," and "data-driven decisions." It doesn't know that these phrases are meaningless—it just knows they frequently appear together in this context.

When ChatGPT writes "Our innovative platform leverages cutting-edge artificial intelligence to deliver best-in-class forecasting capabilities that empower revenue teams to make data-driven decisions," it's not describing your product. It's generating the statistical average of how companies describe forecasting software. Your actual differentiation - the specific architectural choices you made, the particular customer problem you solved uniquely - gets smoothed away into generic platitudes.

Hallucination. LLMs don't actually "know" anything - they predict plausible-sounding text based on patterns. When asked about your integrations, the model might confidently state you integrate with Workday because "enterprise software companies often integrate with Workday" even if yours doesn't. That’s a big problem when it’s buried on sheet 7 of 12, in row 205, and you just lied to Gartner, which can disqualify you from inclusion entirely. You can manage this one (see below), but it’s a real risk.

LLMs answer tactically, not strategically. This one is the killer. AI will write responses that answer the question literally but miss the strategic positioning opportunity. When Gartner asks "How do you handle data governance?" they're not just asking about features - they're evaluating whether you understand the strategic importance of governance, how it connects to broader enterprise architecture, whether you're thinking ahead to regulatory trends.

Properly managed (see below), an LLM can probably handle the feature checklist. But it will gloss straight over the strategic perspective behind the checkboxes, leaving your scores possibly up, but definitely not to the right.

What about Sales’ RFP Automation Software?

I've been on both sides of RFP Automation Software. As a sales engineering leader, I evaluated, purchased, and later advised RFP automation platforms. For sales RFPs where you're responding to procurement questionnaires about pricing, technical specs, and compliance checkboxes, these platforms are game-changers. They cut response time dramatically, ensure consistency across your sales team, and maintain a clean content library. We all joke in sales that no one is reading beyond the RFP cover letter and scope anyway. That’s a perfect use case for automation.

But analyst evaluations aren't sales RFPs.

Here's the fundamental difference: A procurement RFP is asking "Do you meet our requirements?" The questions are largely binary or factual. "Do you integrate with Salesforce?" "What's your uptime SLA?" "Are you SOC 2 compliant?" For these, a well-maintained content library with AI-assisted RAG retrieval works beautifully.

An analyst evaluation is actually asking "How do you think about this problem, and why does your approach matter?" and “What is your likelihood of continuing to win and drive the market vision and direction?” The underlying questions sound factual but they're actually evaluating strategic positioning. When Gartner asks about your data governance approach, they're not just checking a compliance box—they're assessing whether you understand the shift from compliance-driven to business-value-driven governance, how you're thinking about emerging regulatory frameworks, and whether your roadmap reflects where the market is heading.

RFP automation tools fail for Major Evaluations for the same reason general LLM approaches do - they’re all facts, no strategy, and the underlying technology makes everyone sound the same. By all means, use these platforms to manage your content library and track your workflow. Just don't let them write your strategic responses. Exactly the wrong tool for this particular job.

How To Actually Use AI For Analyst Evaluations

Here's my real workflow after years of trial and error:

1. You must do the thinking. Sorry, I know. There’s no skipping this one though. The human has to do the strategic outlining because only you know what's important and resonant. AI can't tell you which three proof points will land with this particular analyst about this particular question, or how exactly our articulation of vision should shift versus last year's questionnaire. That requires judgment. Human judgement.

2. Bullet out key points + proof. Once you know what you want to say, write your response in bullet points with your actual proof points:

• "Reduced implementation time 60% (Fortune 500 financial services, 12K employees, 90 days)"

• "3 regulatory frameworks supported: GDPR, CCPA; added HIPAA this year because of our new strategic focus on healthcare”

• Critical to emphasize our strategic shift from X to Y when we talk about roadmap items A, B, and C.

3. Constrain the AI's context to prevent hallucination. Before you ask AI to help with prose, you must eliminate hallucination risk. The key to doing this is a technique called RAG (Retrieval-Augmented Generation).

Here's how it works: instead of letting the AI generate answers from its general training data (where it makes statistically plausible guesses), RAG forces the AI to pull information only from specific documents you provide. Think of it as giving the AI a closed-book exam versus an open-book exam where you control exactly which books are on the table.

Upload key product docs, recent case studies, customer testimonials, past successful responses, or validated RFP content to a tool like NotebookLM. Now when you ask the AI to draft prose, it can only reference facts from your curated materials - your actual product capabilities, your real customer quotes, your verified metrics. It can't make up an integration that doesn't exist or claim a capability you don't have, because that information simply isn't in its allowed source documents.

This doesn't eliminate the need for human review (AI might still combine information in misleading ways), but it eliminates the category of "completely fabricated" hallucinations that can disqualify you from an evaluation.

4. Let AI expand bullets to prose. NOW, with the human brain’s Generator mode fully tapped and hallucination risks managed, you can ask AI: "Turn these bullets into 1500 characters of compelling prose. Be specific and concrete, not generic. Prioritize brevity. Focus on differentiation and remove all marketing language." This works because you've done the hard cognitive work - now you’re using AI as a tool to assist in your strategy - it’s a decent writer if you tell it what and how to write.

5. Use AI for editing down. AI is also excellent at condensing text to character limits, but it’s less good at prioritizing how to best use your space. Ask: "I need to cut this to 1500 characters. Give me suggestions with character counts." Then YOU do the final selection and any tweaks in the context of your overall strategy. Don't give AI the keyboard directly for implementation. (Also keep in mind that some LLMs are better than others at the actual counting part - check the work here and you may need to tap into additional tools or skills to get it right depending on where you’re working)

6. Have AI check completeness. After you've written everything, ask: "Did I answer every single question with specific proof? Flag anywhere I was vague or used generic language, as well as opportunities for additional proof."

7. Use AI for competitive triangulation. Ask: "Based on publicly available data, how might [Competitor X] and [Competitor Y] have answered this question?" This helps you understand the field and identify where your answer needs to be more differentiated.

What Actually Works

After advising on dozens of Waves and Magic Quadrants, here's what gets strong results, AI-assisted or the old-fashioned way:

Start with your strategic narrative. What's the one thing you want the analyst to remember about your approach? Write that thesis first. Every response should ladder back to it.

Lead with proof. "We reduced implementation time by 60%" is better than "Our streamlined onboarding process delivers exceptional time-to-value." Data beats adjectives every time.

Tell customer stories. "A Fortune 500 financial services company implemented our platform across 12,000 employees in 90 days" is more compelling than "We support enterprise-scale deployments."

Connect to trends. Show you understand where the market is going. "As organizations shift from reactive to predictive customer engagement..." signals you're thinking strategically, not just listing features.

Be honest about trade-offs. "Our platform prioritizes ease-of-use for business users over extensive customization options" shows maturity. Analysts know every platform makes choices. Pretending you're perfect at everything reads as naïve.

The Bottom Line

I know you're going to experiment with AI for these responses. You’d be crazy not to. The technology is remarkable and getting better every day. I’m going to keep experimenting too!

But for now and for the foreseeable future, my emphatic advice is best summed up as “GPTs are for 3s.” If you’re using AI to do the initial thinking and positioning for your response, with a human acting mainly as an editor, you’re likely falling into the cognitive traps above. And you simply can’t afford that in an exercise this important.

On the other hand…the more vendors who fall into that trap, the more my clients will stand out, so… have at it :)

Up, and to the right!

Elena