Creative LLM use cases - Illusion of competence, management exploits and other vulnerabilities
Plus: GraphRAG vs RAG explained, update on trading arenas and four new workflows
Flat Circle tracks creative use cases of LLMs in hedge funds. If you haven’t already, join hundreds of PMs, analysts and engineers reading each week:
Vulnerabilities in LLM investment research
AllianceBernstein Chief AI Officer warns about management altering phrasing to exploit LLMs earnings calls summarizers:
“Companies know we are measuring sentiment, so they adjust. They’ve started using more positive words, even with bad news.” That forces investors to evolve. “If I focus on the prepared remarks, sentiment scores are high. But in the Q&A, it’s much harder to control. That gives you a better read. “It’s a cat-and-mouse game,” he adds. “You have to keep improving to continue to generate alpha.”
New research on hidden text attacks in automated trading systems. A vulnerability when your LLM web search agent discovers text hidden from human readers:
My take: As investors increasingly leverage LLMs, the market will respond with new attempts to manipulate them. Point72 just posted a role for a GenAI Security Engineer. We are experimenting with a few approaches here, so if you’re interested in this problem, please reach out.
Another LLM risk: Illusion of competence
Two LP letters issue similar warnings on the increasing use of LLMs in investment research:
O’Keefe Steven - 4Q 2025 Investor Letter:
The more concerning dynamic is the illusion of competence. There is a risk that access to more contextually rich output leads to overconfidence in areas where the user lacks actual domain expertise. Nowhere is this more dangerous than in highly regulated, technically complex industries like healthcare or energy, where surface understanding is insufficient for investment decision-making. We expect many market participants to expand into unfamiliar sectors with misplaced confidence, armed with tools that enhance comprehension but not judgment
Upslope Capital - 4Q 2025 Investor Letter
AI is also everywhere – particularly on the desktops of buyside analysts and PMs. I suspect this technological shift is part of a not-so-virtuous cycle with the cultural shift towards gambling. A couple years ago legendary investor Stan Druckenmiller noted how he made a quick bet on Argentinian stocks with an assist from AI: ‘…do you want to hear how I invested in Argentina? It’s a funny story…I saw the speech in Davos and it was about 1:00 in the afternoon in my office. I dialed up Perplexity [AI] and I said, give me the five most liquid ADRs in Argentina…It gave me enough of a description that I follow the old Soros rule, invest and then investigate. I bought all of them. We did some work on them. I increased my positions and so far, it’s been great.’
More investor workflows
TMT / Energy investor lays out how he orchestrates sub-agents using the “Great Architect” framework (@TheValueist)
Head of AI at Manulife shares strategies to drive internal adoption (AI Street)
Former Capital Group partner and founder of new LLM driven hedge fund uses LLMs to analyze 2026 outlooks from the top asset managers (Linkedin)
We used a language model to extract thousands of individual statements and organize them by topic and time horizon. Similar ideas were grouped and weighted by how often they appeared across independent firms…On the environment, there was broad agreement….Firms disagreed on where returns are most likely to come from, how durable US market leadership will be, the timing and impact of policy easing, and how investable AI is at current valuations. These differences were not about facts. They reflected judgment…
Slides from new NYU Stern course on AI in Finance (Substack, Slides)
RAG vs GraphRAG
The platform funds are building AI teams to develop workflows for their PMs and analysts. One of the top required skills for these teams is advanced retrieval augmented generation (RAG) techniques such as GraphRAG - an open sourced framework developed by Microsoft Research. Example job post: E.g.,
Millennium: Senior GenAI Engineer - Advanced Rag
Enrichments and Knowledge Graph Construction: Move beyond flat vector search by building GraphRAG systems and advanced annotations such topics, keywords, sentiment, etc. You will extract entities (Companies, People, Metrics) and relationships from text to build a dynamic Knowledge Graph that captures the nuance of the financial markets and its temporal aspects.
Basic vector RAG, which you would experience by attaching files to ChatGPT or NotebookLM, searches documents for relevant excerpts and simply attaches them as context to your prompt. GraphRAG indexes your corpus into entities and relationships then uses that structure to synthesize answers that aren’t obvious from any single chunk. Funds are hiring for this knowledge graph retrieval frameworks since they see their edge buried in writeups, interviews, surveys, notes and other longform text and want to maximize second-level insights.
Update on trading arenas
Soon to be released Grok 4.2 is the only model making money trading weather prediction markets (PredictionArena.ai)
In another arena featuring Grok 4.2, Alpha Arena, xAI’s forthcoming model won as well. We cover trading arenas in more detail in an earlier post.
Follow for more investor LLM workflows
If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.