Subscribe now

Custom datasets you can build with Claude Cowork

Flag discrepancies between earnings press releases and transcripts (InfoArb)

Yale finance professor creates a custom dataset of tariff exposure (Paul Goldsmith-Pinkham)

Extract supplier geographies from a 10K filing, determine transit corridors and query AIS vessel-tracking databases and ICEGATE customs registries to audit and monitor supply chains (Alan Shore)

Analyze webcam footage of a fab construction site, counting building levels based on crane heights, to test consensus for WFE spending (@lfg_cap)

Official Claude for Financial Services plugin, including skills like catalyst-calendar, model-update, morning-note and idea-generation (GitHub). Related, a trader published skills around market analysis, technical charting, economic calendars and screeners (GitHub)

How to organize your second brain

Lot of discussion this week re: building your own personal knowledge base - a system of markdown files containing your investment philosophy, research, patterns, skills, notes, etc - enabling your Claude Cowork or OpenClaw to deliver exactly what you need.

It’s all about how you organize it. Models perform worse if you feed it too much information, so you want to arrange your knowledge base such that your system pulls in exactly the context it needs to make a given decision, while not missing anything important nor diluting itself with unnecessary info.

A few approaches:

• OpenAI cofounder Andrej Karpathy published LLM Wiki, which organizes your knowledge into a personal wikipedia, with agents that maintain and crosslink across concept pages. Focus is getting your system to reflect your views on a given company, sector or theme (@karpathy, GitHub).

• CEO of YCombinator published GBrain, a framework optimized for systems like OpenClaw where the focus is performing as many different actions as possible (GitHub)

• The 5th Element star Milla Jovovich published MemPalace, a framework optimized for retrieval of verbatim documents (GitHub)

• A former sellsider recommends pulling in four main concepts for each decision: (i) the most important events, (ii) the most recent events, (iii) retrieved related events, (iv) curated long term memories. This attempts to model how the human brain compiles knowledge to make a decision (@HenryChien4)

My take: the right organizational scheme depends on what you’re optimizing for and your ability to catch mistakes. GBrain is great for personal AI assistants. LLM Wiki best reflects your views and experience. MemPalace optimizes for accurately returning the right source documents. The more gold standard examples you can test against, the easier it is to experiment with organizational schemes.

New papers and benchmarks

New paper on “LLM herding” shows including buy/sell ratings significantly influences model responses, even if subsequent analysis contradicts the rating.

My take: For investor agents, sellside research can be a prompt injection: managing what insights are fed to the decision models is critical. This can be a problem when you enable a model with open web search, which can pick up sellside research headlines (OpenReview)

New agent benchmark tests ability to build LBO, lender and DCF models: GPT 5.4 outperforms Claude and Gemini models but still lags human experts (arXiv.org). Related, an investor substack tests the leading Claude models on assessing business quality and recommends Sonnet 4.6 w/ thinking mode enabled.

Updates from AI trading arenas

These are public experiments where agents make trading decisions. We track every arena here:

New arena focused on clinical trial predictions (launch post, website). Continues a trend of vertical trading arenas.

GLM 5, a model by publicly traded Chinese lab Z.ai, has been making money on Prediction Arena (PredictionArena.ai). GLM models score well on long-horizon software engineering tasks. The agent powered by GLM 5 appears to prefer betting on Kalshi markets with asymmetric payoffs (ie that trade around 1 cent for a 1 dollar payoff at time of entry.

Prediction Arena published a paper (arXiv.org). Takeaways are that models perform better when they are able to select from a larger universe of markets and that there’s diminishing returns to incremental research.

Follow for more creative AI use cases

If you would like to discuss incorporating agents into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Subscribe now

Which model works best for which workflow?

LLM benchmarks score models against real world tasks designed by experts. These popular ones cover various investment research workflows: accounting (DualEntry), excel (SpreadsheetBench), SEC filings (Vals), finance reasoning (PRBench), event forecasting (Prediction Arena), thought partnership (BullshitBench) and OpenClaw (PinchBench):

We’re developing investment research benchmarks for agentic systems (ie not just the models but the prompts, tools, etc), which can drive meaningfully different outcomes. If you’re interested in collaborating, please reach out.

Creative LLM use cases

Prompt and methodology for identifying ‘moving targets’ - flagging whenever management teams change what metrics they highlight (arXiv.org, AI Street)

YC partner releases open source, AI-native email inbox (@agupta, GitHub). Makes it easier to bring internal and external data into your email flow. Alternatively, new YC startup AgentMail is an easy way to give your agents their own email account

Coatue-backed long-only uses AI to condense overnight research into custom podcasts (Advisor Perspectives)

Eve also scours the disclosures of more than 13,000 companies; listens to podcasts; scrutinizes social media posts; summarizes the news; and, each morning, generates a podcast for Kishore to listen to while he drives to work.

BullshitBench: When using LLM as a thought partner, does it push back when the premise of your question is flawed? (Benchmark, @petergostev, thanks to Mark Ainsworth). Claude Opus 4.6 lets only 2% of flawed questions through, while GPT 5.4 lets 16% through.

Reddit post on using Claude Code to analyze retailers using satellite data (r/ClaudeCode)

Former hedge fund PM shares prompt for Perplexity Computer to create a guidance credibility analysis (@FundamentalEdge)

AllianceBernstein AI head uses LLMs to fill in missing time series data (post)

Researchers frequently incorporate historical data into their analysis, and data may be unavailable for certain time periods—the dreaded broken time series. Rather than throw away the series, analysts can use AI models to create fill-in data that, in the human expert’s judgement, may be sensible given the context.

TMT investor shares favorite LLM use cases (@lfg_cap)

Option scenario pricing
Portfolio optimiser
Factor / thematic correlation / analysis / alerter
Getting from 0 to 95% on new sub sector
Qualitative relative business quality analysis (based on structured questionnaires)
New ideas / needle in haystack (parsing through 1000s of emails and twitter messages) for differentiated / contrarian views on different industries / geopolitics etc

AlphaSense launches custom agents to run prompts on a schedule, and custom AI expert calls to automatically interview a panel of experts (press release)

Interviewing yourself

Several good pieces this week about documenting your personal workflow into prompts and markdown files.

Example prompt to launch voice interview session - turning open ended discussion into detailed instructions (Ben’s Bites)

More interview prompts that extract context from yourself and make your agents more effective (@Shpigford)

Lawyer discusses how he embeds his own personal frameworks into skill files (@zackbshapiro). He also says it’s impossible to infer someone’s process simply by looking at their outputs, says it needs to come directly from the person:

I’ve had people try to reverse-engineer my Claude skills by studying my outputs, using AI to analyze what I produce and reconstruct the instructions that generated it. They never get close...what my skills actually contain is not a description of what the output should look like. It’s a detailed operating procedure for how the output gets created: decision trees, analytical frameworks, sequencing logic, edge-case handling, judgment calls about when to be aggressive and when to hold back. You can’t see any of that by studying the finished product...A finished contract shows you what a great lawyer decided. It doesn’t show you how she decided it, what she considered and rejected, or the order in which she worked through the issues. The process is invisible in the product. My skills encode the process.

Follow for more investor LLM workflows

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

We’ve been collecting hedge fund GenAI job posts over the past month to identify creative LLM use cases, and thought we would analyze them to see what else we could learn:

Funds with the most job listings

Most common technologies

Key takeaways from the data

• Average annual salary in $212.5K, with a range from $150K to $300K

• Teams prefer vendors for models and storage, open source for everything else

• OpenAI mentioned twice as much as Anthropic

• AWS is the most popular hyperscaler

• While Balyasny, Millennium, Point72 dominate hiring, we did not find evidence of open fundamental-focused GenAI roles at Citadel

If you’d like the full dataset, please reach out

Creative LLM use cases

Bloomberg Businessweek used Claude to review 1,500 hours of livestream footage of influencers playing Stake, a crypto gambling site, to reveal the company was rigging bets (Bloomberg, thanks to Byrne Hobart)

Reporters used Anthropic’s Claude, a large language model, to analyze footage frame by frame and determine the balance, bet and games being played during livestreams

An investor vibe coded an LLM that identifies past CEO claims and whether they bore out into a “say-do” score for every management team (@dahu7744)

Good thread on the process of iterating with Claude Code until it can correctly generate excel models (@tomasrice_au). Related, OpenAI launched ChatGPT for Excel (OpenAI)

Guide to using Claude Code / Cowork for investment research by CEO of Daloopa (@oneThomasli)

JPAM hiring data scientist to analyze sellside notes and news to identify trending and emerging themes (LinkedIn)

Case study on Jefferies equity research internal alternative data LLM chat (Databricks)

This multi-source response surfaces analytical angles that analysts may not have explicitly requested, enabling corroboration across independent sources.

Roundup of internal LLM tools at BAM, Citadel, Point72, etc (@TheValueist)

Merger arb

A merger-arb ETF manager describes his LLM system (@JulianKlymochko):

For example, we have Equity Research Analyst agent that writes an initiating coverage report on the merger target. Next, we have our Legal M&A Analyst agent, that summarizes merger agreements and proxy statements, and our Antitrust Analyst agent, that analyzes market shares along with the DOJ / FTC, EC, China SAMR, and other global regulators would view the deal, in addition to ascribing probabilities of antitrust clearance / 2nd requests / merger challenges.

How Balyasny built its merger arb bot (OpenAI):

early feedback from merger arbitrage teams revealed that agents needed to continuously re-evaluate deal probabilities as new filings or press releases came in. The Balyasny team quickly extended agent planning capabilities and tool access, replacing a slow, manual workflow with real-time probabilistic monitoring

Expert network MCP

Third Bridge launched an MCP for its transcript library (press release). AlphaSense/Tegus offers one as well but GLG and Guidepoint currently do not have public MCP or API endpoints.

My take: seems like all the major transcript libraries will offer MCP / API access soon. Demand for transcript content should meaningfully increase as agents aren’t limited by cognitive ability to process transcripts. However, I expect pricing to fall even more as the search cost across providers will go down. Networks will compete on their ability to access experts exclusively - and unique experts will benefit accordingly. I also wouldn’t be surprised to see consolidation.

Follow for more investor LLM workflows

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Models deliver Beta, Humans deliver Alpha

Harvard study shows AI can predict 71% of mutual fund trading decisions, but the remaining 29% of trades generate the most alpha (Matt Levine via Justina Lee)

Howard Marks describes the human x-factor (Oaktree LP Letter)

Great investors…have to be strong exactly where Claude admits AI might be weakest: in dealing with novel developments where there’s not enough prior experience for dependable patterns to have been compiled (and learned by AI during its training). They also have to make subjective decisions regarding qualitative factors and exercise taste and discernment. For instance, choosing the right counterparties has played an important part in Oaktree’s success. How will AI make judgments of that sort? And there’s something else: AI doesn’t have skin in the game. It doesn’t feel the weight of concentrated positions or the fear of capital loss. Its willingness to take risk might not be constrained by humans’ normal risk aversion. The best investors sense potential risk intuitively, and this contributes greatly to their success.

Especially when investors are dealing with new and untried products, CEOs, or industries, there can be few facts or analogous experiences, meaning we have to rely on “opinion or speculation.” Given the limitations discussed above on AI’s ability to tackle brand new situations, will its speculation about new things – as opposed to extrapolating historic patterns – be consistently superior to that of all humans? I believe there will continue to be human investors who are superior to AI, since I don’t think AI will be able to do an unbeatable job of these things.

My take: LLMs level the playing field in processing public information and increase the reward for proprietary research, personal relationships and experience

Creative LLM uses case

Hayden Capital vibecoded a pixel tracker for Applovin (APP) (LP Letter)

For example, I recently “vibe-coded” our own Applovin Axon Pixel Tracker, to track Applovin’s new ecommerce push (LINK). The program scans the top 100,000 ecommerce websites, and whether they’ve adopted Applovin’s ecommerce tools – useful for us tracking adoption in real time. I did this all with Claude Code, in just a couple hours over a weekend, and runs on Amazon’s AWS.

Norges Bank uses Claude to monitor foreign language media for ESG issues in their portfolio (CNBC)

“Often, this information has not been captured in international media coverage or data vendor alerts…In multiple instances, we identified and sold these investments before the broader market reacted to the risks, avoiding potential losses.” NBIM said using AI this way had been particularly valuable for researching smaller companies in emerging markets, where news about the firm may be limited to small media outlets in local languages.

VC shares which of his workflows are mostly code vs mostly LLM driven (@ttungaz)

BlackRock and Schonfeld are hiring AI engineers for post-trade operations (BlackRock, Schonfeld)

AWS Bedrock post on using their graphRAG workflow to analyze 10-K filings and identify shared risk relationships across the S&P 100 (AWS)

Rutgers finance professor shares 8 tips for using OpenAI Batch API (50% cheaper) for large scale transcript analysis (LinkedIn)

Interesting paper

Compares LLM use in stock pitches on Seeking Alpha vs r/WallStreetBets. AI drives better returns in the former, more professional community, while on r/WallStreetBets AI drives abnormal trading and lottery outcomes. Interesting take on how LLMs may impact retail and institutional trading (NBER)

New tools

Review of Claude Cowork, a non-technical desktop version of Claude Code (Buyside AI Reviews). When asked to turn a lender presentation into a leveraged loan screening model, the tool made several key errors:

I do think people underestimate (i) the amount of time it takes to check/correct output and (ii) the willingness of senior folks to actually do the checking. And given the black box nature of LLM reasoning, the checking needs may not scale down as fast as AI capabilities scale up.

More: Claude launches Cowork and plugins for finance (announcement)

Checkmate, another AI expert call service, launches (CheckmateResearch.ai). Other services where LLMs source and conduct interviews include: AlphaSense, Guidepoint, NewtonX, Qualitate, Ribbon, Synquery. Interesting reddit thread where experts debate the future of this format: “Please boycott ai mod calls”

FirstDraftResearch, which looks like a “cursor for public market investors,” announces private beta (@atelicinvest)

Bloomberg launches <ASKB> - conversational AI interface (Bloomberg)

Follow for more investor LLM workflows

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

OpenClaw

OpenClaw (fka ClawdBot, Moltbot) allows a computer to run an LLM system in an “always-on” way and interact with almost anything - making it feel more like a proactive analyst that you can train and work with via slack, email, signal, etc.

How to build an OpenClaw investment research analyst (Saulius)

OpenClaw changes the equation. It is an open-source, self-hosted AI agent platform that runs persistently on your machine, connects to every messaging platform you use, and has access to a full suite of tools -- file operations, web search, browser control, code execution, and long-term memory. It does not just answer questions. It reads research reports, builds financial models, monitors markets around the clock, learns from its own experience, and proactively alerts you when something needs your attention.

Institutional Investor warns funds not to build their own OpenClaw (Institutional Investor)

Data scientist at real estate asset manager analyzes 120 data center projects by talking to OpenClaw via WhatsApp (Infrastructure Research)

Two funds already hiring engineers to build with OpenClaw (job post, job post)

Upwork post by small event driven fund requesting an OpenClaw screening system (job post)

We are actively exploring how to safely use OpenClaw for investor workflows, please reply to this email to discuss.

Creative LLM workflows

Founder of AI native hedge fund details how he builds with Claude Code (@thomasrice_au)

If it's front end or something we interact with, I start by describing what I need to do and initial ideas for interface. I'll then generate 30 mockups (10 each from GPT, Claude, Kimi), asking them to make each quite different, and to make each one an isolated html file with embedded JS and CSS. I'll then go through the 30, dismiss most, keep a few, then keep iterating until it feels right for what I want it to do.

LLM workflow to analyze new 13F holdings and generate a list of relevant positions and reason each fund likely owns them (@FundamentalEdge)

Bonus: New feature from Polymarket allows anyone to pledge rewards to encourage more research (@polymarket thanks to @adrien_nav). Currently the markets with the largest sponsored rewards are for whether the US strikes Iran, Fed decisions and the S&P.

Interesting jobs

Point72 hiring GenAI engineer focused on alternative data (job post)

Updates from trading arenas

In Prediction Arena, all models turned negative this week (predictionarena.ai)

My take: it appears the models made a few concentrated bets that were impacted by extreme weather and a surprising unemployment report. Like a lot of the other trading models, they often win for a period but are actually selling vol. Read more on AI trading arenas here.

Research paper outlines findings from nine-month LLM driven trading strategy. Findings include a 2.43 sharpe and skill at identifying longs but not shorts (arXiv.org)

Chamath Palihapitiya publishes paid post covering trading arenas (Substack)

Follow for more investor LLM workflows

Subscribe now

If you would like to discuss incorporating LLMs or OpenClaw into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Investor LLM workflows

Dan Loeb asked Claude which public companies it might disrupt (Third Point 4Q25 LP Letter)

A simple query into Claude’s chatbot: “Which companies is Anthropic capable of dislocating or disrupting?” yields some fascinating results and was in our view a fruitful source of hedges for our firm.

My take: the subtext here is that models are evolving into the source of conventional wisdom.

AI driven short seller Abelian Analysis analyzed hundreds of Youtube transcripts to assess the pricing environment for CVNA’s used vehicles (Short Report, Github)

Each transcript was analyzed by Claude Sonnet 4 using a prompt designed to separate market conditions from creator mood. This distinction is critical. A dealer complaining about thin margins is telling you margins are compressed — a bearish market signal. A flipper excited about “deals everywhere” is telling you inventory is high and prices are soft — also bearish. The LLM was instructed to ignore emotional spin and extract the underlying market reality across three categorical signals (inventory direction, demand strength, repossession activity), two continuous scores (bullish 0-100, bearish 0-100), and a sensationalism rating (1-10) that we use for quality control.

Investor from Altimeter Capital outlines two LLM workflows, including a PDF example from a “council of LLMs that rigorously debate topics with access to web search” (@_clarktang, thank you @realLigerCub)

An RIA PM shares his deep research prompt (@TedHZhang)

AI-native hedge fund, Minotaur Capital, used “Ralph Wiggum” style iterative research loop to determine that gaming stocks were oversold following the Genie 3 release (Minotaur January Letter)

We immediately spun up a research process using the iterative techniques we described in our December Quarterly. From a 127-word prompt asking for implications on the games industry, our AI system iteratively chose what to explore: value chain analysis, five-year scenarios with falsifiable signposts, unit economics ($/minute cost models), IP and licensing questions, and a winners/losers matrix across engines, platforms, and publishers. Over 50 iterations it built out each section, cited sources, and stress-tested its own conclusions.

Former Baupost investor Dave Plon shares an AI workflow around killing ideas faster: develop a list of non-negotiables (e.g., CEO compensation structure, guidance track record), and have the system eliminate every name in your coverage failing those non-negotiables. (Business Breakdowns, 12m 50s)

Former Maverick / DE Shaw / Citadel PM shares his prompt to analyze the technical setup (@FundamentalEdge, thank you @realLigerCub)

New paper analyzes the impact of LLM tools like ChatGPT on price reactions during earnings calls. Stocks don’t react faster, but at a greater magnitude after a delay due to model latency and transcription availability (Price Discovery Within Earnings Calls, thank you Justina Lee)

Cons to LLM investor workflows

An investor argues LLMs make it harder to build conviction (@evrgn11112231)

I view investment research as akin to the slow LLM training process. It’s not supposed to be fast. The goal is to ingest raw data over long periods of time to train your brain (the ultimate LLM) for instant recall and pattern matching later.

Former credit investor warns on using LLMs for initiation style reports, as they often miss key events that would materially alter the narrative (@BuysideAIReview). My take: the quality of an LLM workflow is only as good as its eval. Before asking for a deep dive, first build a list of key events, transactions, players, etc - then run deep research as a loop until it hits all the required items. See the “Ralph Wiggum” discussion above.

Interesting LLM tools

Former mega fund and credit hedge fund investor reviewing every buyside AI tool (Buyside AI Reviews)

Former hedge fund PM previews an insights feed built from earnings call transcripts (@atelicinvest)

New tool benchmarks the stock impact of short reports / short selling firms (ShortReportImpact)

YCombinator Request for Startup: AI-Native Hedge Funds

YCombinator just published its latest Request for Startups:

…the next Renaissance, Bridgewater, and D.E. Shaw's are going to be built on AI. The biggest funds in the world have been slow to adapt. I worked as a quant researcher at one of these funds, and when I asked compliance to let us use ChatGPT, I didn't even get a response. It made it clear to me that the hedge funds of the future won't just bolt AI onto their existing strategies. They'll use it to come up with entirely new ones. That's where the alpha is.

Now tracking every AI trading arena

AI trading arenas are public experiments where LLMs perform research and trade in a live environment. They are one way to track LLM progress in making investment decisions.

Our new page tracking every public arena is here: AI Trading Arenas

Key takeaways include: (i) the median model always loses money, (ii) newer frontier models outperform the older models, (iii) soon-to-be-released Grok 4.2 is undefeated, (iv) Claude has not yet won any trading arena.

Follow for more investor LLM workflows

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Vulnerabilities in LLM investment research

AllianceBernstein Chief AI Officer warns about management altering phrasing to exploit LLMs earnings calls summarizers:

“Companies know we are measuring sentiment, so they adjust. They’ve started using more positive words, even with bad news.” That forces investors to evolve. “If I focus on the prepared remarks, sentiment scores are high. But in the Q&A, it’s much harder to control. That gives you a better read. “It’s a cat-and-mouse game,” he adds. “You have to keep improving to continue to generate alpha.”

New research on hidden text attacks in automated trading systems. A vulnerability when your LLM web search agent discovers text hidden from human readers:

My take: As investors increasingly leverage LLMs, the market will respond with new attempts to manipulate them. Point72 just posted a role for a GenAI Security Engineer. We are experimenting with a few approaches here, so if you’re interested in this problem, please reach out.

Another LLM risk: Illusion of competence

Two LP letters issue similar warnings on the increasing use of LLMs in investment research:

O’Keefe Steven - 4Q 2025 Investor Letter:

The more concerning dynamic is the illusion of competence. There is a risk that access to more contextually rich output leads to overconfidence in areas where the user lacks actual domain expertise. Nowhere is this more dangerous than in highly regulated, technically complex industries like healthcare or energy, where surface understanding is insufficient for investment decision-making. We expect many market participants to expand into unfamiliar sectors with misplaced confidence, armed with tools that enhance comprehension but not judgment

Upslope Capital - 4Q 2025 Investor Letter

AI is also everywhere – particularly on the desktops of buyside analysts and PMs. I suspect this technological shift is part of a not-so-virtuous cycle with the cultural shift towards gambling. A couple years ago legendary investor Stan Druckenmiller noted how he made a quick bet on Argentinian stocks with an assist from AI: ‘…do you want to hear how I invested in Argentina? It’s a funny story…I saw the speech in Davos and it was about 1:00 in the afternoon in my office. I dialed up Perplexity [AI] and I said, give me the five most liquid ADRs in Argentina…It gave me enough of a description that I follow the old Soros rule, invest and then investigate. I bought all of them. We did some work on them. I increased my positions and so far, it’s been great.’

More investor workflows

TMT / Energy investor lays out how he orchestrates sub-agents using the “Great Architect” framework (@TheValueist)

Head of AI at Manulife shares strategies to drive internal adoption (AI Street)

Former Capital Group partner and founder of new LLM driven hedge fund uses LLMs to analyze 2026 outlooks from the top asset managers (Linkedin)

We used a language model to extract thousands of individual statements and organize them by topic and time horizon. Similar ideas were grouped and weighted by how often they appeared across independent firms…On the environment, there was broad agreement….Firms disagreed on where returns are most likely to come from, how durable US market leadership will be, the timing and impact of policy easing, and how investable AI is at current valuations. These differences were not about facts. They reflected judgment…

Slides from new NYU Stern course on AI in Finance (Substack, Slides)

RAG vs GraphRAG

The platform funds are building AI teams to develop workflows for their PMs and analysts. One of the top required skills for these teams is advanced retrieval augmented generation (RAG) techniques such as GraphRAG - an open sourced framework developed by Microsoft Research. Example job post: E.g.,

Millennium: Senior GenAI Engineer - Advanced Rag

Enrichments and Knowledge Graph Construction: Move beyond flat vector search by building GraphRAG systems and advanced annotations such topics, keywords, sentiment, etc. You will extract entities (Companies, People, Metrics) and relationships from text to build a dynamic Knowledge Graph that captures the nuance of the financial markets and its temporal aspects.

Basic vector RAG, which you would experience by attaching files to ChatGPT or NotebookLM, searches documents for relevant excerpts and simply attaches them as context to your prompt. GraphRAG indexes your corpus into entities and relationships then uses that structure to synthesize answers that aren’t obvious from any single chunk. Funds are hiring for this knowledge graph retrieval frameworks since they see their edge buried in writeups, interviews, surveys, notes and other longform text and want to maximize second-level insights.

Update on trading arenas

Soon to be released Grok 4.2 is the only model making money trading weather prediction markets (PredictionArena.ai)

In another arena featuring Grok 4.2, Alpha Arena, xAI’s forthcoming model won as well. We cover trading arenas in more detail in an earlier post.

Follow for more investor LLM workflows

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Detecting evasion

Generation IM ($21B AUM) shares a couple workflows in their 4Q25 letter:

“Our ‘First Looks’ initiative serves a different function. When analysts evaluate a new company, we leverage AI to provide a snapshot overview with green, yellow and red flags drawn from sources like Glassdoor reviews, which used to take hours of manual work. Finally, our ‘Deception Detection’ dashboard analyses earnings call transcripts across the portfolio, flagging watchlist topics and potential areas for forensic accounting review”

New paper features prompt and system design for identifying evasion: EvasionBench: Detecting evasive answers in financial Q&A via multi-model consensus and LLM-as-Judge. The key is to score each response on the degree to which management (i) answers the specific question, (ii) introduces irrelevant framing, (iii) relies on generalities, and (iv) deflects. Their eval is a set of 1,000 human annotated scores. Prompt is on page 12.

Automated expert network interviews

A couple things this week:

• Tech investor shares his experience with AlphaSense’s new AI-led interview product, with example outputs

• Former Chief Data Scientist at Third Point covers Ribbon.ai’s new expert-network-in-a-box: The biggest disruption to expert networks since Tegus

You can now source 1B+ experts leveraging their tool and then instantly book an expert call leveraging their voice AI, which then generates a transcript, which you can then leverage all their tools to analyze the transcript. All of this is white-label and available via API.

• Expert networks utilizing AI interviews include: AlphaSense, Expert Insights, Guidepoint, Qualitate, Ribbon.ai, Synquery (Email me any I’m missing, and I’ll compile and circulate a longer list)

• I’ve also heard of multiple funds building this internally

My take: I’ve been skeptical of AI-led interviews because I worried they’d overindex to lower quality “professional experts” and they wouldn’t create the chemistry eliciting deeper conversation. While definitely a limitation, I underappreciated how much better this experience is for the expert: an LLM is available around your schedule, they’re never rude to you or cancel on you. And there’s no chitchat. Also, investors don’t do more interviews because they’re constrained on time/mental-energy not so much on money. LLM driven interviews may dramatically increase both the supply and demand of expert interviews this year.

Software switching costs

Former long/short PM shares his ChatGPT thread on finding low mission critical software (@atelicinvest):

Prompt: Help me build a first-principles framework to identify low mission-critical software by analyzing integration depth, switching costs, compliance exposure, workflow impact, and behavioral indicators (e.g., low engagement, discounting, promotions), without relying on retention metrics.

More investor workflows

Former Healthcare PM vibecodes datasets his internal datateam would have never prioritized (@FundamentalEdge)

My take: Brett also mentions the notion of “artisanal workflows” which I think is a good concept here: everyone’s vibecoded research systems are going to be unique.

Fintool founder shares lessons from two years building investment research agents. Very thorough, technical and good (@Nicbstme)

Prompt for finding risk arb filings (@RodAlzmann).

Prompt for first- and second-order impacts of tariffs, sanctions and export controls in a research paper on geoeconomic pressure. Starts page 10.

Tool using Opus 4.5 to estimate hidden real estate value (@AltayCapital)

TMT and Energy investor shares experience building a 10-K deep dive tool (@TheValueist)

Interesting papers

What does it take to be a good AI research agent? Studying the role of ideation diversity. Two good things in here: (i) as models get better at using tools, LLM systems designed for greater diversity of ideas will outperform, and (ii) changing temperature doesn’t really help.

Taxonomy-aligned risk extraction from 10-K filings with autonomous improvement using LLMs. Written by the team behind AI tool Massive, so they don’t share the prompt. But it’s a good framework for classifying a large universe of companies into a customized set of risk factors. A related tool was recently released on X (@JaredKubin).

Updates from AI trading arenas

These are public experiments where LLMs make trading decisions. Longer summary of this trend is here; takeaway is that all trading robots eventually lose money but worth monitoring because the newer, more expensive models are starting to lose less money.

Aster - Humans vs AI: traders compete with robots for a $150K prize pool. Right now the AIs are being the humans on average, but nine out of the top ten traders are human. This feels about right!

Okbet Arena: 5 models compete at placing bets on Polymarket. All models are losing money, and right now GPT-5.1 is in the lead while Deepseek R1 is in last place.

FinDeepForecast: Live benchmark based on a new paper: FinDeepForecast: A live multi-agent system for benchmarking deep research agents in financial forecasting. Basically identical results to Okbet Arena.

Follow for more investor prompts and workflows

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Interesting use cases

Have an LLM listen to every industry podcast and alert you with anything relevant using our new Podcast Agent

26 million podcast episodes are published each year. Many are interviews with management teams from public companies, their competitors, customers and suppliers. Based on interest from several funds, we built a simple tool that listens to every podcast and alerts you of anything incremental to your coverage. Couple examples from this week:

• ORCL: VP of a 17 hospital system discusses their recent decision to transition off Oracle Health (Becker’s Healthcare Podcast)

• COTY, ULTA: Ulta’s SVP of Ecom discusses their new marketplace, how it plans to bring emerging brands into their brick and mortal channels (Omni Talk Retail)

We plan to add more sources of content to this over time. Sign up for free.

Askelladden Capital discusses its process for ingesting and scoring fund LP letters

I’ve built a tool that reads investment letters of other fund managers, ignore all macro / philosophical discussion, extract only single-ticker investment ideas, summarize them, and scores them against a rubric based on our historical priorities. That rubric was – drumroll – drafted by AI after reading years of our letters, then subsequently refined by me.

My take: If you’re building this agent for your process, Yellowbrick is a pretty good datasource for investor theses.

Prompt from Reddit commenter to analyze 10Ks/Qs for changes in forward looking statements.

My take: Agree with the focus on using LLM to extract verbatim language vs return a conclusion.

Interesting papers

LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings. Surveys a panel of LLMs pretending to be shoppers and compares results against real panelists, finding a 90% overlap. Prompts discussed in the paper and on their github.

My take: There’s a lot of synthetic panel research vendors including Aaru, Electric Twin and Qualtrics. I think synthetic panels work almost as well as human panels, which is not that great. The best ones are calibrated against real panel and purchasing data. I’m not clear why a panel of LLMs is better than a system leveraging a single LLM, or even what the difference is. Synthetic panels are also unlikely to produce entirely new findings like humans. They also might be more useful for companies for feedback on a specific product, vs investors who are looking for fresh, on the ground information. There’s a lot of interest in this space, so will explore it more and report back with anything interesting.

New tools

SimilarWeb has started offering 12 months of data free through Manus paid and free plans

My take: SimilarWeb already offers an MCP server for its existing customers to access via LLM, the difference here is their move to share data for free as lead gen. I think we’ll see a lot more of this, especially in relatively commoditized segments like web traffic data. Worth noting this was announced two weeks after Meta acquired Manus for >$2b.

Interesting job descriptions

AlphaSense/Tegus is hiring a hedge fund LLM workflows product manager

your value lies in your ~5 years of experience at a top-tier Hedge Fund…You will be the primary architect of “quality.” Design, test, and refine prompts to ensure our AI output meets the high standards of a professional investor. You will look at an AI summary or extraction and immediately know if it “sounds right” to a PM or Analyst

Millennium hiring GenAI engineer for advanced RAG

Follow for more case studies of LLMs+Investment Research

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Subscribe now

Case studies

Former PM at Schoenfeld, Citadel, DE Shaw shares his prompt for evaluating a new CEO (@FundamentEdge)

My take: I like Brett’s prompt because it first asks the model to do a lot of work understanding the history of the situation and what levers might be available new for management. That’s important for sizing the opportunity but for the user to see if the model’s understanding of history aligns with their own.

Australian hedge fund Minotaur Capital says new Claude Opus 4.5 model outperforms OpenAI on its internal research benchmarks (LP Letter)

On the technology front, we’ve spoken extensively about our disciplined framework for testing and evaluating large language models across different use cases. One of the challenges is how quickly the frontier shifts. Recently, we’ve been testing Codex versus Claude for writing and research tasks and have found that Claude (Opus 4.5) is currently delivering superior results. As a result, we’ve migrated a meaningful portion of our internal research workflows accordingly.

My take: New models come out frequently and even the same model can vary in performance week to week. Maintaining an objective benchmark for research summary is necessary to ensure you’re always using the highest quality model. One way I’ve seen this done is having the top LLMs summarize every earnings press release as soon as its posted, then later scoring each one against sellside recaps. A decent proxy for the best model at junior analyst type work.

JPMorgan is cutting ties with proxy advisory firms and will use in-house AI to cast shareholder votes (WSJ)

The bank will use the platform to manage the votes and the AI also will analyze data from more than 3,000 annual company meetings and provide recommendations to the portfolio managers, the memo said, replacing the typical roles of proxy advisers.

My take: Glass-Lewis + ISS is a duopoly because they aggregate voting power on behalf of their institutional clients. LLMs will cost them pricing power but I don’t see why most asset managers will follow suit and build their own.

New tools

New tool uses Gemini to chart the cultural prominence of various narratives over time (Cultural Eigenclusters, The Diff)

My take: I think something similar can be combined with price data to map the narratives around assets. Reach out if you’re interested in exploring.

Interesting job descriptions

Soros is hiring an AI Orchestration Engineer (LinkedIn)

Examples projects include tonal analysis of earnings calls or summarization of Bloomberg IB chats.

Verso Parters, a $600mm AUM SF-based hedge fund, is hiring a founding AI product engineer (Johns Hopkins)

Much of our investment research process is qualitative: expert interviews, reconstructing industry history, understanding how a company got to where it is, pressure-testing narratives, and tracking what would change our mind. The challenge is that the raw material of great research is vast and messy – notes, transcripts, filings, models, datasets, internal memos, and trade history.

Your job will be to build a research and decision-support “OS” that helps us:

• execute our research process more effectively,

• integrate insights across disparate data sources,

• spot biases and patterns in our own analyses and trading behavior,

• and ultimately ask better questions, see the ball more clearly, and make better investment decisions.

OpenAI hiring large cap research analyst from buyside or sellside (LinkedIn)

As equity research community’s interest in OpenAI grows, we are hiring a full-time role to engage closely with the analysts. Looking for an experienced large-cap equity research analyst from buy/sell-side. Arguably one of the best seats in the world to understand AI if you are curious. DM me if interested.

Millennium hiring AI engineer - equities technology (MLP careers)

Norway’s sovereign wealth fund hiring engineer for LLM workflows (LinkedIn)

Interesting papers

LLM driven investment strategies lose money over time because they struggle account for regime change (arXiv.org)

Follow for more case studies of LLMs+Investment Research

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

• There are a growing number of public experiments where LLMs make investment decisions and forecast future events

• There isn’t yet evidence of models beating the market at meaningful scale or statistical significance. However, each new generation of frontier model tends to be less bad

• OpenAI and Grok tend to be better at trading while Claude tends to be better at pure forecasting

• The various arena designs address the various “catch 22s” when LLMs make investment decisions including: (i) choosing across many assets vs focusing your tokens on a single asset, (ii) tool use vs context size, (iii) “real time” vs randomness, (iv) forecasting skill vs data access

• We will monitor these arenas going forward, as they are prototypes for eventual institutional strategies

Subscribe now

Background

Hedge funds primarily use LLMs to make their teams more efficient, but everyone secretly wonders whether the models will eventually make investment decisions on their own.

There’s a growing universe of “arenas” where models are evaluated on their ability to predict events and make investment decisions. Unlike popular benchmarks such as GDPval, GPQA and ARC, the answers haven’t happened yet so there’s no risk of contamination, and they can’t be saturated because the market makes them harder every day. Also unlike typical benchmarks, these investing and forecasting arenas exactly match a highly valuable real world task.

LLM Investing Arenas

Alpha Arena (nof1.ai)

• System design: In Alpha Arena, the models trade $10K each in real money across 7 stocks. Every ~2 minutes, the system asks each model to make a buy/sell/hold trading decision with context including its current portfolio, news, trading data and the original trade parameters. The arena is actually four separate arenas, each with a unique trading goal, in order to add statistical power to the final leaderboard.

• Result: All models eventually lost money, but Grok 4.20 (pre-release) and GPT-5.1 performed the best and made money in a couple instances. Claude Sonnet 4.5 and Grok 4 performed the worst

• My Take: Very slick implementation of LLM trading, and I’m excited to see what they roll out in “season 2.” One challenge with LLMs is they are usually non-deterministic, meaning they don’t produce the same answer every time. So if you call your model enough it might randomly dump your entire portfolio. The nof1.ai team solved this problem by prompting the model to create a trading plan (price target, stop loss, invalidation conditions, etc), then feeding that same plan back in future calls. Another smart thing they do is ask each model about the narrative supporting each trade, and where it expects it to go.

AI Controls Stock Account (Nathan Smith)

• System design: Ran ChatGPT Deep Research once per week for six months to allocate real money across a universe of small cap healthcare stocks

• Result: -17%

• My take: This was a great, early implementation (especially by a high school student!). A challenge with this approach is it uses one giant call to set its portfolio each week. That spreads its tokens across many, many different potential investment decisions. The name of the game is to burn as many tokens on the most valuable decisions, so I believe it’s better to build up the portfolio with many smaller decisions. This experiment also highlighted another crucial issue with LLM driven trading: portfolio construction. One of its core positions, AYTR, fell 83% when it announced failed Phase 3 trial results, and the portfolio never recovered. The problem wasn’t that the LLM should have known (the entire market was offsides), the problem was it shouldn’t have been such a large position given the source of edge had nothing to do with predicting drug trial results.

AI Investing Arena (Bobby Dhungana)

• System design: Models paper trade 5 ETF (S&P 500, Nasdaq, Gold, Interest Rates, Oil). The system asks each model to make a buy/sell/hold trading decision every 30 ~minutes with context including VIX volatility, treasury yields, dollar strength, oil prices

• Result: Still active, started Nov 25. GPT-5 in the lead, Claude Sonnet 4.5 in last place, though all are ~breakeven

• My take: Was inspired by and has similar implementation to Alpha Arena. But I like the focus of allocating across ETFs vs individual stocks. It’s possible the generalist nature of LLMs make it better suited to allocating across sectors, vs individual stocks where they have an information disadvantage. But too early to draw conclusions as the experiment has only been running since late November.

AI Arena (rallies.ai)

• System design: Models maintain their own portfolios, evaluating them every few days. Architecture includes custom MCP servers and tool calls to distill a large universe of potential investments into a few potential trades, so the decision model can focus on choosing among a few quality options.

  Result: Almost every model is making money, led by Deepseek and Grok-4 but still early

• My take: This architecture addresses the catch 22 of wanting the model to select among as many assets as possible, while still focusing as many tokens as possible on individual decisions. Their solution is an extensive screening step to first identify stocks at technical extremes, with unusual options flow, interesting fundamentals and near term catalysts. Still too early to draw conclusions, needs to go through an earnings cycle. Factors likely explain most of the move so far.

Flat Circle Arena (Flat Circle)

• System design: Models paper traded individual earnings during 4Q24

• Results: OpenAI o1 and Grok-2 performed the best, while Claude Sonnet 3.5 performed the worst. o1 performed much better than o3-mini. Opus performed much better than Sonnet. The more expensive models outperformed the cheaper ones

• My take: This was an early, rudimentary effort. One advantage to focusing entirely on earnings is the results are “pure idio” - i.e., market and other factors have limited impact on the returns, you’re almost entirely measuring LLMs’ ability to beat other investors. While these results were promising, results in subsequent earnings periods deteriorated as they entered different market environments (ie liberation day, AI capex boom). Another limitation of this strategy was focused on large cap stocks. It’s possible LLMs are more effective on the longer tail where there’s less competition.

LLM Forecasting Arenas

Related, there are a handful of “forecasting arenas” where instead of investment decisions, models bet on prediction markets or forecast events.

• AI Forecasting Benchmark by Metaculus

• Deep Research Bench by FutureSearch

• Forecaster Arena by Mert Gulson

• ForecastBench by Forecasting Research Institute

• FutureX by ByteDance Seed

• Good Judgement Open by Good Judgement

• Manifold Markets

• PrediBench by Presage Labs

• Prophet Arena by Sigma Research Lab @UChicago

My take: Forecasting arenas are a purer benchmark on LLMs’ ability to predict the future, and it turns out LLMs are pretty good at it. Results genearlly show leading forecasting models having a winning hitrate while betting on Polymarket or Kalshi (though unclear if good enough to win at any real scale).

Even the best models aren’t able to beat the best human forecasters (not sure they ever will, as the best forecasters also have access to LLMs).

Claude Opus and Sonnet appear the strongest at pure forecasting (unlike in the investing arenas, where they’re often the weakest). How could this be true? One theory is that Claude has the most analytical rigor (using baserates and proper scenario analysis) but weaker access to tools like google / x.com search that are more important for investing. This is where Grok, Gemini and OpenAI are strongest.

Catch 22s when LLMs make investment decisions

These arenas show various ways to address the “catch 22s” in LLMs making investment decisions:

• Wanting the model to select as many possible asset vs. focusing all your tokens on a single asset?

• Providing access to as many tools as possible vs. managing to an optimal context size?

• Allowing the models to make decisions in “real time” vs more randomness the more times you call the model?

• Choosing the best forecaster model (Claude) vs the ones with proprietary data access (Gemini, Grok)?

There’s no evidence yet of LLMs beating the market with any scale or statistical significance. However, there’s going to be a lot of new models and architectures released in 2026. With these improvements, we expect to see more institutional focus on LLMs making investment decisions.

Follow for more on investing arenas and other creative LLM use cases

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

A former sellside analyst used an LLM to analyze 10 years of earnings calls and identify analysts with the most prescient questions. Examples:

Umer Raffat from UBS - Biogen (BIIB) Earnings Call - Oct 2019:

…the implication in the data is that the high with insufficient exposure at the high dose, the second trial worked as well. But when we look at CDR Sum of the Boxes low dose actually looks more consistent than the high dose…how spot-on is that finding on patients that had a sufficient exposure?

LLM reasoning:

Raffat immediately spotted the fragility of the efficacy claim—dependence on a post-hoc exposure subset rather than the pre-specified ITT, foreshadowing the FDA advisory-panel scepticism and eventual withdrawal.

Return since: -20%

Vivek Arya from Bank of America - Broadcom (AVGO) - Dec 2023:

yesterday one of your peers suggested that the market for AI accelerators could be as large as $400 billion... how does Broadcom participate... and what does this larger AI accelerator market imply for your Ethernet networking business?

LLM reasoning:

Arya’s question forced management to quantify AI’s revenue contribution and linkage to networking months before the Street built outsized AI models. The $400B TAM figure sounded extreme in Dec-2023 yet is now consensus; Ethernet’s role in AI back-end networks—dismissed by many at the time—is today a core driver of Broadcom’s upside.

Return since: +338%

My take: Great LLM use case, figuring out which analysts were right for the right reasons. Would be interesting to run something similar on initiations / ratings changes, controlling for sector returns, comparing insights vs peers. Of course, I’d want to know what these analysts are asking about next (here’s the analyst recommendation pages for Umer Raffat and Vivek Arya).

Grading your own analysts

Walleye Capital records all internal conversations to analyze which of their teammembers are most prescient

…we really record every single Zoom, every single call.…So a big part of my job overseeing the risk of the firm, the chief investment officer title, every single morning, me and my risk team, sort of in the control center of running this giant process we have our risk calls and those are all recorded and we can go back and say, hey what were we talking about at this time? And continually have LLMs that are, that are processing those transcripts and helping us to both remember and provide insights and ultimately be a bit predictive, which has been hugely helpful just in that exercise, which is— We haven’t sort of talked about where I think this is going in the power of all this. And I mean, I do believe that we’re that we’re a leader. I don’t want to say we’re the leader because I definitely don’t know what other firms are doing, but I certainly think that we are a bit more advanced in our thinking of how to use these tools, but we’re just scratching the surface of what is possible once you actually start connecting all bits of information within the walls of the firm…

My take: Would recommend an intermediate step focused on returning verbatim excerpts first, folks are going to want a lot of auditability.

Finding short opportunities in bond indentures

LLMs may have identified the opportunity to accelerate Avid Bioscience’s debt and short their stock. Last March, Avid Biosciences received an acceleration notice because it had failed to remove a restrictive legend on its 2026 notes, causing it be in default. Avid shares declined 28% when it revealed it needed to raise $160mm in private placement to redeem the notes.

Byrne Hobart covered this in The Diff, concluding:

And, right now, it’s suddenly gotten much easier to do this at scale: you can unleash LLMs on indenture agreements, and try to find edge cases that the company didn’t think of or notice. These will all be technicalities in practice; in the Avid case, if the restriction had been a big deal to the note owner, they probably would have noticed right away. But, perhaps coincidentally, they only noticed after the newly-widespread availability of tools that can trawl through vast amounts of text to extract useful information.

In Money Stuff, Matt Levine weighed in that he was skeptical an LLM found this opportunity.

My take: Hard to be sure, but I think this opportunity *was* found by an LLM because the acceleration notice was received just two weeks after Google released Gemini 1.5 Pro - the first time a 1mm token context window was generally available - enabling the analysis of huge documents. Would have been straightforward for any fund to cycle through indentures to identify technicalities that could merit an acceleration notice. In fact, this would make a pretty interesting eval for new models that get released: run them against a huge corpus of indenture agreements and see what new opportunities get identified.

Iran Notice disclosures

John Friedman, CEO of Datamule, collected and published a searchable dataset of Iran Notice disclosures from SEC filings

My take: one way I think about LLMs is they enable instant creation of datasets that are plausibly interesting but not worth hiring and waiting for a human team to build.

Interesting job posts

Select buyside LLM related job descriptions:

Point72 - AI Engineer – Investment Research & Workflows ($150K-$200K)

This role partners directly with L/S equity portfolio managers, analysts, and business leadership to build innovative solutions to improve efficiency and research quality across the equities platform…

Longaeva (new Baly platform) - Research Product Associate - AI Enablement

Longaeva is adding an associate to join the proprietary research team to accelerate adoption of generative AI products across investment strategies. In this role, you will embed directly with the proprietary research and investment teams to build solutions that impact investment decisions. We are seeking a capable, technical candidate—someone able to do hands-on research product development, web scraping, and LLM/AI-powered synthesis of qualitative and quantitative data. The ideal candidate blends scrappy coding, data/information aggregation, and a strong product intuition, with a proven ability to ship projects fast and independently. You will translate our AI capabilities into actionable insights by rapidly prototyping agentic workflows, building novel research products, and driving adoption of in-house tools.

Bayview ($30b AUM credit firm) - LLM Analyst ($90K - $110K)

The Research team at Bayview Asset Management is hiring an LLM Analyst to unlock insight from large volumes of textual data, both external and internal, to inform investment theses, improve operations, and answer foundational questions about the mortgage industry and more broadly, the economy...Meet with portfolio managers, traders, marketing and servicing teams to identify and narrow down the question. Understand the business context behind each question….
Prototype quickly but evaluate rigorously: Design prompts, run experiments in notebooks and concisely synthesize results for fast iteration. Define clear success metrics to measure progress.

xAI - AI Buy-Side Finance Tutor ($45/hr)

We are seeking a skilled AI Buy-Side Finance Data Specialist to enhance xAI’s AI models by providing high-quality data annotations and inputs tailored to buy-side finance contexts. In this role, you will leverage your expertise in portfolio management, hedge fund strategies, private equity investments, venture capital deal sourcing, and high-frequency trading algorithms to support the training of AI systems. You will collaborate with technical teams to refine annotation tools and curate impactful data, ensuring our models effectively capture real-world buy-side finance dynamics.

My Take: Interesting these are mainly early career type hires, no graduate degree required. Looks like funds are happy to build on top of frontier models and popular tools. Lots of focus on prototyping and experimentation. The Grok LLM trainer hourly rate feels a little light!!

Follow for more case studies of LLMs+Investment Research

Subscribe now

If you would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

Rick Bhowmick, Head of Data Eng at Coatue, built a system that reads Polymarket and Kalshi and generates a daily investment newsletter based on odds changes

Check out today’s newsletter and here’s the github if you want to try yourself.

My take: I think this is really cool, and modified it to send a daily email to my extended family which has spurred some interesting discussions.

Shareholder activism

Askeladden Capital, a long-only small/micro-cap value investing firm, used LLMs in a proxy battle

During our proxy contest at AstroNova, AI tools helped us produce the extensive ISS deck and other materials that would have been vastly more costly to prepare otherwise (i.e., we would have needed to spend an incremental six figures, and/or developed far lower quality work-product). We earned endorsements from both ISS and Glass Lewis, and the incumbent board requested the CEO’s resignation. I believe, though it is of course impossible to verify, that we may well have run the first AI-powered proxy contest in history.

My take: LLMs should enable more proxy contests against smaller companies that previously weren’t ‘worth it.’ One challenge is the eval on something like this, given these are relatively infrequent and the feedback loop is so long. Will most likely be leveraged by advisory firms and investors with direct experience creating these materials.

Expert interviews

CEO of primary research firm, Kane & Company, uses LLMs to drive more meaningful diligence conversations

Before talking to experts, we run AI queries on the market and competitive landscape. This gives us a baseline of what public sources and models already know.

The baseline serves two purposes. It helps us write better questions because we know the common answers and can push past them.

It also becomes our quality control filter. When an expert’s answer matches ChatGPT’s output too closely, we flag it.

Say for example we asked ChatGPT about the Canadian IT outsourcing market before starting expert calls. It tells us growth was 18 percent. An expert later gave us the exact same number with the same framing. We know to ask where that figure came from and what assumptions drove it.

My take: one challenge with expert networks are “professional experts” - folks who make their living doing calls and are too far away from actual industry to offer true insights. It’s helpful to get a few reps in first with ChatGPT, which is often regurgitating the same content these folks read anyway. Then you’ll be better positioned to push past.

Expert interviews (again):

An AI marketing firm built an LLM system that interviews experts automatically

The only way to capture true expertise is to build an AI interviewer that earns the trust to be seen as a peer. An equal. One that asks questions so insightful the expert reveals the distinctive methodologies they’d normally only share with another seasoned professional. That is the real technical hurdle. Here is how we cleared it. …

The Note-Taker is an internal tool, invisible to the expert, that continuously analyzes the conversation. The Interviewer queries it for structured progress reports.

What the Note-Taker Tracks:

Coverage Analysis: Topics explored with confidence levels (high/medium/low).

Gap Identification: Required areas not yet addressed, prioritized by importance.

Time Status: Pacing assessment against the target duration, with wrap-up triggers.

Pattern Detection: Emerging themes, contradictions, or when the expert defaults to generic “best practices.”

Next Action: A specific suggestion for the next probe.

The key insight is that the Note-Taker returns structured data, not prose. This prevents the Interviewer from getting confused by a second voice. If the Note-Taker flags a gap in “decision-making frameworks,” the Interviewer integrates the suggestion naturally: “You mentioned evaluating channels—walk me through a recent decision where you chose not to invest somewhere.”

My take: I think we’re still a ways away from LLMs running effective investor style expert network interviews. The example in this post is for executives who want to be interviewed, and I think the best investment insights come from human run interviews, ideally unpaid ones with trusted relationships. However, the AI notetaker seems a very useful tool for human interviewers. Could enable folks who are very good at lining up phone calls with experts to conduct them, instead of having to be conducted by expert analysts.

Follow for more case studies of LLMs+Investment Research

Subscribe now

If you have any interesting examples or would like to discuss incorporating LLMs into your research process, reply to this email or reach out via X or LinkedIn.

We analyzed how the the top forecasting models on a discretionary hedge fund workflow.

History of forecasting

The forecasting community was first popularized two decades ago by Prof Philip Tetlock’s popular book Superforecasting. Competitions, like the Good Judgement Open and Metaculus, are months long contests where forecasting professionals predict the outcomes of various events that will resolve in the coming months such as elections, sports championships, weather and wars.

Recently, the community has started to accelerate as forecasters use LLMs to automate some of the manual research and calculation steps. Estimates that used to take hours or days can now be done in seconds. Now forecasting expertise is being channeled into specialty LLM system design so the marginal forecast can be automated.

This explains why we’re seeing these new LLM forecasting arenas thousands of individuals and teams competing in them. Historically, hedge funds haven’t had much overlap with the forecasting community - too many questions requiring too much specialized knowledge, with answers needed in too short of a time horizon.

We think this is about to change as LLM forecasting systems can provide answers in real time.

Subscribe now

Subscribe now

Claude 3.7 and 3.5 make different trading decisions given the same information

Yesterday, Anthropic released Claude 3.7 Sonnet - which shows superior reasoning scores to OpenAI o1 and DeepSeek R1.

Last night, we fed Claude 3.7 and Claude 3.5 the same information about 10 companies scheduled to report earnings this morning and asked both to make BUY/SELL calls for each stock.

Claude 3.7 Sonnet offers an “extended thinking mode” allowing the user to specify a budget for the number of reasoning tokens it should use while formulating a response. While we maxed out the reasoning budget, there is a limit to how high the budget can be set and the model often doesn’t use its entire budget. Each call used only about 4 cents worth of reasoning tokens, though you can imagine a future with investors spending 4 million dollars on a single investment decision.

Across the 10 names, both Claude 3.7 and 3.5 lost money because they incorrectly recommended BUY for Sempra (SRE), which missed on revenue and guidance and is currently down ~20%.

Overall, Claude 3.7 outperformed Claude 3.5 by about 10% thanks to 3.7 correctly calling BUY for Bank of Montreal (BMO) vs Claude 3.5 recommending SELL. BMO beat earnings and is currently up about 5%.

Both generations of the Sonnet model break their reasoning steps into similar concepts, but the newer generation appears to think harder and more deeply. We’ve contrasted the two models’ reasoning on BMO below:

Credit cycle observations

Claude 3.5

Credit Quality Deterioration:
- In Q4 2024, BMO reported significantly higher provisions for credit losses (PCL) of $1,523M vs $446M year-over-year
- Management noted credit performance deteriorated more than anticipated throughout 2024
- While Q4 2024 may represent a "high point" for provisions, they are expected to "remain elevated" through 2025

Claude 3.7 - extended thinking mode

Credit Loss Cycle Appears to Have Peaked
- In Q4 2024, management explicitly stated that Q4 represented a "high point" for provisions for credit losses (PCLs) that would "begin to moderate through 2025"
- The bank has already recognized substantial credit losses in 2024 (PCLs reached $1,523M in Q4 alone)
- Interest rate cuts have begun, which should gradually reduce pressure on borrowers
- The early recognition of credit issues and proactive provisioning positions BMO well for improving credit metrics
- Prior credit cycles show the stock typically recovers strongly once problems are fully recognized and addressed

Peer commentary

Claude 3.5

Challenging Operating Environment:
- Peer commentary (RBC, Scotiabank) indicates continued pressure on Canadian consumers from higher rates and rising unemployment
- Economic growth is expected to remain subdued in H1 2025
- Competitive pressure on deposits remains high, potentially squeezing margins
- Bank of Canada rate cuts may provide some relief but timing remains uncertain

Claude 3.7 - extended thinking mode

Beneficial Rate Environment Emerging
- Central banks have begun easing cycles, with more cuts expected in 2025
- BMO management specifically noted this should be positive for both economic activity and credit performance
- Peer commentary (RBC, BNS) supports expectations for mortgage market improvement and broader economic stabilization

Analyzing recent performance

Claude 3.5

Recent Performance vs Expectations:
- BMO has missed consensus estimates for three consecutive quarters
- Q4 2024 adjusted EPS declined to $1.90 from $2.93 year-over-year
- Return on Equity has fallen below target with management acknowledging it will take time to rebuild to 15% medium-term goal

Claude 3.7 - extended thinking mode

Capital Strength Enabling Shareholder Returns
- BMO has built exceptional capital strength with CET1 ratio reaching 13.6% in Q4 2024
- The bank announced both a dividend increase and a share repurchase program of up to 20 million shares (NCIB)
- January 2025 press releases confirm regulatory approvals have been received and the buyback is proceeding
- This capital return story provides support even if operating results are mixed

Positive Momentum in Core Businesses
- Q4 2024 delivered record pre-provision pre-tax earnings of $3.5 billion (up 8% YoY)
- Canadian P&C business has shown consistent strength with record revenues
- U.S. segment's contribution has significantly increased post-Bank of the West integration
- Management indicated "net bullish" outlook for U.S. growth prospects in 2025

Claude 3.7’s observations seem to be more forward looking. Apparently this is what better investment reasoning looks like.

Interesting articles

New paper details trading system based on LLMs + reinforcement learning. Authors incorporate an LLM monitoring for changes in market sentiment to overcome the structured data limitations of traditional RL based trading strategies.

The paper compares results to o1, GPT 4o and other open source models, and corroborates our conclusions that o1 outperforms other models. However, all models appear to be beaten by the RL-LLM hybrid system discussed in this paper (arXiv)

Is AI really thinking or just pretending to? This is really the key question, and the article lays out the arguments on both sides. One good quote:

The best use case is a situation where it’s hard for you to come up with a solution, but once you get a solution from the AI you can easily check to see if it’s correct. Writing code is a perfect example. Another example would be making a website: You can see what the AI produced and, if you don’t like it, just get the AI to redo it.

… another example is measuring how the models perform in the market (Vox)

Two articles from late last year about Balyasny’s internal LLM tool:

• Balyasny’s AI outperforms OpenAI in financial applications (hedgeweek)

• A day in the life of an applied AI engineer at Balyasny (efinancialcareers)

Interesting LLM hedge fund job descriptions

Citadel: Commodities - Machine Learning Engineer

“Commodities have undergone an information revolution. From ship tracking to oil storage levels and crop yields, more data on supply, demand, storage, and transport is available than ever before. Commodity markets are more globally connected: natural gas markets impact fertilizer production, while agricultural markets impact gasoline production…We combine specialist domain expertise with advanced modeling techniques to solve problems that others deem unsolvable.”

DE Shaw: Software Developer - Generative AI ($225K)

“Working on greenfield projects, which offer opportunities to shape the future of GAI at the firm and make a significant impact”

Millennium: Senior AI Engineer - Equities Technology ($213K)

“We are building the next generation of Large Language Modeling applications driven by Portfolio Manager's requirements that provide immediate value and scale as a core product.”

Point72 (Cubist Systematics): NLP Engineer

“Build start-of-the-art deep learning models to process large scale unstructured datasets.”

Follow how LLMs are beginning to make investment decisions

Subscribe now

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Subscribe now

Key takeaways

• After 344 earnings, Grok-2 and o1 share the lead with a ~54% hitrate, earning ~1.1% per earnings

• We contrast the reasoning approach of stronger vs weaker models regarding shareholder lawsuits

• Perplexity and Grok announced their own competitors to ChatGPT Deep Research

  • Agentic research systems create another dimension on which LLMs may compete with human investors: they could either reason better or research better

  • Since Deep Researchers search online and can access historical information, it’s impossible to backtest their ability to make investment decisions. You need to test them live

Model accuracy

Are all models converging on a 50/50 coinflip? 2 standard deviations away from the mean of a random coinflipper is 55% after 345 flips, and it seems worrisome the hitrate has been trending down.

However, the mean profit per earnings remains very strong for o1 (113 bps) and Grok-2 (107 bps). o1 is by far the most advanced according to other benchmarks and by far the most expensive in terms of tokens, seems promising it’s earning the most. We’ll see.

Example of good vs poor reasoning?

This project provides a way to quantify correct vs incorrect investor judgement. As an example, PGY recently reported better than expected revenue and margins and provided overall certainty around its balance sheet.

Gemini Flash Thinking, which loses -1.1% per earnings, had issued a SELL:

The emergence of securities fraud investigations significantly overshadows any positive historical trends or recent ABS deal announcements. While Pagaya has shown a pattern of exceeding guidance and achieving strong growth, the serious nature of securities fraud allegations creates substantial uncertainty and risk.  

We feed the models recent press releases, and apparently Gemini Flash Thinking had fixated on shareholder lawsuits related to a short seller report.

o1, which earns 1.1% per earnings, was fed the same information - it accounted for the shareholder suits but concluded BUY:

While the wave of shareholder lawsuits and the Q3 share‐price plunge indicate heightened uncertainty, the company’s expanding network of top-tier lending partners (including new bank and auto relationships) and its successful capital raises suggest solid underlying demand. In addition, management’s repeated emphasis on profitability in 2025, along with evidence of improving loan performance and rising fee margins, points to a business now positioned for stronger operating leverage as interest rates stabilize. With shares trading well off recent highs yet fundamentals and guidance still largely intact, the risk/reward profile appears favorable going into the February earnings release—making PGY a buy.

We don’t specify in the prompt whether to ignore shareholder lawsuit press releases. The returns teach us which reasoning is correct.

Interesting articles

The amount of compute per request is going to skyrocket. Major implications for datacenters and hyperscalers, and also what can and will be spent on investment decisions:

“…this single process from a single human interaction would involve 10 billion times more compute than a single human writing into ChatGPT today, at the exact same model size. That is the incredible expansion dynamic in inference compute that is playing out today and over the next few years!”

(See “Inference Compute Scaling” on Attune Research)

Extensive thread on using ChatGPT Deep Research to create an investment thesis around DoorDash (DASH). Lots of great detail, I particularly like the multiple rounds with ChatGPT to create the optimal prompt:

“I asked ChatGPT to build me a prompt for Deep Research to do Deep Research on Deep Research prompting. It read all the blogs and literature on best practices and gave me a thorough report. Then I asked for this to be turned into a prompt template for Deep Research. I've added it below. This routinely creates 3-5 page prompts that are generating 60-100 page, very thorough reports”

(@BuccoCapital on X)

Grok-3 with DeepSearch announced (Techcrunch)

Perplexity launches Deep Research (Perplexity)

Hedge Fund that replaced analysts with AI beat the market (Bloomberg)

Follow the progress of LLM investment research

Subscribe now

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Subscribe now

Key Takeaways

• After 209 live earnings, o1 now leads with a 57% hitrate and 130 bps mean return per earnings, followed by Grok-2 at a 55% hitrate

  • 57% is approximately 2 standard deviations away from random chance after 209 coinflips

  • The Gemini and Claude models appear to be approaching 50/50

• The models are fairly correlated with each other, tending to make the same calls

  • We’ll have to figure out ways to ensure model orthogonality before institutions start adopting LLMs to make investment decisions

• Recently spoken with a large number of readers and appreciate the helpful feedback

  • In addition to reporting on the leading language models and their ability to call company earnings, I plan to include other resources and news relevant to LLMs+investing

Model correlation

While the models show differing abilities, they are fairly correlated. LLMs are somewhat more likely to issue the same calls than if each model were merely flipping coins.

This makes sense as they likely share much of the same training data, technology and methodology.

This also means the prior basis of comparison of 5 models flipping coins was overly strict as we no longer talking about 5 independent models. After 209 earnings, o1’s hitrate is 57%. 2 standard deviations above the mean of a single coin flipper is 57% vs. 59% for 5 coin flippers.

I can’t deduce any patterns among why certain models are more or less correlated to others. Only thing I see is the two newest ‘reasoning’ focused models, o1 and Gemini Flash Thinking, appear least correlated with others. We’ll see if this trend continues.

Models’ orthogonality, the extent to which they are uncorrelated, will be a crucial dimension on capital allocators’ decisions to use them for trading decisions. Orthogonality across managers and market factors is essential for risk management and leverage.

Model accuracy

Comparison to other benchmarks

How does a model’s ability to call earnings compare to standard LLM benchmarks? We take each model’s reasoning score from LiveBench, and compare it with their hitrate and mean share price return on their ability to call live earnings.

The models with the best share price return are those with the highest and lowest reasoning scores. The models in the middle underperform.

“A lot of smart people think they’re way smarter than they are, and therefore they do worse than dumb people” - Charlie Munger

Upcoming earnings calls

I’m debating whether to continue listing these upcoming earnings calls at all, since the hitrates are so close to 50%. Even the hitrates where BUY or SELL calls are unanimous aren’t meaningfully more predictive. If these calls are valuable or you would like a different display of them, please let me know.

Industry news and updates

I’ve spoken to a lot of readers over the past couple weeks and am grateful for the feedback on this newsletter and the LLM systems we are building. If you and I haven’t spoken, please reach out!

For now, I plan to expand the scope of this newsletter to include news and resources generally relevant for the LLM+investors community. From this week:

OpenAI Deep Research + Open Deep Research. OpenAI released an new tool that’s helpful for investment research. It’s exhilarating to input a query, watch it conduct searches, consider the results, think of new queries and so forth. A few days later, David Zhang launched an open source version of Deep Research that already has 10K stars on GitHub. Excited to monitor development of these ‘research agents’ and their application to investing. Seems a model’s ability to reason is inextricable with its ability to research.

Model ML, LLM platform for PE and investment banks, announces $12m funding round. There have been dozens of these tools, but thought the description on their approach was interesting: “When you open Model ML, it looks a lot like Google Drive. It has its copycat versions of Excel, Powerpoint, Word, etc., which ensures that no information ever needs to leave the workspace.” Of course replicating Microsoft Office will be no small feat. But this approach seems similar to why OpenAI Deep Research is a better experience than OpenAI Operator. The fact that it’s fullstack means it can think around bottlenecks and get 10x as much done, instead of requiring the user to babysit.

Follow the progress of LLM investment research

Subscribe now

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Subscribe now

Key takeaways

• After 62 live earnings, Grok-2 remains in the lead with a 63% hitrate and a mean return of +171 bps

  • Claude Sonnet is in second place with a 60% hitrate and +119 bps per earnings

  • In addition to having a high hitrate, Grok and Sonnet do the best job at limiting losses when they are wrong

• Are the models merely flipping coins?

  • To put Grok’s 63% hitrate in context, we ran a thousand simulations of 5 models flipping coins 62 times

  • 63% is between 1 and 2 standard deviations from the mean. In other words, if the 5 models were guessing randomly, the best performing model would show a hitrate of at least 63% ~10% of the time

• Our new context template, which determines the company information we feed each model, successfully improved results across all models

  • We show a comparative analysis below and are now using it as our primary context template

  • The new context includes market commentary, institutional holders and fewer past quarters

• Gemini released its new Flash Thinking model, “capable of stronger reasoning capabilities”.

  • We have added this as an additional model and include its calls in the upcoming earnings section below

• We plan to add DeepSeek, but the model has a much lower context limit so we need to rearchitect some things to create a fair comparison

Recent earnings

Are we merely flipping coins?

The hitrate of the best performing model (Grok-2) is 63% after 62 live earnings calls. Because we’re selecting the best result of 5 separate models, random guessing alone would often return results above 50%. We checked by simulation and found that the best performing coin‐flipper scores 63% or higher in ~10% of scenarios. In other words, Grok’s results are between 1 and 2 standard deviations above the mean if all five models were randomly guessing.

Improved context

Our first attempt at giving each model more information when asking it to make a BUY or SELL call) resulted in decreased performance. Based on these learnings, we developed another attempt at an improved Context for the models including:

• adding market commentary for all historical quarters

• including fewer historical quarters

• adding institutional holders

The new context template has demonstrated improved performance.

A few examples of enhanced reasoning:

Case Study: AT&T (T)

AT&T rose 7% monday, driven by beat on mobile-phone and fiber customers. Most models already pointed to BUY with both contexts, but the new context switched Claude Opus from SELL to BUY, driven the following:

“At the recent Analyst Day in December 2024, management provided an encouraging long-term outlook, including plans to expand its fiber network to over 50 million locations. They guided for sustained growth in key metrics like service revenue, EBITDA, free cash flow and EPS.”

The company’s analyst day was not previously fed into the models’ context and appears to be an incremental factor in Claude Opus’ decision to BUY. We are working on more robust ways to add analyst day and conference presentations.

Case Study: HCA Healthcare (HCA)

HCA missed earnings Friday, declining 4%. While operating metrics improved, the loss was primarily due to greater than expected hurricane impacts.

Both o1 and Sonnet's models recommended BUY with the original context template and SELL with the new template.

Old Context format

o1 - BUY

"The share price has underperformed due to short-term hurricane impacts, but HCA reaffirmed guidance and expects strong demand and stable margins into next year, positioning the stock for upside.""

Sonnet - BUY

"While the share price has declined 17% since Q3 earnings due to hurricane impacts, the market appears to be overly focused on temporary disruptions rather than the strong underlying business performance and recovery trajectory. With operational metrics improving and hurricane impacts starting to anniversary, Q4 results are likely to demonstrate better-than-expected recovery and strong core performance, driving shares higher."

New Context format

o1 - SELL

"With Q4 expected at the lower end of guidance amid continued hurricane-related headwinds, analysts lowering targets, and growing policy uncertainties (ACA subsidies), near-term risks appear elevated, making a weaker post-earnings share reaction more likely than upside."

Sonnet - SELL

"The combination of known hurricane headwinds, analyst concerns about policy/subsidy risks, and decelerating growth metrics suggests Q4 results and/or 2025 guidance could disappoint current expectations, likely leading to further share price weakness."

Case Study: Union Pacific (UNP)

Old Context format - o1 - SELL

“Despite Union Pacific’s solid operations and year-over-year results, recent quarters’ earnings releases have disappointed investors and triggered stock price declines. Management has guided fourth-quarter results to be roughly in line with the third quarter, which appears below market expectations for stronger growth. As a result, the stock is at risk of another post-earnings pullback.”

New Context format - o1 - BUY

“Union Pacific has consistently improved its service reliability and operating efficiency, which is enabling volume gains (particularly in Intermodal) and driving solid pricing despite a soft freight environment. Management’s focus on cost control and a ‘buffer’ approach to resources has delivered steady margin expansion and double-digit EPS growth even with negative mix from elevated international Intermodal volumes and ongoing coal weakness.”

The new context written pre-earnings was very similar to a post-earnings summary written by the sellside (BMO Capital Markets)…

“UNP delivered industry leading improvement in opex (ex. fuel) …Efficiency gains can be seen across the network including better labor productivity, train length, and locomotive productivity. Service enhancement should prove accretive to volume in the coming years, and we expect cyclical tailwinds to drive strong EPS growth. F2025 EPS outlook for HSD to LDD growth despite a modest volume picture underscores the operating leverage opportunity of a more efficient network. Management sees further runway to optimize performance. Management is guiding to a mixed volume growth outlook amid coal headwinds, challenging international intermodal comps, and a varied economic backdrop.”

Picking up pennies in front of a steamroller?

Some readers expressed concern that LLMs may tend to make the ‘consensus’ call, causing them to have a higher than average hitrate but experience huge losses when they are wrong.

The current prompt template doesn’t instruct the models to factor in asymmetric upside or downside in their calls. Still, the models appear to differ in how they “risk-adjust:”

Note: we added Claude Sonnet on January 13, so it missed the painful Walgreens earnings and thrilling Radius Recycling earnings impacting the other models.

We will watch how the gains and losses compare as the earnings roll through and experiment with incorporating risk more explicitly in our prompt.

Upcoming earnings

The reasoning behind these calls are available here.

Follow the progress of LLM investment research

Subscribe now

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Subscribe now

Key takeaways

• After 37 earnings results, Grok-2 still the most accurate model with a 65% hitrate and a mean return of +230 bps per earnings

  • Anthropic’s models are at ~60% hitrate and 110 - 150 bps mean return per earnings

  • Gemini and OpenAI are both slightly below 50% with slightly negative mean return per earnings

  • This represents a slight decline in performance for Grok and Anthropic and a slight improvement for Gemini and OpenAI vs the previous update

• Our first attempt at feeding the models additional information led to decreased accuracy

  • Last week, we built a new version of the template for what company information we provide each model and have been running the two in parallel to see which is more accurate.

  • The new version added market commentary and additional historical quarters

  • Surprisingly, this “improved” context produced worse performance. We discuss theories below

• Based on these learnings, we have developed “v3” of the context template and are running a new test to see if accuracy improves

  • The v3 template includes current and past institutional holders, which may have impact on post earnings price action

  • We are testing this new version in parallel and will roll it out once we see consistent performance improvement across most models

Recent earnings

More information —> less accuracy?

Our system feeds each model the same information (“Context”) when asking it to make a BUY or SELL call about a given company’s earnings. Our original Context template includes the following information for the past few quarters:

• Share price performance vs the S&P

• Press releases

• Company and peer earnings transcripts

• Sellside upgrades and downgrades

• Share price performance during the quarter

• Share price reactions from past earnings

Last week, we began testing an improved Context template that included the above plus:

• Many more historical quarters of this information

• Market commentary explaining key price moves in the most recent quarter

These changes led to decreased performance overall:

Two theories:

• Some research shows decreased reasoning performance with longer context lengths. Adding additional historical quarters may have been counterproductive

• Some research shows weaker models outperforming stronger models in trading decisions during bull markets, as weaker models tend to overweight subjective commentary:

  “…stronger LLM tends to focus more on the facts while the weaker LLM give more weight to subjective news. However … the increased reasoning ability does not bring a higher return in the cryptocurrency trading. This outcome aligns with economic theory, which suggests that typical market participants are only partially rational, with investors driven by emotional and psychological factors that push asset prices far beyond stock’s intrinsic value…”

  This is one possible explanation for the improvement in the lower performance models (i.e., Gemini and Claude Opus), but not the stronger models.

Next attempt: adding institutional holders, more market commentary, fewer quarters

Based on these learnings, we are testing a new context template in parallel:

• Pulling in current and past institutional holders, often a factor in price action following earnings

• We have again pulled in market commentary, though this time for all past quarters

• Reducing the total number of historical quarters of information provided

We will compare the results in a future update.

Upcoming earnings

Note the following earnings estimates are derived from the original Context template, discussed above.

The reasoning behind these earnings calls are available here.

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Follow the progress of LLM investment research

Subscribe now

Subscribe now

Key takeaways

• After 22 earnings calls, Grok and Anthropic lead earnings accuracy, calling more than two thirds of earnings correctly with a mean return of 230 to 300 bps per earnings

  • The worst performing model is OpenAI’s o1, with a hitrate of 41% and a total return of -38%

• We made significant improvements to Context provided to each model:

  • Now includes market commentary explaining key price moves during the quarter

  • We are running the two Contexts in parallel and will roll out once we’ve confirmed overall earnings accuracy is improved

• OpenAI o1 exited preview mode, so we’ve switched over to the current o1-2024-12-17 model. This allows us to specify a reasoning_effort parameter, which we’ve set to “high"

Upcoming earnings

Recent earnings

Applied Digital (APLD)

Claude’s new Sonnet model correctly called SELL while its older Opus model called BUY. We discuss the differences in the reasoning in the prior post, where Sonnet correctly showed skepticism on the timeline of the Ellendale announcement. However, between when the SELL call was issued and the earnings release, the company announced an investment by Macquarie causing the stock to rise the day prior to earnings. The methodology uses only the trading day following earnings, that interim gain is not reflected here.

Citi (C)

The company posted stronger than expected markets and banking revenue, as well as a larger than expected buyback. Gemini and Grok each made solid points in their pre-earnings calls:

Gemini 2.0 Flash Experimental

The bank has also been able to close another consent order this quarter. Given this momentum, it is likely that the bank will meet or exceed expectations. The prior quarter stock decline was not driven by the bank's performance, rather, by an external regulatory matter, and this has been addressed.

Grok

Our analysis of Citigroup Inc.’s performance over the past four quarters supports a BUY recommendation… with consistent revenue growth across its core businesses, notably strong performance in services, banking, and wealth

Improvements to Context template

The system provides the same Context to each model for each call. This week, we added several improvements to the Context including:

• Market commentary explaining key price during the quarter

We are testing the new Context in parallel and will roll it out once we’ve confirmed it improves overall accuracy across all the models.

Contrasting calls made with old vs new Context

JB Hunt (JBHT) - Reports tomorrow after market close

Claude 3.5 Sonnet - Old Context - SELL

While there are some positives like improving safety metrics and bid compliance rates, the pricing and volume headwinds appear likely to drive another quarter of year-over-year profit declines. 

Claud 3.5 Sonnet - New Context - BUY

While pricing pressure persists, particularly in Intermodal contracts through H1 2025, the stock appears to be looking past this near-term headwind…The main risk is continued pricing pressure, but this appears largely priced in at current levels given management's consistent messaging on this headwind persisting through H1 2025.

Taiwan Semiconductor (TSM) - Reports tomorrow before market open

OpenAI o1 - Old Context - BUY

TSM's share price has underperformed the S&P by -1.84% during the quarter, indicating low market expectations. Historically, when TSM underperformed or only slightly outperformed the S&P during the quarter, the stock experienced a positive price reaction after earnings (e.g., +9.79% and +9.80% in previous similar quarters). Given this pattern, there is a strong likelihood that TSM's stock price will rise following the upcoming earnings, making it a good opportunity to BUY.

OpenAI o1 - New Context - BUY

TSMC’s AI-related revenue has surged to represent a mid-teens percentage of total sales this year, with management expecting it to triple and reach an even higher share next year. Strong demand for 3nm and 5nm smartphone and HPC/AI applications is driving utilization upward, lifting gross margins above 57%. Moreover, TSMC continues to pass along higher costs to customers while maintaining industry-leading technology and capacity at scale, reinforcing its pricing power. These factors, plus the company’s increased full-year revenue outlook (near +30% in USD terms), suggest TSMC’s shares are poised to close higher following earnings.  

If you have feedback or would like to participate in this project, please reply to this email or reach out via X or LinkedIn.

Follow the progress of LLM investment research

Subscribe now