Intro
Running financial research agents on smaller models like Samaya Sterling changes the math for developers who are tired of watching their cloud bills explode every time a user queries a database. For a long time, the default strategy for complex agentic workflows meant pointing heavy, proprietary frontier models at every single task. That approach works fine during initial prototyping. It stops working the moment you run those queries against actual production traffic and realize you are paying nearly seven dollars every time a user asks a nuanced question about a balance sheet. Samaya AI recently introduced Sterling, a post trained model built on NVIDIA Nemotron 3.5 Lightning that manages to beat larger proprietary systems on benchmark tests while dropping the cost per query down to sixty-one cents.
This price discrepancy matters because running a SaaS tool on raw API calls from massive models eats your margins before you even acquire your first ten paying customers. Sterling accomplishes its performance by utilizing a 30B mixture of experts architecture with only 3B active parameters per token. That architectural choice means the base model fits comfortably onto a single H100 80GB instance in BF16 format, giving infrastructure engineers predictable hardware footprints instead of requiring massive multi-node clusters just to handle inference.
What makes this release particularly interesting for anyone building domain specific agents is the training methodology. The team did not just pull an open weight model off the shelf and write a standard system prompt. They co-trained the small model alongside a custom use, stripping out verbose instructions and merging tool definitions so the limited active parameter count would not get confused by context bloat. You can read the original announcement and breakdown over on Marktechpost (opens in new tab). Co-training small models with specialized execution uses proves that you do not always need a massive frontier model to extract deep, reasoned insights from complex financial documents.
Background on Financial Research Agents
If you have ever tried to run deep financial analysis using raw frontier models, you already know the pain of watching your token budget drain away on recursive tool calls. The traditional playbook relies on throwing massive proprietary models at complex tasks, assuming that more parameters automatically equate to better reasoning. When an agent has to parse SEC filings, pull historical stock data, cross-reference balance sheets, and format the output into a coherent report, the token count balloons rapidly. You end up paying nearly seven dollars per query in API fees alone. For a small business or an independent SaaS operator, that kind of burn rate makes autonomous research agents economically unviable outside of a well-funded enterprise pilot.
Mixture of experts architectures change the resource planning math entirely. By packing 30 total parameters while only activating 3B per token, models like the underlying Nemotron 3.5 Lightning deliver the inference speed and memory footprint of a much smaller network without sacrificing the breadth of the pre-training. On a single H100 80GB instance running BF16, you can host the base model and maintain a substantial context window without provisioning a distributed cluster of GPUs. That predictability is what lets infrastructure engineers actually budget for compute instead of guessing what their cloud bill will look like after a traffic spike.
Yet, raw compute efficiency means nothing if the agent cannot use its tools properly. Small models struggle when handed the massive, verbose system prompts that engineers typically write for frontier systems. They get lost in the instructions, misinterpret arguments, and fail to chain tool calls correctly. Building a reliable financial research agent requires more than just swapping out the base weights. You have to design specialized execution uses that manage state, handle long context windows, and feed data cleanly between the planning module and the execution tools.
Managing this overhead involves specialized helper models running alongside the primary agent. A condenser shortens long tool results to keep the prompt lean, while a compactor summarizes the execution trace as the research session grows longer. Without these structural guardrails, a smaller model will inevitably derail itself before finishing the task. Treating the use and the model as a single, co-developed unit is the only way to squeeze enterprise-grade output out of a 3B-active footprint.
What is Samaya Sterling and How Was It Trained?
Samaya Sterling departs from standard base models by functioning as a LoRA post-trained version of NVIDIA’s Nemotron 3.5 Lightning. Rather than acting as a general-purpose conversational assistant, it has been specifically tuned to plan financial research, execute specialized tool calls, and pass raw data cleanly to a separate report writer. The team behind the project started with a 30B total parameter mixture-of-experts architecture with 3 active parameters, adapting their production finance use to work within those constrained limits.
The training pipeline relied heavily on Tinker, an API provided by Thinking Machines Lab, using a four-stage process that combined supervised fine-tuning with reinforcement learning. In the first stage of supervised fine-tuning on synthetic traces, the team generated 968 synthetic analyst questions. A teacher model, Kimi K3, answered each question twice, and an independent LLM judge selected the superior response. This forced the small model to learn how to call 37 distinct finance tools with valid arguments.
The second stage scaled this approach by having Kimi K3 answer 1,819 expert-written questions using eight different research strategies. The best-graded answer for each question became part of the training data. After filtering out lower-quality traces, exactly 1,157 questions remained for the final dataset. This iterative filtering ensured the model only absorbed high-yielding execution patterns rather than mimicking every trial-and-error loop of the teacher.
Following supervised fine-tuning, the team applied offline Generalized Range Preference Optimization, or GRPO, which scored all eight teacher answers against each other without requiring any new generations. Because no fresh inference was needed during this step, the compute cost remained low. The final training stage shifted to online GRPO, where the model answered each validation question eight times and actively learned from its own graded attempts. A truncated importance sampling mechanism was used to correct a known sampler mismatch inherent in mixture-of-experts architectures.
Throughout both the training and execution phases, helper models played a crucial structural role. A condenser model shortens long tool outputs before they hit the main context window, while a compactor summarizes the accumulated execution trace as the research session stretches on. You can read more about the architectural specifics in the original report on Marktechpost (opens in new tab). These guardrails prevent the 3B-active model from getting overwhelmed by verbose data, proving that smaller models require solid auxiliary plumbing to compete with proprietary frontier endpoints.
What It Means in Practice for Developers
When you are running a SaaS tool that charges customers a flat subscription fee, every backend token eats directly into your margin. If an automated financial research query costs nearly seven dollars to process using a massive proprietary frontier model, your pricing page has to be remarkably aggressive just to keep the lights on. Samaya Sterling cuts that execution cost down to sixty-one cents per question. For a small business operator or solo developer building domain-specific agents, dropping your compute overhead by an order of magnitude changes the unit economics from a speculative science experiment into a viable product.
Lower inference costs do not come without operational compromises, and the latency figures tell that story clearly. Sterling takes roughly two hundred and eighty-two seconds to work through a single query, which is a eternity in terms of web application responsiveness. If you are plugging this model into a chat interface, you cannot leave a user staring at a blank screen for nearly five minutes without severe churn. You have to build asynchronous worker queues, progress bars, and streaming status updates that show what tool calls the agent is currently executing.
The integration work also demands a heavy rethink of prompt engineering and tool design. The team behind Sterling stripped detailed guidance straight out of the system prompts and merged multiple tool definitions together just to keep the small model on track. If you hand a 3B active parameter model the kind of sprawling, highly verbose system prompts that a larger frontier model can easily ignore or parse, the small model will quickly lose its reasoning thread. You have to treat the use and the model weights as a single tightly coupled unit rather than swapping components out at will.
I expect we will see more teams leaning toward specialized open weights models paired with custom helper architecture, simply because renting massive APIs for narrow tasks stops making financial sense once your user volume scales past a few hundred daily requests. If you want to test these efficiencies on your own terms, audit your current agent workflows and identify which narrow tasks are currently routing expensive tokens to general-purpose frontier endpoints. Spin up a single H100 instance, grab an open base model that fits your hardware budget, and start experimenting with LoRA fine-tuning tailored strictly to your own tool definitions.
What to Expect Next in Agentic AI Infrastructure
The release of Samaya Sterling highlights a broader shift in how engineering teams approach specialized agent design. When you look at the economics reported in the Marktechpost coverage of Samaya Sterling (opens in new tab), paying nearly seven dollars per query for a general-purpose frontier model is unsustainable for high-volume SaaS applications. I expect we will see public weights or managed APIs materialize for similar fine-tuned variants as more developers demand cost-effective domain tooling. Commercial API providers will face mounting pressure to slash pricing on complex multi-step reasoning tasks, or risk losing those workloads entirely to custom, in-house infrastructure running on smaller hardware footprints.
Running these setups introduces its own operational overhead, however. You are no longer just calling a simple endpoint and hoping for the best. You have to maintain helper models alongside your primary agent, such as a condenser that shortens long tool results and a compactor that summarizes the growing trace. Managing that orchestration layer requires solid monitoring to ensure the condenser does not strip away crucial financial figures before the report writer processes them.
Scaling a system with a mixture of experts architecture means tracking active parameters, GPU memory limits, and batch sizes carefully on your own iron. When I manage hosting resources for my own boxes, adding secondary helper services always increases the surface area for unexpected memory leaks and latency spikes. Developers adopting this architecture will need to build resilient fallback mechanisms for when the helper models lag behind the primary agent.
Next Steps for Your Tech Stack
If you want to move away from expensive general-purpose APIs, you should start by auditing your current agent workflows to see if smaller open weights models can handle narrow domain tasks. Look at the prompts and tool calls in your logs. Identify the exact tasks where a specialized model with a tailored use could replace a massive frontier endpoint without dropping accuracy below your acceptable threshold.
Once you find a candidate workflow, spin up a single H100 instance to test domain-specific fine-tuning. You can use tools like Tinker, the training API from Thinking Machines Lab, to run LoRA experiments on smaller models like Samaya Sterling, which is based on NVIDIA Nemotron 3.5 Lightning as detailed in the original Marktechpost report (opens in new tab). Set up a local test environment where you can measure both inference latency and per-query costs directly.
Do not try to migrate your entire production traffic over on day one. Route five percent of your non-critical analytical queries to your new fine-tuned model and compare the output against your current frontier setup. Pay close attention to how the model handles complex tool calls and whether your helper models keep the context window under control. Adjust your system prompts by stripping away unnecessary instructions and merging redundant tools, exactly as the Samaya team did to keep their small model on track.