Baseten

nference is everything

BasetenSep 16, 2026

nference is everything

Baseten launch preview
9119727K40

Tags

<500KB2D (developers)Series C+Feature launchDemoUS

TLVC Rating

2.20
Hook
Editing / Creativity
Copy
Sentiment of launch
Distribution strategy

Community Rating

No ratings yet

to rate this launch.

Baseten Hosted Tools and Baseten Grounded Inference add server-side capabilities to the open-source and custom models running on Baseten's inference platform, aimed at developers who have been stitching together their own tool loops to make models useful in production. The first capability shipping is web search, which gives models hosted on Baseten access to information that is as recent as the current day, specific, and absent from their training data . Instead of managing separate API keys and orchestration code, developers add a tool selector to a standard Chat Completions, Messages, or Responses request and Baseten runs the model and the search provider together in a server-side loop. At launch, Baseten is offering access to four web search providers: Exa, Keenable, Parallel, and You.com , with pricing passed through to the Baseten bill. The model decides whether a prompt needs web search based on its native tool use capability and may execute multiple iterations until it is confident , and teams can tune that behavior through a configurable iteration budget that scales from one-shot lookups to deeper research workflows. Baseten interleaves model inference and search requests in a server-side loop while streaming tool-use and tool-result blocks as live progress to the client , which cuts round-trip latency compared with client-side tool orchestration. The launch matters because open-source models like Llama, DeepSeek, and GLM are increasingly viable substitutes for closed frontier APIs, but they have lacked the batteries-included tool infrastructure that makes those APIs easy to ship against. Baseten was founded in 2019 in San Francisco by Amir Haghighat (CTO), Tuhin Srivastava (CEO), Pankaj Gupta, and Philip Howes (Chief Scientist), and this release extends the company's inference stack from raw model serving toward the agent and grounding layer that application teams have been building by hand.

Comments (15)

Sign in to join the discussion.
P
Priya Kothari26d ago

The tagline says 'nference is everything' and I'm choosing to believe that's a typo you're keeping on purpose for the memes.

M
Milos V.26d ago

Whoever wrote that opening line about 'latency taxes' deserves a raise. Actual pain, actual solution, no fluff intro.

D
Devika R.26d ago

We built something like this internally at Meta in 2019 and killed it because the round-trips were miserable. Curious how they're actually cutting the latency and not just hiding it.

K
kenji off the bench26d ago

ok wait, hosted tools AND grounded inference in the same drop? Someone at Baseten skipped the 'save it for Q2' meeting.

A
Amara Okonkwo26d ago

Cute launch but who's actually paying for grounded inference on open-source vs just using a frontier API? Show me the retention curve on the tool-calling users specifically.

R
Roland Beck26d ago

Every infra company is stapling web search onto inference this quarter. The moat here is a strong breeze.

Z
Zied H.26d ago

Before my team touches this: SOC2 Type II, SSO via Okta, and where does the search query data live at rest? Please don't make me file another ticket.

H
Hugo Marchetti26d ago

hot take: 'grounded inference' is going to be the 'RAG' of 2025, everyone claims it, nobody defines it the same way.

I
Ida Nurmi26d ago

Still evaluating whether LLMs need internet access at all, so this is a bit ahead of my current 2019 vibe check.

N
Nadia Farooqi26d ago

Pricing page or it didn't happen. If search calls are metered separately from tokens the finance team is going to have opinions.

L
Lin H.26d ago

Called this bundling play months ago in our infra memo. Baseten is quietly becoming the layer people forget to mention but can't ship without.

R
Romila Sen26d ago

This slots perfectly next to what two of my portfolio companies are doing on the agent side. Distribution is going to be the interesting fight, not the tech.

B
Bakr Osei26d ago

Is the demo pre-recorded or can I hit a real endpoint right now? Asking because 'real-time' has done a lot of heavy lifting in launch videos this year.

G
gm.eth in disguise26d ago

Would be sicker if the search index was verifiable onchain but I say that about everything so ignore me.

T
Tomasz K.26d ago

Love it. One more thing: let me pass my own re-ranker into the search step so I can bias results for my domain.