Inkling-Small Open-Source AI Model Explained

If you want an open model that can handle long inputs without the same cost as many top systems, Inkling-Small is the point. On July 30, 2026, Thinking Machines released a 276-billion-parameter multimodal MoE model with only 12 billion active parameters per request, plus a 1 million-token context window and an Apache 2.0 license.

Here’s the short version:

• I can use it with text, images, and audio
• I get outputs in text, code, and JSON
• I can tune thinking effort from 0.2 to 0.99 to balance cost and reasoning depth
• I can self-host it for the full 1,000,000-token context window
• I can use Tinker if I want hosted access, with a 256,000-token context cap
• I can download the open weights from Hugging Face
• I can use a 4-bit NVFP4 version on NVIDIA GPUs
• I can fine-tune it on private data and run it inside my own setup

A few numbers stand out. Inkling-Small scored 77.6% on SWE-bench Verified, matched Nemotron 3 Ultra on Terminal Bench 2.1 at about one-third the token cost, and posted 98.6% on StrongREJECT. So even though the name says Small, the model is aimed at serious coding, document work, support flows, and agent tasks.

Inkling-Small Isn't Small At All: Let's Test this 276B Model

Quick Comparison

My takeaway: this release gives U.S. teams a clear option for lower-cost deployment, private setup, and long-context multimodal work without being locked into a closed API.

Inkling-Small Overview: Architecture, Size, and Capabilities

Inkling-Small is a decoder-only Mixture-of-Experts (MoE) transformer with 276 billion total parameters and about 12 billion active parameters per token. For comparison, the flagship model has 975 billion total parameters and 41 billion active parameters per token ^[2][1][3]. That gap matters. It helps explain why Inkling-Small can feel lighter to work with while still handling demanding jobs. You see that most clearly in its multimodal input support and its long context window.

Multimodal Inputs, Text Outputs, and Long-Context Support

Inkling-Small takes text, images, and audio as inputs and returns text outputs, including natural language, code, and JSON ^[2][1]. It was pretrained on 45 trillion tokens of multimodal data across text, image, audio, and video ^[1][3].

It also supports a context window of up to 1 million tokens ^[1]. That's a big deal for teams working with large codebases, long documents, or extended transcripts. Instead of chopping material into tiny pieces and hoping nothing gets lost, they can hand the model far more of the full picture at once. That tends to matter when you're weighing self-hosting, fine-tuning, or using the model through tooling.

Benchmark Performance and What 'Small' Actually Means

The benchmark results help put the name Small in perspective.

On SWE-bench Verified, Inkling-Small scored 77.6%, ahead of Nvidia's Nemotron 3 at 71.9% ^[2]. On Terminal Bench 2.1, it matched Nemotron 3 Ultra at about one-third the token cost ^[1][3]. It also scored 98.6% on StrongREJECT, which points to strong refusal handling ^[2].

Put simply, Small doesn't mean weak. In practice, the MoE setup and long-context support show up in work that teams care about: code review, document analysis, and agent workflows.

How to Access Inkling-Small: Weights, Tooling, and Integration Options

Open Weights, Model Card, and Downloadable Files

Inkling-Small’s weights are publicly available on Hugging Face under the thinkingmachines organization ^[2][6].

The release comes with the core files most teams expect: weights, a model card, tokenizer, config files, and modality encoders ^[4][2]. So you’re not starting from scratch or piecing things together by hand.

If you’re deploying on NVIDIA GPUs, there’s also a 4-bit quantized version called Inkling-Small-NVFP4 for better performance on that setup ^[4][5]. And if your team wants to try the model before taking on self-hosting, Tinker is the fastest way in.

Self-Hosting, Hosted APIs, and Fine-Tuning: A Comparison

Thinking Machines offers Tinker, a testing and fine-tuning API that lets developers try the model, cap response lengths, turn on web search, and run enterprise-grade fine-tuning ^[2][1].

Tinker also supports day-one integrations with major inference and deployment tools ^[6][1]. That matters because the model itself is only part of the job. You also need a path to get it into a live stack without a pile of setup work.

The tradeoff is pretty simple:

• Self-hosting gives you full control and access to the 1 million-token context window ^[6][1].
• Tinker cuts down the ops work, but context is capped at 256,000 tokens ^[6][1].

So the choice comes down to speed versus control. If you want managed access and less infrastructure work, Tinker is the easy route. If you need the full context window and tighter control over deployment, self-hosting makes more sense.

Where APIMart Fits in Multimodal Production Workflows

After a team picks an access path, there’s still the practical side of production: getting multimodal requests to move through one clean workflow.

For teams that need analysis and generation in the same flow, APIMart provides a single integration layer for multimodal APIs.

Use Cases and Cost-Efficient Deployment for U.S. Teams

Coding Assistants, Agents, Support Tools, and Document Analysis

Now that access is clear, the next step is simple: where does Inkling-Small pull its weight? Its MoE design helps keep inference lean for high-volume production work ^[1].

For software teams, that makes it a strong fit for repo-level bug fixes, PR reviews, and multi-step coding agents that need to work across large codebases ^[2].

That same setup also works well for jobs that need fast classification and long-context reading. Think support copilots that handle ticket triage, answer policy questions, and send escalations to the right place. At scale, those small gains in latency and compute add up fast.

It also fits large-document analysis, including contracts, policy files, and technical manuals ^[1]. If your team deals with dense documents all day, this is the kind of workload where long-context reading starts to matter.

For education platforms, Inkling-Small can power math and logic tutoring with adjustable thinking effort ^[2]. That gives teams a way to match model effort to the task instead of paying the same compute cost every time.

Typical fits include:

• Coding agents
• Support copilots
• Document analysis
• Tutoring
• Multimodal tagging

Cost, Latency, and Budget Planning for U.S. Teams

The main cost lever here is the MoE design. It helps keep inference cost and latency lower for repeated requests ^[1].

Developers can also tune thinking effort in code, from 0.2 for simpler tasks up to 0.99 for harder reasoning ^[2]. In plain terms, teams can use less compute for high-volume, lower-complexity work and save more thinking time for the cases that actually need it.

That kind of control matters when you're planning spend across a U.S. team. A support flow handling thousands of routine requests per day doesn't need the same setting as a coding agent working through messy edge cases. One model, different effort levels, and a clearer way to manage budget.

Because the model is released under an Apache 2.0 license, U.S. teams can also run it on-premises or inside VPCs when data control is a priority ^[2][1].

Those deployment options set up the research takeaways in the next section.

Research Takeaways and Final Summary

Why the Open-Weights Release Matters for Testing and Customization

Now that access and deployment are covered, the main research angle comes down to what open weights let teams do in practice.

Because the weights are open, researchers can inspect the architecture and reasoning controls directly. They can also test the model on internal data without sending that data through an external API. That level of visibility helps with safety testing too. Instead of taking published claims at face value, organizations can verify results on their own infrastructure.

Domain adaptation is another big plus. Teams in areas like financial analysis or software engineering can fine-tune the model on proprietary data instead of relying on a general-purpose system ^[2]. The same openness also supports local deployment and domain-specific adaptation, so teams can run independent audits on their own terms.

Key Points to Take Away

A few points matter most here:

• Open weights let teams test, fine-tune, and benchmark the model on their own infrastructure.
• The weights are available on Hugging Face, and Tinker gives teams a way to pressure-test settings before moving to production ^[2][1].
• Inkling-Small is built for controlled, cost-sensitive deployment, with open weights that support testing, fine-tuning, and independent benchmarking.

For teams that need control, customization, and independent validation, that setup makes Inkling-Small a practical option.

FAQs

What hardware do I need to self-host Inkling-Small?

Inkling-Small is built around a 276-billion-parameter architecture and uses native NVFP4 quantized checkpoints for NVIDIA Blackwell systems.

Your setup should also support open-source inference libraries like SGLang, vLLM, TokenSpeed, or llama.cpp. While Thinking Machines used GB300 NVL72 systems during development, the small variant is meant to be a more cost-effective, lower-latency choice for local deployment.

When should I use self-hosting instead of Tinker?

Use self-hosting when your organization needs full control over agentic AI workloads. That can mean running models on-premises or inside a virtual private cloud, with your team in charge of the setup and day-to-day operations.

It’s also a good fit for teams that want to manage their own infrastructure, cut ongoing token costs, or meet specific data privacy requirements.

Tinker is the better choice if you want a convenient, low-friction setup for research and fine-tuning. Self-hosting gives you more room to tune performance and cost around your own hardware.

How does thinking effort affect cost and response quality?

Inkling’s controllable thinking effort gives developers a simple way to tune the model’s reasoning budget from 0.2 to 0.99. Higher settings use more compute for complex, multi-step reasoning. Lower settings cut token use and latency for simpler tasks.

Because Inkling condenses chain-of-thought reasoning, it can often reach accurate results with fewer tokens. That gives users more control over cost and performance based on what their deployment needs.