Getting Started With Inkling 2 Mini For Local AI
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Getting Started With Inkling 2 Mini For Local AI on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local AI use, with support for several inference runtimes. The company reports competitive benchmark results, but the supplied information does not include independent validation, complete hardware requirements or detailed real-world speed measurements.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model designed for local AI setups that the company says has about 26 billion total parameters and activates about 3 billion per token, as described in the original analysis. The release includes support for Transformers, llama.cpp, MLX and SGLang, alongside a research version with adjustable layer-loop passes; independent performance and practical hardware needs are not established in the supplied material.

Inkling 2 Mini accepts text and image input and is described as having a 1 million-token context window. Its architecture has 24 physical decoder layers. The first six and last six run once, while the 12 middle layers are reused in three passes. This produces 48 logical layer visits per token without assigning a separate set of weights to each visit, according to the company’s technical description.

The model uses a mixture-of-experts design: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 mixed-precision formats and includes a DSpark drafter intended to accelerate generation with a small memory cost. The fixed-pass model is supported by Transformers, llama.cpp and MLX; SGLang support is described as available for datacenter deployments.

A separate research artifact, TML-d24-varloop-research, allows the number of passes through the repeated layers to be changed. The company says using fewer passes can increase inference speed, with a potential quality trade-off. It does not provide a complete hardware configuration or measurements showing how the model performs on specific consumer devices.

At a glance
announcementWhen: Release date not specified in the suppl…
The developmentThinking Machines Lab released Inkling 2 Mini, a local-AI model with a research variant that allows users to adjust how many times selected layers run.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

A New Option for Local Model Users

Inkling 2 Mini gives people building local AI systems another multimodal model to test, with ports to several widely used inference runtimes. Its reported architecture seeks to combine substantial model capacity with weight reuse and selective expert activation. Those design choices may affect the balance between capability, memory use and computation, but they do not establish how the model will perform on a given machine.

The release is relevant to users comparing models for image tasks, coding or other local workflows because the company also claims a million-token context window and reports strong benchmark results. However, day-to-day value will depend on factors such as output quality, image handling, long-context reliability and generation speed. Without independent evaluations and device-specific measurements, readers cannot yet determine whether the model’s reported advantages translate into practical gains.

Products Worth Considering

DJI Mini 4K Fly More Combo Drone with 4K Camera and 3 Batteries

DJI Mini 4K Fly More Combo Drone with 4K Camera and 3 Batteries

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, White Smoke

Bose QuietComfort Ultra Earbuds (2nd Gen), Wireless Noise-Cancelling Earbuds, White Smoke

As an affiliate, we earn on qualifying purchases.

How the Repeated Layers Work

The central design choice is to run the same 12 middle physical layers three times, rather than creating distinct weights for all 48 logical layer visits. According to the technical description, each pass begins with the same input from the first six layers; learned messages carry information between passes, and a learned gate controls how much information is passed along. The company says each visit retains its own key-value and short-convolution state, so reusing weights does not mean the model skips those computations.

Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as making 48 layer visits per token, but Mini has a smaller hidden width—2,304 rather than 4,608—and routes among 128 experts per layer rather than 384. The supplied material does not state a complete downloadable file size or minimum memory requirement. It also cites earlier exploration of looping architectures, including Nanbeige, without providing further comparative results.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Independent Results and Hardware Needs

The performance claims remain company-reported. Thinking Machines says Inkling 2 Mini is state of the art in its class on the Artificial Analysis Intelligence Index v5 and leads in 32 of 48 comparisons against several named models. The supplied material does not include the full score table or independent replication. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6; those comparisons likewise are not independently established here.

Important practical details are still absent from the provided information, including complete memory requirements, measured generation speeds across consumer hardware, and independent tests of image and long-context performance. The quality of the research version at one or two passes is also not fully assessed. The supplied source says a full model card may contain more detail, but that material was not available for this report.

Model Card and Runtime Testing

The next useful evidence would include the full model card, complete benchmark scores and reproducible comparisons conducted outside the company. Tests that identify hardware, runtime, quantization settings and prompt conditions could help users judge whether the reported results are repeatable and what resources are required.

Reports from users running the model through Transformers, llama.cpp, MLX and SGLang can also clarify compatibility and real-world speed. Testing the adjustable-pass research artifact would help show whether fewer passes deliver a useful speed improvement and how much quality changes. Until those details are available, the release establishes that the model and runtime support have been announced, not how well every local setup will handle them.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab for local AI use. The company says it accepts text and images and has about 26 billion total parameters, with about 3 billion activated per token.

Can Inkling 2 Mini run on a regular consumer computer?

The supplied material does not give a minimum hardware configuration or complete memory requirements. It is not possible to determine from these details alone which consumer computers can run it or at what speed.

Which inference tools does it support?

The fixed-pass model is listed for Transformers, llama.cpp and MLX. The release also describes day-one SGLang support for datacenter deployments.

Are the benchmark results independently verified?

Not in the supplied material. The reported comparisons are claims from Thinking Machines Lab; the source does not provide independent replication or the full score table.

What does the research version change?

The TML-d24-varloop-research artifact lets users change the number of passes through the repeated middle layers. The company says fewer passes may improve speed, but could reduce quality; the size of that trade-off is not established here.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

The Impact Of AI On Incident Analysis Speed: Insights From NTT DATA Group

NTT DATA Group claims to cut incident analysis to 30 minutes with OpenAI Codex, but details on measurement and impact remain unclear.

The Next Generation of Aircraft Maintenance Is Powered by AI and Augmented Reality – i-hls.com

Next-gen aircraft maintenance now leverages AI and augmented reality for improved efficiency and safety, marking a major technological shift.

Automation ROI: The Spreadsheet That Stops Bad Projects

With this automation ROI spreadsheet, discover how to identify and eliminate unproductive projects—your key to smarter, resource-saving decisions.

Anthropic’s Chatbot Abuse Ban: How AI Experts View The Proposal

A headline says Anthropic wants to ban chatbot abuse, but the available material gives no policy text, definition, timeline or expert comments.