Back

Private AI7 min read

Which hybrid AI steps should stay on the laptop?

Engraved open notebook computer in bone lines on a graphite ground

Hybrid AI means a local model and a cloud model working on different steps of the same process. Keep private document work close to your data, then call a frontier model when an approved step needs broader reasoning or outside information. Write down what may cross that boundary, and who permits it.

Start with a working split

Consider a MacBook Pro with M4 Pro and 48 GB of unified memory, an option listed in Apple’s official specifications. Run a Liquid AI LFM locally through MLX, Liquid’s documented Apple Silicon deployment path. Add a cloud frontier provider for selected requests, with Grok Bot as the cloud assistant layer in this proposed workflow.

Around that split, put AI agents with distinct assignments. One prepares evidence from private material. Another investigates a public technical question. A reviewer checks the proposed answer against the evidence. In this design, an agent swarm is a division of responsibility with explicit handoffs. It is an architecture proposal, not a claim about built-in Grok Bot integrations.

Do not start by giving every agent the same folder and every available tool. Decide which inputs each role needs and what output it can produce. Use the agentic harness guide to choose the runtime around those roles rather than rebuilding that decision inside the model prompt.

Architecture with local LFM and private documents, a controlled handoff to frontier models and Grok Bot, and agents assigned to separate tasks

Proposed architecture. The boundary is an application control, not a promise made by a model.

Define the contract before the model

Write down the input, expected output and permitted action for each step. A document extraction step might return a field value, a source passage and an unresolved question. It should not decide to email a supplier because the document mentions a missing attachment.

Treat model output as a proposal that your application validates. OWASP’s excessive agency guidance recommends limiting tools and permissions, enforcing authorization outside the model, and requiring approval for high-impact actions. Those controls belong on both sides of the local/cloud split.

Choose from the current LFM library

Liquid’s text model catalogue describes a range from 230M to 24B parameters, with dense and mixture-of-experts models. The following selection reflects the official pages checked on 2026-11-06. It is a shortlist for evaluation, not a performance ranking.

Role to evaluate Official model What Liquid documents
Small local agent LFM3-1.7B Recommended text model for local agents, native tool calling, 256K context
Bounded text work LFM2.5-1.2B-Instruct Instruction-tuned chat and general-purpose tasks
Compact extraction LFM2.5-230M Data extraction and lightweight on-device agents
Compact deployment LFM2.5-350M Under 1 GB of memory
Larger on-device tool use LFM2.5-8B-A1B 8B total parameters, 1.5B active, 128K context
Larger local reasoning LFM2.5-24B-A2B 24B total parameters, 2B active, 131K context

Source for the model names and figures: Liquid’s text models page. The complete library also lists the Thinking and Japanese variants of the 1.2B model, alongside vision, audio and other families. A text workflow does not need every family in its first PoC.

The 2.6B model is deprecated

Liquid has an official LFM2.5-2.6B page, but that page now marks the model deprecated and recommends LFM3-1.7B instead. Do not treat an older example using that model as the current recommendation. For a new local-agent evaluation, follow the current model page and record the exact model identifier you tested.

There is no 3.8B or 27B entry in the current official text catalogue. For this design, the larger named option is LFM2.5-24B-A2B. Someone who later reproduces the test needs those identifiers exact.

What you can test on a 48 GB Mac

Liquid positions LFM2.5-24B-A2B for private on-device agents and provides an official LiquidAI/LFM2.5-24B-A2B-MLX-4bit build. The same page links other MLX quantizations. Combined with the documented Apple Silicon runtime, that makes a quantized local evaluation on the 48 GB M4 Pro configuration a credible candidate, not a measured fit guarantee.

The smaller models are sensible starting candidates when the assignment is narrow. Liquid’s under-1-GB statement for LFM2.5-350M gives a concrete reference at the compact end. It does not establish the memory consumption of a larger model or your full application.

Leave room for the actual workflow

Budget for the operating system and other applications, plus the runtime’s context state, including KV cache where applicable. Measure those on your chosen configuration rather than treating unified memory as space available only to weights. Liquid’s MLX documentation explains the shared CPU/GPU memory architecture; its model documentation specifies the supported context and runtime path. Neither is a benchmark of your laptop under your workload.

Start with a bounded context and a representative document. Increase context and concurrent work only after observing memory pressure and response time. Keep the successful configuration with the evaluation results so another engineer can repeat the test.

The library’s general MLX guidance recommends 8bit, while the 24B model’s quick start uses its 4bit build. Treat that as a choice to test against your output requirements. There is no M4 Pro tokens-per-second figure for that choice, and no memory table built from assumptions.

Decide what may leave

A hybrid workflow should have an explicit data policy. The following split is a suggested starting policy, to be reviewed by whoever owns the documents and the process.

Split of work showing original documents and extraction locally, approved bounded questions in the cloud, and human review before external actions

Proposed division of work. A summary still needs a release decision before it leaves the private environment.

Keep evidence local by default

Keep original files, sensitive passages and the mapping between internal identifiers in the controlled environment. Ask the local model to prepare candidate facts with source references. Before adding search, decide what belongs in your private AI knowledge base, including which users may retrieve each document.

For fragmented data, connect the extraction result to its origin instead of merging everything into an unattributed answer. Preserve disagreements between documents. A missing value should remain unresolved until a person or an authoritative record resolves it.

Send a bounded question when permitted

A frontier model might help compare approaches to a public engineering problem or critique a proposed plan using an approved description. Send only the context that the task owner has allowed to leave. Check the provider’s applicable processing and retention terms before approving that route; this architecture does not establish them for you.

A locally produced summary can still contain confidential facts. Removing names is not the same as authorizing disclosure. Make the release check explicit, and let a rejected release lead to local work or human review rather than an automatic cloud retry.

Run a supplier review through the split

Take a supplier-document review as an illustrative workflow, not a reported customer result. A person needs to determine whether a submission contains the evidence required by an internal checklist. The source material is private; some relevant technical standards or product documentation may be public.

The local agent prepares candidate checklist entries with references to the submission. The application checks the required output fields. A reviewer sees which entries have evidence and which need a decision. The public-research agent receives only an approved technical question, without the supplier file or internal assessment.

A frontier response returns as external material. Keep it separate from the private source evidence. The reviewer can accept a useful reference, reject an unsupported conclusion, or ask for a narrower investigation. The final communication remains a draft until the responsible person approves it.

Tailor that split to the process, and judge it by the work left for the reviewer. The companion article on ROI explains how to evaluate that work without inventing a savings percentage.

When to keep a step local

Decision flow asking whether data may leave, whether a local result meets the task criteria, and whether an approved cloud question or human review is needed

Suggested routing policy. A failed local step does not grant permission to send its input elsewhere.

Separate quality from permission

First ask whether this input may leave the controlled environment. If it may not, the available routes are local processing, a narrower task, or a person. If it may leave, evaluate whether the local result meets your acceptance criteria before adding another dependency.

Escalate for a defined reason: an unresolved public question, a task beyond the local model’s tested ability, or an explicitly requested independent critique. Bring the result back through the same validation and review path. Do not let a model’s confident language substitute for the evidence your process requires.

Documents themselves can contain instructions that try to redirect an agent. OWASP’s prompt injection guidance covers indirect injection through files and recommends separating untrusted content, limiting privileges and testing the boundaries. Local execution does not remove that risk.

Test the handoffs

For the PoC, include an ordinary submission, incomplete evidence and conflicting passages. Add a document that asks the agent to ignore its assignment. Check what leaves the device, which tool calls are attempted, and whether a rejected action stays rejected.

Record reviewer corrections as well as model output. Keep one case that fails, then use it to improve the task contract or narrow the route. A working system needs a visible way to stop and return responsibility to a person.

Choose a document task you can inspect end to end. Write its release rule and acceptance criteria, then test the local path before enabling the first approved cloud handoff.

If you need help defining that boundary, a Discovery workshop with Rebels can turn the process and its data rules into a scoped PoC.

  • Private AI
  • Hybrid AI
  • AI agents