This directory covers 11 companies and projects using recurrent, state-space, linear-attention, convolutional, associative-memory, and graph-based language architectures, with public information checked on July 31, 2026.
1. Liquid AI
Company or project: Liquid AI builds the Liquid Foundation Model family and the LEAP platform for adapting and deploying models on cloud, desktop, mobile, and embedded hardware. The company grew from research on liquid time-constant networks and closed-form continuous-time neural networks. Those earlier systems used state-dependent dynamics for time-series processing. The architecture inside the current language models differs from that research lineage. The LFM2 technical report describes a discrete, hardware-searched hybrid rather than an ordinary differential-equation solver.
Published architecture: LFM2 mixes gated short convolutions with a small number of grouped-query attention blocks. A convolutional block moves information along the sequence through learned local filters and gates. It can process a prompt in parallel and can be expressed as a compact recurrent update during generation. The attention blocks provide direct content-based access to earlier tokens at selected depths. Liquid used hardware-in-the-loop search to choose block layouts under measured latency and memory constraints. The search objective therefore includes execution measurements rather than relying only on parameter count or theoretical operation count. The report covers dense 350M, 700M, 1.2B, and 2.6B models, plus an 8.3B mixture-of-experts model with about 1.5B active parameters. The listed text models use a 32K context window.
Documented difference from dense transformers: A dense decoder transformer uses full causal attention in every block and stores keys and values from every earlier token for every attention layer. LFM2 routes most token mixing through gated convolutions. Those blocks avoid a growing KV cache during recurrent decoding. The remaining grouped-query attention layers still create a cache, so LFM2 does not have fixed memory for the full network. The design trades some exact token lookup for lower state and better execution on CPUs and edge accelerators. Its short convolutions also have a different bias from Mamba-style selective state spaces: they emphasize a bounded learned filter plus gating, with attention handling longer exact relationships.
The distinction between a convolutional block and an attention block also affects prompt processing. Convolutions apply a learned filter with regular local structure, while attention forms content-dependent relationships between token positions. The hybrid preserves both operations instead of claiming one mixer can provide every memory behavior.
Named artifacts and current status: Liquid publishes weights, a model report, model cards, quantized packages, and deployment integrations. Its official text-model catalog lists the supported LFM2 variants and their intended use. The company also documents integration paths for llama.cpp, ExecuTorch, vLLM, OpenVINO, AMD targets, and Qualcomm hardware. These artifacts support inspection through more than one runtime. Proprietary models or hosted services may expose features that the open report does not fully describe. The named downloadable checkpoints have the most complete public technical record.
Reported benchmark results and test conditions: Liquid reports training its dense models on roughly 10 to 12 trillion tokens and reports up to about two times faster prefill and decoding on CPUs than selected similarly sized models. The number is a vendor result. Data volume, quantization, CPU instruction set, thread count, compiler flags, prompt length, batch size, runtime, and sustained device temperature can all change the comparison. A 2.6B model can also be faster because it is smaller or more heavily optimized, even when architecture contributes. The cited report does not provide results for every device, quantization level, exact-copy task, citation task, structured-output schema, or tool-call workload.
API and commercial availability: The family is available through downloadable checkpoints. Liquid also offers LEAP for model specialization and deployment, with options aimed at local and managed use. The catalog spans text, vision-language, audio, and embedding products, but the architecture and terms can differ by model. Model IDs, quantization, tokenizers, and runtime versions identify distinct deployment configurations. The cited public material does not establish that every hosted demonstration uses the same checkpoint and kernel as every production contract.
Dated company and funding facts: Liquid AI announced a $250 million Series A led by AMD Ventures in December 2024. The announcement supports the amount, date, and named lead investor. It does not report model-quality measurements or future support commitments.
License and IP terms: Liquid says the LFM2 license is based on Apache 2.0 and adds a commercial threshold. The published terms allow commercial use for organizations below $10 million in annual revenue and require larger organizations to contact Liquid. The exact license bundled with a checkpoint controls its definitions for revenue, affiliates, hosted use, redistribution, and derivatives. Software integrations can use different licenses from the weights.
Documented limitations and unavailable facts: The remaining attention layers consume cache. Hardware support varies by model and precision. The custom commercial threshold applies after the stated revenue level. The public report does not establish long exact recall across the entire advertised context window.
Primary sources: LFM2 technical report, official model catalog, LFM2 release and license summary, and Series A announcement.
2. AI21 Labs
Company or project: AI21 Labs develops Jamba, a family of production language models that combines Mamba state-space layers, transformer attention, and mixture-of-experts feed-forward blocks. Jamba is one of the earliest company-backed attempts to turn a nonstandard language backbone into a hosted enterprise product. The family includes API models, private-deployment options, and a smaller downloadable checkpoint. Model generations and licenses differ, so “Jamba” is not one fixed package.
Published architecture: The original Jamba paper interleaves Mamba layers with transformer layers. Mamba uses an input-dependent state-space update that decides what information enters, stays in, or leaves a fixed-size state. The model can process training sequences through parallel scan-style operations and decode recurrently. Jamba adds occasional full-attention layers for exact content-based access. It can also route feed-forward computation through selected experts. Total parameters describe stored expert capacity, while active parameters describe the subset used for one token. The later Jamba 1.5 report describes a 52B-total, 12B-active Mini model and a 398B-total, 94B-active Large model.
Documented difference from dense transformers: A dense transformer performs attention and activates the same feed-forward parameters in every block. Jamba replaces most attention mixers with selective recurrent state and can activate only a subset of expert parameters. The Mamba layers keep constant-size recurrent state during token-by-token generation. The attention layers retain a smaller growing KV cache. This hybrid preserves direct retrieval at some depths while reducing memory compared with attention in every layer. Active parameter count lowers arithmetic relative to a dense model of the same total size, but all expert weights still need storage and placement.
Named artifacts and current status: AI21 publishes architecture papers, API documentation, a small downloadable model, model cards, and deployment information. The Jamba foundation-model documentation identifies hosted Mini and Large products and distinguishes the downloadable 3B model. Public evidence supports the hybrid structure and product access. It does not disclose the training data or provide all materials needed to reproduce every hosted efficiency claim. API aliases may change independently of dated model snapshots.
Reported benchmark results and test conditions: The original paper reports a configuration that fit a 256K context model on one 80GB GPU. Jamba 1.5 reports ExpertsInt8 deployment of the larger model on eight 80GB GPUs at 256K. These are system results from AI21, not universal memory guarantees. Expert placement, precision, batch size, runtime, and cache policy affect the result. AI21's official Jamba Reasoning 3B changelog entry reports two-to-five-times efficiency gains for the 3B model; the company's release measurements include 40 output tokens per second on an M3 MacBook Pro at 32K context. That result uses company-selected input, output length, quantization, sampling, and thermal conditions. Maximum context establishes accepted sequence length but does not establish retrieval quality across positions, similar keys, late corrections, or multi-hop joins.
API and commercial availability: Jamba Mini and Large are available through AI21's model platform and private deployment arrangements. The 3B checkpoint is intended for download and local use rather than the same first-party API path. Region, data retention, throughput commitments, version pinning, retirement notice, fine-tuning, and memory per concurrent long-context session depend on the service arrangement. A successful single 256K request does not establish a concurrency level.
Dated company and funding facts: AI21 announced a $208 million Series C in November 2023 through a company-supplied press release. The announcement identifies the round and investors. It does not report recall measurements or serving cost.
License and IP terms: Jamba Reasoning 3B is released under Apache 2.0. Older large Jamba weights have used the Jamba Open Model License, while API access is governed by service terms. The license attached to the exact artifact controls its use. Code, weights, fine-tuned derivatives, and hosted output can carry separate duties. Accepted license text and dates can therefore differ among distributions.
Documented limitations and unavailable facts: Attention layers still create a KV cache. MoE total weight storage includes inactive experts. Long-context retrieval quality is separate from successful context allocation. Family members have different licenses and access paths. Private deployment uses AI21-specific distribution and support terms.
Primary sources: Jamba paper, Jamba 1.5 report, AI21 model documentation, official AI21 changelog, and company-supplied Series C announcement.
3. Zyphra
Company or project: Zyphra develops Zamba2, a family of compact hybrid language models, and later Hybrid Associative Memory research. Zamba2 has downloadable base and instruction-tuned checkpoints at sizes below the multi-node requirements of frontier models. Its architecture, weights, code, and training-data mixture are publicly inspectable to a degree that closed APIs do not provide.
Published architecture: Zamba2 places Mamba-2 state-space blocks through the backbone and reuses two attention blocks in an alternating pattern. Mamba-2 connects structured state-space models with a restricted form of attention through state-space duality. During decoding, its state update compresses prior tokens into fixed-size state. The shared attention modules periodically recover direct global token access. Zamba2 also uses LoRA projections to let a reused feed-forward block behave differently at different depths. The Zamba2 report covers 1.2B, 2.7B, and 7.4B variants trained for up to three trillion tokens.
Documented difference from dense transformers: A dense transformer has separate attention and feed-forward weights at every layer. Zamba2 uses recurrent SSM blocks for most sequence mixing, shares a small number of attention and feed-forward modules across depths, and adds low-rank depth-specific changes. This cuts the share of the network that needs a growing KV cache. Weight reuse also changes parameter allocation because the same full module contributes at several depths while the low-rank adapters provide layer-specific variation. The model depends on custom recurrent kernels and carries compressed state between attention layers.
Named artifacts and current status: Zyphra released weights, instruction variants, code, model cards, and the Zyda-2 training mixture. The Zamba2-7B release describes training on 128 H100 GPUs for about 50 days. The official 7B instruction model card provides files, usage information, and license information. The published artifacts permit architecture inspection and local deployment. Zyphra's newer Hybrid Associative Memory work routes information between recurrent memory and attention-like precise storage, but it has fewer public product artifacts than Zamba2.
Reported benchmark results and test conditions: Zyphra reports 25 percent faster time to first token and 20 percent higher output-token throughput than selected 7B transformer comparisons. Those figures came from the vendor's runtime and hardware setup. Training data, prompt shape, precision, compiler, batch size, kernel maturity, compilation time, and fallback behavior can change the order. The released model permits independent runs, but the cited sources do not report every standard Transformers integration or production-server configuration.
API and commercial availability: Zamba2 is primarily a self-hosted model family. The public evidence does not show a broad first-party text API with published lifecycle commitments. The checkpoints run through community and custom runtimes. Autoscaling, security, monitoring, and updates are therefore outside a first-party managed text service in the cited materials.
Dated company and funding facts: Zyphra's official technical pages identify the company as the model developer and publish the hardware and duration used for the 7B training run. This entry does not use a funding total because the reviewed primary sources do not need one to establish model status.
License and IP terms: The named Zamba2 model card states Apache 2.0. Model cards, code repositories, tokenizers, and datasets can have distinct notices and terms. Apache licensing for weights does not grant rights to every document in a training mixture. Hybrid Associative Memory research can also carry separate implementation or patent considerations from Zamba2.
Documented limitations and unavailable facts: Runtime support depends on project-specific kernels and trust_remote_code paths. Shared modules can produce depth-dependent fine-tuning behavior. The published materials do not establish exact recall between attention layers across the full context window. A broad first-party managed API and public SLA are not listed.
Primary sources: Zamba2 report, Zamba2-7B release, official model card, and Hybrid Associative Memory research.
4. Moonshot AI
Company or project: Moonshot AI develops Kimi Linear and the frontier-scale Kimi K3 family. Kimi Linear supplies matched architectural experiments, open checkpoints, kernels, and serving support. Kimi K3 applies recurrent linear attention inside a very large mixture-of-experts model offered through weights and an API. Together they provide evidence from a smaller controlled research setting and a current production-scale system.
Published architecture: Kimi Delta Attention uses a delta-rule associative memory with fine-grained gates. Each token updates a matrix-like recurrent state that stores key-value associations. The gates control erasure and writing so a new association can replace an old value without clearing unrelated memory. Kimi Linear uses three KDA layers for each global multi-head latent attention layer. Moonshot trained its 48B-total, 3B-active release on 5.7 trillion tokens. The matched research design compares KDA with other mixers inside otherwise aligned model recipes. The Kimi Linear repository includes the report, weights, a custom kernel, and vLLM work.
Kimi K3 scales the pattern. Its official repository and report specify 2.8 trillion total parameters, 104 billion active parameters, 93 layers, 69 KDA layers, 24 gated multi-head latent-attention layers, 896 experts, and 16 selected experts per token. The model adds attention over residual streams and quantization-aware training with MXFP4 weights and MXFP8 activations. The published context window is one million tokens.
Documented difference from dense transformers: Most Kimi blocks replace softmax attention and its per-token KV storage with fixed-size associative state. Periodic global attention layers preserve exact source access and still grow a cache. Multi-head latent attention compresses that cache relative to conventional multi-head attention. MoE routing activates a fraction of the feed-forward weights for each token. This reduces arithmetic compared with a 2.8T dense model, while the stored weights and interconnect requirements remain far beyond a consumer machine.
Named artifacts and current status: Moonshot publishes architecture details, weights, code, kernels, serving instructions, and API access. The artifacts make the sequence mixer inspectable. The frontier training data and full production system remain partly closed. K3's July 2026 report gives layer counts, numeric formats, and serving guidance. The hosted model is available by API, and the weights support vLLM, SGLang, or TokenSpeed subject to substantial hardware requirements.
Reported benchmark results and test conditions: In Moonshot's matched comparisons, Kimi Linear reports 51.0 on MMLU-Pro at 4K, 84.3 on RULER at 128K, and about a 3.98-times speedup at that RULER length. The repository reports up to 6.3-times faster time per output token than multi-head latent attention at one million tokens. Matched training makes these more informative about architecture than unrelated leaderboard rows, but the model authors ran the tests. Kernel, GPU, precision, output length, and batch still define the result.
K3 benchmark tables combine first-party runs, third-party results, different tool harnesses, and different reasoning settings. They measure the complete model recipe, including scale, data, post-training, tools, and inference protocol. They do not show that KDA caused a final task score. RULER and needle tests do not cover every level of many-key interference, conflicting updates, distractor similarity, or long multi-hop reasoning.
API and commercial availability: Moonshot provides OpenAI-compatible and Anthropic-compatible APIs. Open weights allow self-hosting on distributed hardware. K3 requires preserved reasoning history in supported multi-turn workflows, so dropping prior reasoning content can alter behavior. Cached and uncached prompts have different service conditions. Regional availability, data handling, snapshot pinning, and equivalence between API aliases and released checkpoints depend on the service documentation and agreement.
Dated company and funding facts: The official Kimi repositories and technical reports identify Moonshot AI as the developer and document the release dates and model configuration. This entry omits outside estimates of company valuation or funding because those figures are unnecessary for the technical comparison and were not verified through a matching primary announcement.
License and IP terms: Kimi Linear code uses MIT terms, while model artifacts can state separate terms. The Kimi K3 license grants broad use and adds conditions for some model-as-a-service businesses and commercial products above specified revenue or scale thresholds. It contains clauses covering attribution, hosted resale, revenue, user scale, and derivatives. An MIT kernel license does not replace the weight license.
Documented limitations and unavailable facts: The 2.8-trillion-parameter model requires distributed storage and serving. Associative state can overwrite similar keys. Attention layers still consume cache. Performance depends on custom kernels and supported hardware. The API requires preserved reasoning history for specified multi-turn workflows. One-million-token allocation does not establish recall at every position.
Primary sources: Kimi Linear repository and report, Kimi K3 repository and technical report, and Kimi K3 license.
5. MiniMax
Company or project: MiniMax built MiniMax-Text-01 and MiniMax-M1 with Lightning Attention, full attention, and mixture-of-experts layers. The company later returned its M2 flagship family to full attention. The company published a scaled alternative, operated it, and then described deficits found through internal evaluation.
Published architecture: MiniMax-Text-01 has 456 billion total parameters and about 45.9 billion active parameters. Seven of every eight sequence-mixing blocks use Lightning Attention, while one uses full softmax attention. Lightning Attention rearranges attention into a recurrent or blockwise form so processing scales linearly with sequence length and decoding uses bounded state for those layers. Full-attention blocks retain direct lookup. The MiniMax-01 report distinguishes training on one-million-token sequences from inference extrapolation to four million tokens; those two context claims use different evidence. The MiniMax-M1 report adds reasoning post-training and retains a native one-million-token context.
Documented difference from dense transformers: A dense transformer applies full softmax attention throughout. MiniMax-01 and M1 send most sequence mixing through recurrent linear attention and reserve periodic full attention for exact access. MoE routing selects a subset of feed-forward experts. The hybrid cuts KV-cache growth across most layers but preserves a smaller cache in the full-attention layers. Its finite recurrent state can suffer interference during long or complex chains.
Named artifacts and current status: MiniMax published technical reports and weights for M1. Its API documentation has listed hybrid model identifiers, while newer product guides emphasize the full-attention M2 family. Public documentation does not state current regional availability and retirement terms for every older snapshot. MiniMax's official engineering note, “Why Did MiniMax M2 End Up as a Full Attention Model?”, says the earlier hybrid looked competitive on MMLU, BBH, MATH, and LongBench but showed deficits on complex multi-hop reasoning. It also describes less mature linear-attention infrastructure and failed attempts to convert a trained model to sliding-window hybrids without long-context degradation.
Reported benchmark results and test conditions: The reports give long-context and reasoning scores for finished models. Different data, post-training, scale, and harnesses prevent a clean architecture conclusion. MiniMax's engineering report directly documents its decision. It states that standard scores failed to expose some multi-hop deficits. A single needle-in-a-haystack task measures exact retrieval under narrower conditions than joins among remote facts or resolution of a late correction. The public report does not publish results across every combination of key count, reasoning depth, and output length.
API and commercial availability: Hybrid MiniMax checkpoints and model references have been publicly available, but the strategic product line has moved. Current endpoint availability, dated snapshots, weights, tokenizers, regions, data policies, and notice periods vary by model. An active endpoint does not establish a future support period. Newer product material centers on M2 rather than the earlier hybrid family.
Dated company and funding facts: MiniMax's official papers and platform note establish the company as the developer and document the architectural change. This entry does not repeat press estimates about funding or valuation because a primary company source was not required to evaluate the model.
License and IP terms: The weights, inference code, paper, and API are separate artifacts. The license attached to each MiniMax-Text-01 or M1 checkpoint and the service terms for each endpoint control their respective use. A public repository or downloadable weight does not by itself establish Apache-style rights. Redistribution, derivatives, output, data use, and geographic restrictions depend on the exact artifact and service terms.
Documented limitations and unavailable facts: MiniMax moved its current flagship family from the earlier hybrid architecture to full attention. Its architecture note reports multi-hop reasoning deficits and less developed linear-attention infrastructure. Linear-attention kernels can remain memory-bound. Current endpoint retention terms for older hybrid snapshots are not publicly listed.
Primary sources: MiniMax-01 report, MiniMax-M1 report, and MiniMax M2 architecture note.
6. NXAI
Company or project: NXAI develops xLSTM, a modern recurrent architecture based on extended long short-term memory. The company publishes architecture and training code and released a 7B base language model. Its commercial work also includes time-series products, while the public language checkpoint is better suited to research and self-hosted adaptation than a plug-in chat API.
Published architecture: The xLSTM paper defines two main memory forms. sLSTM uses scalar memory with exponential gating and memory mixing. mLSTM uses a matrix memory that stores key-value-like associations. Exponential gates expand the range over which the network can write, retain, and forget information, while normalization controls numerical growth. These additions address limited memory capacity and gate saturation in conventional LSTMs. The xLSTM 7B paper scales the design into a recurrent base language model that keeps constant memory during decoding.
Documented difference from dense transformers: xLSTM carries history in recurrent cell state rather than storing keys and values for every previous token. The state size stays fixed as generation grows. Matrix memory gives the model associative storage without full pairwise softmax attention. Training uses parallel formulations and custom kernels to avoid the slow step-by-step training associated with older LSTMs. Exact facts still have to survive repeated state updates, so retrieval can degrade as information competes for finite memory.
Named artifacts and current status: The official xLSTM repository contains architecture code, training components, and license information. NXAI's 7B release announcement identifies the checkpoint as a pretrained base model and discusses edge, embedded, robotics, and industrial uses. The released artifact supports inspection and benchmarking. It is not a managed general-purpose LLM API. Chat alignment, safety behavior, and tool calling would require separate post-training components that are not part of the base checkpoint.
Reported benchmark results and test conditions: NXAI reports downstream results comparable with selected models of similar size and faster inference than tested Llama and Mamba systems. These are author-run model and system comparisons. Training data, token count, kernel, GPU, precision, batch, prompt length, and output length affect them. Constant-state decoding changes the cost curve most at long outputs, while mature transformer kernels can produce lower latency at short lengths. The cited results do not report every output length or a same-data domain fine-tune comparison with task acceptance and schema validity.
API and commercial availability: The public path is downloadable code and model materials. NXAI does not document a general hosted xLSTM chat endpoint in the cited sources. Inference servers, autoscaling, adapters, moderation, and updates are not included in that public path. The base model also lacks the instruction or task tuning used by most chat applications.
Dated company and funding facts: NXAI's official announcement identifies the model team and the company's focus on industrial, edge, and sequence applications. The cited primary technical materials do not provide a funding figure.
License and IP terms: The repository code uses Apache 2.0. NXAI distributes model materials under a separate agreement with attribution requirements. Files shipped with the exact checkpoint define its terms. A repository badge can describe code while weights and datasets use different terms. Commercial adaptation, redistribution, attribution placement, and derivative-model duties can therefore differ by artifact.
Documented limitations and unavailable facts: The released 7B artifact is a base model without a first-party general chat API in the cited materials. The runtime depends on specialized kernels. Reduced precision can affect recurrent state, and finite associative memory can produce collisions. Public support for adapters, quantizers, state serialization, and production serving is limited.
Primary sources: original xLSTM paper, xLSTM 7B paper, official repository, and NXAI release announcement.
7. RWKV
Company or project: RWKV is an open recurrent language-model project with commercial use paths rather than a single conventional model vendor. It publishes architecture papers, training and inference code, checkpoints, and data-component records. The project operates under LF AI & Data; the Linux Foundation's 2023 annual report records RWKV as a hosted project. Companies can self-host its weights or build support around them.
Published architecture: RWKV combines transformer-like parallel training with recurrent weighted key-value updates at inference. Earlier Eagle and Finch versions refine receptance gates, token shifts, and decay. The RWKV-7 paper adds vector-valued gating, an in-context learning rate, and a generalized delta rule. These mechanisms update a compact state that stores and revises associations. The recurrent inference form consumes one token at a time, while the training form exposes sequence-level parallelism. The project reports checkpoints from 190M to 2.9B parameters trained on a 3.1-trillion-token multilingual corpus. Earlier Eagle and Finch work describes models up to 7.5B and shows that prompt templates materially affect results.
Documented difference from dense transformers: RWKV does not keep a per-layer KV record for every earlier token. It compresses history into recurrent state and performs constant work per generated token for a fixed model size. Training can evaluate sequence operations in parallel, which removes a main scaling problem of classic RNNs. The cost is irreversible compression: once a detail leaves state, a later token cannot attend back to its source. Vector gates and delta-style writes improve memory control but do not make finite state unlimited.
Named artifacts and current status: RWKV has open code, weights, papers, community runtimes, and an institutional open-source home. These materials document the architecture and released artifacts. Commercial support is fragmented across the project and third parties. Mainstream serving libraries tend to optimize transformer shapes first, so a nominally supported RWKV model can take a slower fallback or require a fork. Checkpoint and runtime support vary independently of the project name.
Reported benchmark results and test conditions: RWKV papers report language tasks, scaling behavior, memory claims, and system comparisons. The project authors produced those results. Corpus composition, prompt template, model size, quantization, runtime, and hardware can dominate a cross-family comparison. Constant theoretical decode cost can still produce higher latency at short lengths because GPUs run large attention matrix operations efficiently. The cited papers do not establish a universal crossover length or report every combination of exact recall, similar identifiers, long distractors, uninterrupted generation, and multi-hop joins.
API and commercial availability: The documented access path is self-hosted weights and community-maintained inference. Some third parties offer hosted RWKV access, with provider-specific models, SLAs, security controls, and snapshot policies. No single vendor owns project-wide uptime or compliance. Embedded deployments can use pinned checkpoints without dependence on a first-party hosted endpoint.
Dated company and funding facts: RWKV is a Linux Foundation-hosted open project, a fact supported by the foundation report. This entry does not present startup funding because the project is not organized as one ordinary venture-backed vendor in the cited evidence.
License and IP terms: The project states Apache 2.0 terms for RWKV-7 code and weights. Each checkpoint and derivative model card can state its own license. Training datasets, tokenizers, chat fine-tunes, and third-party runtimes can impose separate terms. Open code does not provide a support warranty.
Documented limitations and unavailable facts: Quantization, speculative decoding, batching, adapters, and observability depend on project and third-party runtimes. Recurrent state requires an explicit reset between sessions. Reduced precision can accumulate state error. A single project-wide commercial SLA and hosted snapshot policy are not publicly listed.
Primary sources: RWKV-7 paper, Eagle and Finch paper, and Linux Foundation annual report.
8. Cartesia
Company or project: Cartesia builds real-time speech products based on state-space-model research. Sonic generates speech from text, and Ink transcribes speech. The products apply a nonstandard sequence architecture to maintained generative APIs. Speech quality and streaming latency are different tasks from long-document reasoning.
Published architecture: Cartesia's founders worked on Mamba and related state-space methods. The company describes its products as SSM-based. An SSM moves a compact latent state forward as audio or text tokens arrive. Input-dependent gates can preserve relevant history and discard noise. Streaming speech is well matched to recurrence because the model receives an ordered signal and must emit output incrementally. The public product material does not disclose a complete Sonic layer diagram, parameter count, training set, or the contribution of its vocoder.
Documented difference from dense transformers: A full-attention speech model compares each new step with stored keys and values from the preceding sequence. An SSM processes the stream through a bounded recurrent state, reducing cache growth and supporting incremental generation. It can begin output without revisiting the whole history at every step. Speech also has strong local continuity, so the sequence task differs from exact retrieval across a long document. Public Cartesia material attributes the products to SSMs without disclosing how much computation occurs in other service components.
Named artifacts and current status: Cartesia offers a commercial cloud API, WebSocket streaming, dated model snapshots, and on-premises options. The official product site establishes current product access. Its documentation also records breaking changes and retirement schedules. The 2026 API migration page says older Sonic snapshots and voice-embedding endpoints retired on June 1, 2026. API availability is directly testable. Architecture attribution is less reproducible because the weights and full report are closed.
Reported benchmark results and test conditions: Cartesia markets low first-audio latency and high throughput. Those measurements describe a full service path, including the acoustic model, vocoder, network region, streaming protocol, and serving stack. They do not isolate the SSM. First byte is also different from first intelligible word or a completed conversational turn. The cited sources do not provide complete p50 and p95 results for time to first intelligible audio, interruption recovery, pronunciation, speaker similarity, packet jitter, or drift over ten minutes across client regions and text-normalization settings.
API and commercial availability: Sonic and Ink are available through commercial APIs, and Cartesia offers enterprise or self-hosted deployment paths. Weight-version pinning, supported GPUs, concurrency, telemetry handling, and voice migration after snapshot retirement depend on the deployment terms. Dated model identifiers and a moving “latest” alias can refer to different service configurations.
Dated company and funding facts: Cartesia announced a $64 million Series A in March 2025. The announcement identifies the funding amount and company focus. It does not report speech-quality results.
License and IP terms: Sonic and Ink are proprietary services. API and enterprise agreements govern generated-audio rights, input retention, model training use, voice cloning, consent, publicity rights, biometric rules, subprocessors, deletion, indemnity, and snapshot retirement. On-prem delivery can have different restrictions from cloud access.
Documented limitations and unavailable facts: Sonic weights, parameter count, training set, full layer diagram, and vocoder contribution are not publicly listed. API latency includes network and service time in addition to model execution. The 2026 retirement schedule shows that model snapshots and voice-embedding endpoints can be withdrawn. Long-duration drift results are not publicly listed.
Primary sources: Cartesia product site, Sonic API changes, and Series A announcement.
9. Together AI and Arc Institute
Company or project: Together AI released StripedHyena language checkpoints and hosted a general chat model, then carried the architecture into biological sequence modeling with Arc Institute and Stanford through Evo and Evo 2. The general endpoint was retired. The Evo 2 genomic project remains active in the cited repository.
Published architecture: StripedHyena alternates gated long-convolution blocks with grouped-query attention. A Hyena block uses implicit long filters and data-dependent gates to mix tokens across a large receptive field. During training, convolution can use parallel frequency-domain operations. During autoregressive generation, the filter can be represented through recurrent state. StripedHyena 2 adapts that family to DNA sequences, where tokens represent biological sequence rather than natural-language words. The Evo 2 repository lists 1B, 7B, 20B, and 40B-class artifacts and supports sequences up to one million base pairs for named models.
Documented difference from dense transformers: Long convolutions replace most pairwise attention with learned filters whose work scales close to linearly with sequence length. Grouped-query attention remains in selected blocks, which restores exact access and creates some KV cache. The model's inductive bias favors patterns that repeat or extend across a sequence. Genomic input makes this attractive because a one-million-base region is a normal scientific object and every position has local biochemical structure.
Named artifacts and current status: Together released StripedHyena-Nous-7B under Apache 2.0, with a model card that documents architecture and numerical constraints. Together's official deprecation page records the removal of its serverless StripedHyena endpoint in August 2024. Evo 2 has an active official repository, model packages, training guidance, and NVIDIA-hosted or self-hosted routes. The architecture survived, while the general-purpose distribution path changed.
Reported benchmark results and test conditions: StripedHyena papers and cards report constant-memory decoding for Hyena blocks and compare language quality and throughput with selected transformers. Evo 2 reports genomic sequence tasks at very long context. These tasks do not predict chat quality. Scientific sequence likelihood does not prove biological function, treatment value, or safety. The model card warns that poles and residues must remain in 32-bit precision for long prompts, showing that quantized weights do not make every internal state safe to quantize. Runtime results therefore depend on the required precision and specialized kernels.
API and commercial availability: The general StripedHyena serverless model is no longer available from Together's documented endpoint. Its weights remain downloadable for self-hosting. Evo 2 offers official code and links to hosted access through NVIDIA services, plus local deployment paths for suitable hardware. The installation requires Linux, recent CUDA, and specialized packages; the 7B path has lower hardware requirements than the largest releases. Hosted models and terms can change independently of the downloadable artifacts.
Dated company and funding facts: The Evo 2 repository identifies Arc Institute, Stanford, and Together AI as project contributors and maintainers. This entry does not include venture-funding totals because the project sources already establish the collaboration and access path.
License and IP terms: The StripedHyena checkpoint and Evo 2 code state Apache 2.0 terms in their respective artifacts. Model weights, data, hosted endpoints, and downstream biological inventions can have additional terms. Scientific or clinical use also creates privacy, biosecurity, validation, and regulatory duties that a model license does not cover.
Documented limitations and unavailable facts: Together AI retired the StripedHyena chat endpoint. Specialized kernels and FP32 recurrent state limit runtime support. Reported genomic benchmarks are computational results and do not establish wet-lab validity. Large checkpoints require multi-GPU hardware. Clinical validation and regulatory status are not publicly listed for the research models.
Primary sources: Evo 2 repository, StripedHyena model card, and Together model deprecations.
10. DeepSeek
Company or project: DeepSeek published Engram, a research architecture that adds deterministic n-gram memory to a mixture-of-experts language model. Engram is an active company research project rather than a supported DeepSeek model family. The work studies whether static lexical knowledge can use a lookup table instead of consuming neural compute every time the model sees a familiar phrase.
Published architecture: Engram normalizes token n-grams, hashes them into large embedding tables, retrieves candidate vectors in constant time, and uses context to gate the retrieved information into the model. Large tables can live in host memory and be prefetched while the accelerator runs neural layers. The memory address is determined by the normalized n-gram rather than generated by a learned document-retrieval query. The Engram paper studies how to divide a fixed sparse-parameter budget between conditional MoE computation and static memory. The official repository provides a demonstration implementation and report.
Documented difference from dense transformers: A dense transformer stores factual and lexical patterns only in learned weights and recomputes activations through every layer. Engram moves some repeated, discrete patterns into a deterministic lookup table. It does not replace the entire transformer: the neural backbone and attention remain, while the memory module changes how parameters and bandwidth are allocated. Unlike external RAG, the lookup occurs inside the model and uses token n-grams rather than a separate document retriever.
Named artifacts and current status: The paper, repository, implementation sketch, and reported ablations establish a concrete architecture. The public repository does not provide a maintained full-scale checkpoint, priced Engram endpoint, production serving image, or support policy. The cited documentation does not establish that DeepSeek's existing APIs use Engram. This is a research-stage entry.
Reported benchmark results and test conditions: The Engram paper reports gains under fixed parameter and FLOP budgets and studies host-memory offload. Matched budgets help isolate the allocation choice, but the authors ran the experiments. Hashing quality, n-gram vocabulary, table size, host bandwidth, PCIe or interconnect behavior, prefetch hit rate, language mix, and sequence distribution can change the result. A FLOP match does not include all memory traffic. The reported end-to-end latency and power conditions depend on the memory tier used for the table.
API and commercial availability: The cited sources offer research code and a small demonstration, not a supported commercial Engram endpoint or full released model. Any independent implementation would require separate training, kernels, serving, operations, and rights analysis. No public production service or commercial distribution path is listed.
Dated company and funding facts: DeepSeek's official repository and paper identify the organization and research team responsible for Engram. This entry omits funding and valuation claims because they do not affect the project's current lack of a deployable Engram product.
License and IP terms: The official repository releases the demonstration code under Apache License 2.0. Any future model card could state separate terms. A paper explains a method but does not grant rights to weights, data, trademarks, patents, or a hosted service. Deterministic lookup tables can reproduce memorized strings. The cited materials do not publish production procedures for data provenance, removal, or copyright handling.
Documented limitations and unavailable facts: A production package, general API, public checkpoint, and commercial SLA are not publicly listed. Hash collisions and normalization choices can retrieve an unintended memory entry. Large tables move cost from arithmetic to storage and host bandwidth. Static n-grams encode lexical patterns rather than general compositional reasoning. Published update and deletion mechanisms for individual learned entries are not listed.
Primary sources: Engram paper and official Engram repository.
11. Pathway
Company or project: Pathway published Baby Dragon Hatchling, also called BDH, and later company material refers to Dragon Hatchling. The project explores a graph-like language model with locally interacting neuron particles and adaptive synapses. Pathway is an operating data-processing company, but BDH remains architecture research without a downloadable production checkpoint or general text API.
Published architecture: The BDH paper describes sparse positive neuron activations, local interactions on a learned graph, and working memory expressed through synaptic plasticity. The system updates internal connections as it processes context. Its GPU implementation studies models from roughly 10 million to one billion parameters. The temporary state resides in changing synaptic connections rather than a transformer's stored sequence of key and value vectors. The local graph also changes the computation pattern from regular dense matrix operations.
Documented difference from dense transformers: A transformer stacks blocks that apply global attention and feed-forward transformations to token vectors. BDH represents computation as local activity among particles connected in a graph. Context changes its temporary synaptic state. Sparse local activation aims to limit how much of the system works at each step and gives the model a native changing memory. Exact implementation and scaling behavior differ enough that standard transformer serving kernels do not apply.
The state in BDH is part of the model's changing connection structure during context processing. That differs from a dense transformer's KV cache, which stores past layer projections without changing the trained weights. It also differs from Engram's static hash table, whose addresses come from token n-grams.
Named artifacts and current status: The preprint, project material, and Pathway's company page about its AWS collaboration show active company research. The paper reports GPT-2-scale language experiments. The cited public materials do not provide a maintained production weight, priced endpoint, standard server, safety report, or support commitment. The architecture has published technical evidence but no public commercial access path.
Reported benchmark results and test conditions: BDH reports scaling experiments against GPT-2-like baselines up to one billion parameters. That scale does not establish frontier reasoning, long-context reliability, tool use, or production economics. Comparisons depend on training data, tokens, optimizer, parameter accounting, sparsity, and GPU implementation. Sparse activation can lower arithmetic while graph scheduling and memory access raise wall-clock cost. The cited results do not provide complete accelerator-utilization, host-work, state-size, batch-scaling, power, or accepted-task-quality measurements.
API and commercial availability: Pathway's core enterprise data-processing products are commercial, but BDH is not listed as a product. The cited collaboration page signals ongoing work rather than general availability. No stable model identifier, downloadable weights, API, deployment requirements, versioning policy, support terms, or public evaluation suite are listed. A custom research partnership could use private terms without establishing a public product.
Dated company and funding facts: Pathway's official company page establishes its role, enterprise focus, and AWS collaboration around the research. This entry omits funding totals because the primary sources used here do not need them to establish project status.
License and IP terms: A preprint grants no automatic right to commercial weights, code, training data, or patents. The terms for Pathway's existing software do not automatically cover BDH. No separate public BDH repository license is cited here. Terms for a private evaluation could address retained state, input use, derivative research, publication, and deletion.
Documented limitations and unavailable facts: A public production model, hosted API, commercial SLA, and large-scale checkpoint are not listed. Published evidence covers modest model scale. Dynamic synaptic memory requires session-state isolation and serialization, but production mechanisms for those operations are not published. Sparse graph execution has different hardware characteristics from dense matrix kernels.
Primary sources: BDH paper and Pathway's official project and AWS collaboration page.