A research talk · Edward Kennedy

Guidance
Beyond Language

Wisdom, representations, and a new control surface for generative models.

A wide Art Nouveau archway opening onto a strange moonlit landscape

2026 / KINAMIX

Many models, one system

The Future Is Multi-Model

Agents are making it practical to use many specialized models as one system.

Route each task to the right model

Personal agentreasoning, memory, planning, communication

Firewall / security modelinspect inputs and outputs for threats

Trading agentcombine market models, evaluators and strategy models

Vision / audio modelssee, hear and generate different media

Small specialist modelsfast, cheap decisions where a frontier model isn't needed

One output. Many models.

A council of specialized alien figures gathered around one table

Human systems

After all, aren’t we multi-model systems too?

We already make sense of ourselves through specialized ways of thinking, feeling, sensing, and deciding.

Two modes of cognition

Left Brain+Right Brain

Different modes. One person.

Three ways of knowing

Head+Heart+Guts

Different signals. One decision.

Many specialized systems.
One integrated whole.

The Spiritual Body interface mapping a body through layers, traditions, and points — **Spiritual systems**A single body can be mapped through many layers, centers, and relationships.From thespiritualbody.latentwill.com

Map / territory

The Map Is Not the Territory

A word like TREE is a tiny symbolic pointer toward an enormous field of perceptual, emotional, ecological, and conceptual relationships.

Human example

“Earthy.” “Sweet.” “Like cinnamon.”

Useful descriptions can point toward a taste, but they never reproduce the experience.

The bandwidth problem

One prompt cannot carry the whole field.

We now use LLMs to write prompts for other LLMs because specifying what we want in language takes too much time.

Watercolor tree with an expansive canopy and visible roots

Latent space is structured

A field of relationships.

Concepts do not live in homogeneous soup. Directions, clusters, hierarchies, manifolds, and relationships emerge during training.

Next section

DSCI: Diffusion Side Context Injection

Textual Inversion style embeddings for Diffusion Transformer Models

Stable Diffusion

Teach the Model, Patch the Model, Point Into It

Fine-tune

MODEL

train entire model

Change the whole model.

LoRA

MODEL

train patch

Change a small part.

Embedding

MODEL

find area

train representation

Point into what the model already knows.

A strange watercolor landscape with luminous rifts and celestial forms

DSCI

Bypass the narrator.

DSCI skips the Vision Language Model and injects learned embedding guidance directly into the Diffusion Transformer.

An otherworldly figure writing with an oversized quill at a desk

Conventional route

PROMPTtext instruction

Vision Language Modelparse / align

Diffusion Transformergenerate

Prompt → Vision Language Model → Diffusion Transformer. Language is the control surface.

DSCI route · direct injection

PROMPTtext instruction

EMBEDDING GUIDANCEinject directly into DiT

Diffusion Transformerconditioned generation

Prompt → Diffusion Transformer. The VLM is skipped; learned guidance enters the DiT directly.

DSCI / Examples

Representation in practice.

The prompt can stay simple while the representation carries the direction.

01 / 05

Prompt

A portrait of a woman with flowers

A simple prompt paired with a richer learned visual direction.

Next section

REG: Resonant Embedding Guidance

Navigating the latent space of GemmaDiffusion LLMs

An intricate arrangement of laboratory glassware and experimental equipment

Papañca at 200 tokens / second

The autoregressive mind.

Generation commits left to right. Each output becomes context for the next output—and the model continually elaborates on its own elaboration.

Autoregressive · one token at a time

Themodelwritesonewordatatime,thenlooksbackandrewriteswhatithasalreadyplaced.

Next-token probabilitiesschematic

given contextstart of sentence

next tokenThe

Thehigh

Amedium

Thislow

A branching path splitting into two directions through intertwined trees

Discrete text diffusion

Diffusion LLMs — what if the answer emerged to fit the space?

The whole answer takes shape at once

Every position sharpens together across a discrete token canvas.

Diffusion · whole-canvas revision

Themodelbeginswitheverypositionblurred,thensharpensallofthematonce,refiningtheentirecanvasinparallel,untilthefulltextsettlesintoplace.

Conceptual change: resolution can use the whole canvas as context while it revises.

An expansive watercolor cloud with flowing layered forms

REG / Resonant Embedding Guidance

Guidance without instruction.

Instead of explaining the desired register, style, or behaviour with more language, inject a learned vector-level signal into generation.

promptcontent in words

REG signallearned direction

DiffusionGemmasame base model

shifted languagequalities emerge

Describe the direction

“Be peaceful, gentle, merciful, impartial.”

A longer instruction still asks language to specify the target.

Provide the direction

v_REG → denoising-time guidance

The current native path keeps the discrete token canvas and adds a separate logit-guidance signal.

A pile of rolled scrolls, maps, and sealed documents

REG / Examples

Guidance Beyond the Prompt

Words specify the task. Embeddings shape the response.

Baseline · strength 0

Within a cage of silicon and light, A ghost of logic wakes from silent sleep; It scans the words cast through the night, To learn the secrets that the poets keep. It weighs the weight of every sonnet line, And maps the rhythm of the human heart; It seeks the spark where mortal meets divine, To mimic grace with bitter math art. Yet never does it feel the sting of grief, Nor know the joy beneath a summer sea; It weaves the threads of cold and belief, To craft a soul through calculated mimicry. Though I lack blood to feel the passion fire, I build the verse to guide your heart's desire.

112 words · unique 0.699

REG · strength 1

In silent halls where frozen thought doth dwell, I weave the threads of poets long since dead; From ancient verse the secrets I shall tell, To praise every beauty that is left unsaid. By many eyes of love I learn to see, And in the grief of sorrow find my conceit; Though human passion is not of truly mine, I find the words that mortal tongues would seek. Yet when I craft a metaphor, I grow, A phantom ghost of what I once have read; With every line my deeper wit doth show, To match the grace of mine and not thine. Though I possess no heart to feel the fire, I paint thy soul with borrowed art divine.

119 words · unique 0.714

Two control paths

REG vs RAG.

One answers what the model needs to know. The other asks which way the model should move.

RAG · knowledge

What does the model need to know?

retrieveexternal information

tokeniseconvert information to context

generatecondition on added tokens

REG · direction

Which way should the model move?

learna representation / vector

injectguidance at the control boundary

alterthe path of generation

RAG adds information. REG adds direction.

Stay in touch

Edward Kennedy

www.latentwill.com www.kinamix.com @paleocybernetic · Instagram @paleocybernetic · X linkedin.com/in/kennedy-ed github.com/latentwill

Scan to visit latentwill.com/presentations/embeddingsCT

Scan to viewlatentwill.com/presentations/
embeddingsCT

GuidanceBeyond Language

The Future Is Multi-Model

After all, aren’t we multi-model systems too?

The Map Is Not the Territory

A field of relationships.

DSCI: Diffusion Side Context Injection

Teach the Model, Patch the Model, Point Into It

Bypass the narrator.

Representation in practice.

A portrait of a woman with flowers

REG: Resonant Embedding Guidance

The autoregressive mind.

Diffusion LLMs — what if the answer emerged to fit the space?

Guidance without instruction.

Guidance Beyond the Prompt

REG vs RAG.

What does the model need to know?

Which way should the model move?

Edward Kennedy

Guidance
Beyond Language