The Missing Layer Between Data and Prompts

May 31, 2026 1 min read

A small lesson from building with LLMs: when generated messages feel unnatural, the issue is often not the prompt or the model — it is how the underlying information is represented.

I recently ran into an interesting problem while experimenting with LLMs.

I was feeding a model everything I knew about a person:

  • connection date
  • status
  • labels
  • relationship history

The generated message was technically correct, but felt unnatural.

Things like:

"Now that we've been connected since November 2025..."

Nothing factually wrong.

Just not how humans talk.

Two types of information

After a few experiments, I realised I was mixing two different types of information.

System metadata

  • when we connected
  • labels
  • status

Conversation context

  • how we know each other
  • what we've worked on together
  • what we previously discussed

The first helps the system make decisions.

The second helps humans have conversations.

Once I separated the two, the generated messages became noticeably more natural.

The issue often appears one layer earlier

What struck me is how similar this feels to software architecture.

When AI systems produce poor results, many discussions jump straight to:

Data Prompt LLM Result Flow A vertical flow diagram showing data to prompt to LLM to result using the Shift Advisory color palette. DATA PROMPT LLM RESULT Data Prompt LLM Result Flow

and focus on:

  • better prompts
  • agents
  • orchestration
  • workflows

But the issue often appears one layer earlier:

Reality Representation Context Model Prompt LLM Result Flow A vertical flow diagram showing reality to representation to context model to prompt to LLM to result using the Shift Advisory color palette. REALITY REPRESENTATION CONTEXT MODEL PROMPT LLM RESULT Reality Representation Context Model Prompt LLM Result Flow

Modeling, not AI

In my case, the prompt wasn't the problem.

The model wasn't the problem.

The representation was.

I had treated system metadata and conversation context as if they were the same thing.

Once the representation became clearer, the behaviour improved.

The lesson wasn't about AI.

It was about modeling.