What is Prompt Architecture in LLMs?

From prompt engineering to prompt architecture

Summarize.

It started with a single word.
Not happy with the results, we tried again.

Summarize the most important points of the article.

Prompt engineering teaches us that more specific prompts are better.

Identify the three most important arguments made in the article and evaluate the strength of the author’s argument based on the evidence provided. Are there any points where you feel the argument could be stronger or more convincing?

Over time, we learned to include more details to guide our favorite LLMs to provide the best answers.

Prompt engineering techniques are becoming more complex and elaborate systems, sometimes made of many components. The definition of prompt engineering might be limiting in defining such intricate systems.

In this article, I want to propose a more accurate label for multi-component systems that interface with LLMs:

Prompt Architecture.

The history of prompt engineering

Modern language models have developed an impressive capacity to take on novel tasks after seeing only a couple of examples. This ability is called in-context learning, and it’s the main reason why we prompt engineering works so well.

Researchers think in-context learning works because pretraining teaches the model the general skills needed for language tasks. Then, at test time, it just has to recognize the pattern and apply its skills. Bigger models do this even better, making them surprisingly adaptable at various natural language tasks. [2]

In the past, you’d need thousands of labeled examples to fine-tune a language model for a new task. But with in-context learning, you can give the model the task description in its context window, and it…