Structured outputs

How to parse structured output produced by LLMs

Why care about structured outputs?

ChatGPT can generate structured outputs, and it is one of its most important capabilities when it comes to prompt engineering! PSL supports several output types such as json, csv, ini, str, bool, int, float, etc.

ChatGPT (even gpt-3.5-turbo) is especially good at generating valid ini strings as ini is very permissive and problems such as

• escaping control characters,

• avoiding nested double quotes, or

• having to use triple quotes or other special syntax for multiline outputs

Thus PSL also supports some custom output types that are used for embedding other types of structured data in the ini format and parsing that. These custom types include for example lineplot_ini that contains csv lineplot data embedded in ini. More on that later.

How to make chatGPT generate structured output?

It is just a matter of priming it. Here is an example where chatGPT is primed to generate a story blueprint in INI format.

Example 1

[ask.topic]
description = Please tell the topic for your kid story:

[ask.image_style]
description = What styles would you like the images to be in? Here are some popular options:\n
    - Digital Art: This style is created using digital tools like Photoshop or Illustrator. It often has a modern and polished look and can range from highly realistic to cartoon-like.\n
    - Cartoon/Comic: Inspired by comic books or animated shows, this style often features bold lines, bright colors, and exaggerated features. Examples include the styles of "South Park" or "SpongeBob SquarePants".\n
    - Collage: This style uses cut-out pieces of paper, photos, fabric, etc., to create an image. It can be done physically and then photographed, or it can be done digitally.\n
    - Watercolor: This can be a more traditional style, but it has been modernized by many contemporary illustrators. It is often soft and dreamy, with a touch of realism.\n
    - Graphic: This style is modern and bold, using strong shapes and often a limited color palette. It's seen frequently in contemporary picture books.\n
    - Impressionistic: This style uses loose brushwork and vibrant colors to capture the feeling or impression of a scene rather than the exact details.\n
    - Mixed Media: This style combines different methods, such as painting, drawing, and digital manipulation, to create a unique look.\n
    - Minimalistic: This style uses simple shapes and a limited color palette to convey the story. It is often sophisticated and modern.\n
    - Pop Art: Inspired by the art movement of the same name, this style uses bold, bright colors and often incorporates elements of popular culture.\n
    - Vector Art: This style is made using vector graphics software and is characterized by clean, precise lines and flat colors. It's a common style for modern children's books.\n

[prompt.create_a_story]
model_name = gpt-3.5-turbo
display = False
priming.0.user = Output a 4 page children story. Each page is a description of an `image` relevant for kids, a `caption`, a `plot`, and a `learning`. Images should be attractive to kids and should be peaceful. Image description should be short (about 400 characters) and caption should also be very short (just a line). Each page should have a new thing to learn for a kid. Here are other parameters to keep in mind:
    Topic: Snow White
    Images style: Glossy and Contrasting
    Output format: ini with section headers [page_1], .., [page_4]. Each page has fields image, caption, plot, and learning
priming.0.assistant = [page_1]
    image = The image should show a bright and colorful forest with Snow White in the center. The animals should be cute and friendly, with big eyes and happy expressions. The image could use bright colors and make the image look like a scene from a fairy tale. It should be glossy so that kids can appreciate it.
    caption = Snow White meets new friends in the forest.
    plot = Snow White was walking through the forest when she met some friendly animals. There were bunnies, deer, and even a friendly bird that perched on her hand. Snow White was happy to make some new friends and play with them.
    learning = Making new friends can be fun and exciting. It's important to be kind and friendly to others.

    [page_2]
    image = The image should show Snow White holding the apple that the evil witch gave her. The apple should look shiny and tempting, but with a sinister quality to it. The artist could use darker colors and shadows to create a sense of danger and unease. The image should be glossy and attractive for kids.
    caption = Snow White learns a valuable lesson about trusting strangers.
    plot = The evil witch came to Snow White and gave her an apple. Snow White didn't know that the apple was poisoned, and she ate it. She became very sick and fell asleep for a long time. When she woke up, she learned that she should be careful about trusting strangers.
    learning = It's important to be cautious around strangers. You should never take anything from a stranger or eat something that they give you without checking with a trusted adult first.

    [page_3]
    image = The image should show the cozy cottage where Snow White is living with the seven dwarfs. The cottage should look warm and inviting, with a fire burning in the hearth and comfortable furniture. The image should be glossy and attractive for kids.
    caption = Snow White finds a safe and welcoming home with the seven dwarfs.
    plot = After waking up from her long sleep, Snow White found a cozy cottage in the woods. Inside, she met seven friendly dwarfs who welcomed her into their home. Snow White was happy to have a safe place to stay and new friends to spend time with.
    learning = Home is where you feel safe and loved. It's important to find a safe place to live and people who care about you.

    [page_4]
    image = The image should show Snow White and the prince standing together in front of a beautiful castle. The castle should look grand and impressive, with turrets and flags waving in the wind. The image should be glossy and attractive for kids.
    caption = Snow White finds true love and lives happily ever after.
    plot = Snow White met a prince who fell in love with her. They got married and lived happily ever after in a beautiful castle. Snow White was happy to have found true love and to live in a place where she felt safe and loved.
    learning = Love is an important part of life. When you find someone you love, it's important to treat them with kindness and respect.

message = Great! Can you now provide another children story in the same Output format and 4 pages:
    Topic: {{input.topic}}
    Images style: {{input.image_style}}
output_type = ini
display = False

How to use structured data in subsequent prompt sections?

PSL runtime converts structured output to an object that can be accessed using the conventional format:

obj ("." attribute | "." function | "." dynamic_content)*

In the example above, the objectified ini is stored in response.create_a_story. One can access the learning from page 3 by simply writing this:

response.create_a_story.page_3.learning

Behind the scene the run time will remove any redundant text like "Sure, here is your ini:" that the chatGPT outputs and also retry if chatGPT hallucinates completely.

Here is a continuation of the previous example:

Example 2

[prompt.background]
model_name = dall-e
message = A translucent wallpaper. Use blue and green theme. {{input.image_style}}
display = False

[prompt.plot_1_image]
model_name = add_caption
image_url = {{response.background}}
caption = {{response.create_a_story.page_1.plot}}
display_option = image