Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Survey with Industry Professionals 2 Survey with Industry Professionals 3 RQ1: Real-World use cases that necessitate output constraints 3 RQ1: Real-World use cases that necessitate output constraints 4 RQ2: Benefits of Applying Constraints to LLM Outputs and 4.1 Increasing Prompt-based development Efficiency 4 RQ2: Benefits of Applying Constraints to LLM Outputs and 4.1 Increasing Prompt-based development Efficiency 4.2 Integrating with Downstream Processes and Workflows 4.2 Integrating with Downstream Processes and Workflows 4.3 Satisfying UI and Product Requirements and 4.4 Improving User Experience, Trust, and Adoption 4.3 Satisfying UI and Product Requirements and 4.4 Improving User Experience, Trust, and Adoption 5 How to Articulate output constraints to LLMS and 5.1 The case for GUI: A Quick, Reliable, and Flexible Way of Prototyping Constraints 5 How to Articulate output constraints to LLMS and 5.1 The case for GUI: A Quick, Reliable, and Flexible Way of Prototyping Constraints 5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints 5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints 6 The Constraint maker Tool and 6.1 Iterative Design and User Feedback 6 The Constraint maker Tool and 6.1 Iterative Design and User Feedback 7 Conclusion and References 7 Conclusion and References A. The Survey Instrument A. The Survey Instrument 5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints Respondents found natural language easier for specifying complex constraints than GUIs, especially for extended background contexts or numerous choices that wouldn’t reasonably fit into a GUI. Natural language was also preferred for expressing vague, nuanced, or open-ended constraints, like “don’t include offensive words” or “respond in a cheerful manner.” At a high level, respondents emphasized that natural language provides a more natural, familiar, and expressive way to communicate (potentially multiple) complex constraints, and, “trying to figure out how to use a GUI might be more tedious.” “don’t include offensive words” “respond in a cheerful manner.” “trying to figure out how to use a GUI might be more tedious.” Additionally, some respondents noted that, despite their preference for using GUIs to define constraints from time to time, they ultimately have to use natural language prompts due to API limitations. Moreover, some wished to reference “external resources” in constraints that are not feasible to directly include in prompts (e.g., “a large database / vocabulary”). These suggest that a dedicated “output-constraints” API field for specifying constraints could be advantageous, potentially through the use of a formal language or notation. “external resources” “a large database / vocabulary” This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv Authors:
(1) Michael Xieyang Liu, Google Research, Pittsburgh, PA, USA (lxieyang@google.com);
(2) Frederick Liu, Google Research, Seattle, Washington, USA (frederickliu@google.com);
(3) Alexander J. Fiannaca, Google Research, Seattle, Washington, USA (afiannaca@google.com);
(4) Terry Koo, Google, Indiana, USA (terrykoo@google.com);
(5) Lucas Dixon, Google Research, Paris, France (ldixon@google.com);
(6) Michael Terry, Google Research, Cambridge, Massachusetts, USA (michaelterry@google.com);
(7) Carrie J. Cai, Google Research, Mountain View, California, USA (cjcai@google.com). Authors: Authors: (1) Michael Xieyang Liu, Google Research, Pittsburgh, PA, USA (lxieyang@google.com); (2) Frederick Liu, Google Research, Seattle, Washington, USA (frederickliu@google.com); (3) Alexander J. Fiannaca, Google Research, Seattle, Washington, USA (afiannaca@google.com); (4) Terry Koo, Google, Indiana, USA (terrykoo@google.com); (5) Lucas Dixon, Google Research, Paris, France (ldixon@google.com); (6) Michael Terry, Google Research, Cambridge, Massachusetts, USA (michaelterry@google.com); (7) Carrie J. Cai, Google Research, Mountain View, California, USA (cjcai@google.com).

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How to Articulate Output Constraints to LLMS

Structuring Technology FTW

Too Long; Didn't Read

Make resilience your competitive advantage

The Case for NL: More Intuitive and Expressive for Complex Constraints

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

How to Articulate Output Constraints to LLMS

"We Need Structured Output": Towards User-Centered Constraints on Large Language Model Output

How We Deployed an Online Survey to Users of an Internal Prompt-Based Prototyping Platform

Real-World Use Cases That Necessitate Output Constraints

The Benefits of Applying Constraints to LLM Outputs

How to Articulate Output Constraints to LLMS

"We Need Structured Output": Towards User-Centered Constraints on Large Language Model Output

How We Deployed an Online Survey to Users of an Internal Prompt-Based Prototyping Platform

Real-World Use Cases That Necessitate Output Constraints

The Benefits of Applying Constraints to LLM Outputs

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps