paint-brush
PromptDesk: Simplifying Prompt Management in a Rapidly Evolving AI Landscape by@justinmacorin
239 reads

PromptDesk: Simplifying Prompt Management in a Rapidly Evolving AI Landscape

by Justin MacorinApril 3rd, 2024
Read on Terminal Reader
Read this story w/o Javascript

Too Long; Didn't Read

Effective prompt management is the key to success in today's rapidly evolving AI market. Our ability to quickly build, iterate, and organize prompts is imperative to create value for businesses and customers.
featured image - PromptDesk: Simplifying Prompt Management in a Rapidly Evolving AI Landscape
Justin Macorin HackerNoon profile picture


Why I Started PromptDesk

As a Machine Learning engineer at Seismic, the world's leading AI-powered sales and marketing enablement platform, I've witnessed firsthand the incredible pace at which the AI market is moving. This rapid evolution, coupled with market hype and confusion, inspired me to create PromptDesk, a 100% open-source project to streamline prompt-based development.


Design, fine-tune, and evaluate your prompts using a user-friendly interface with an unlimited number of models.


Focus on One Task, Very Well

The primary goal of PromptDesk is to serve as a foundational component for most of my LLM and prompt-based development work. In this fast-paced industry, our ability to organize prompts effectively is imperative. Prompts should be quick to build and iterate on so we can focus on innovation and creating value for businesses and customers.


Access detailed logs of your prompt's performance, raw API requests, responses, tokens to accelerate debugging and troubleshooting.


Navigating the Crowded Landscape and Premature Expansion

The prompt-management space is crowded, with many players expanding into RAG, Agent, LLM training/fine-tuning, and other areas.


However, my view is that this expansion is premature for several reasons:

  1. RAG is a challenging use-case-specific process for many organizations
  2. RAG has complex integration requirements and diverse data source needs
  3. Increasing LLM context windows may render RAG-based approaches unnecessary
  4. Building an Agent is exceptionally complex and use-case-specific
  5. Best practices in this space are in their infancy
  6. Training and fine-tuning LLMs may become less critical as model cost and quality improve


Given these factors, I would feel uncomfortable building something that may quickly become obsolete.


Immediate Value

PromptDesk is decoupled from commercial LLMs, allowing quick integration with any LLM API without waiting for teams or contributors to build those integrations. PromptDesk is also vendor-agnostic and can be hosted internally. This flexibility is crucial as data privacy considerations and the complexity of future AI applications are expected to increase exponentially with the development of AI agents.


PromptDesk's goal is not to be an all-encompassing AI app. Instead, it is designed to excel at one thing: prompt-based development.


PromptDesk aims to provide unparalleled value to its users by focusing on this core functionality.


Immediately integrate with an unlimited number of LLM models using simple code blocks and a model-adding wizard.


Examples of Real Success

Since implementing PromptDesk, I've experienced a significant acceleration in development and engineering speed, both at work and with side projects. Friends and colleagues who have used the project have also expressed their appreciation for how it has facilitated their prompt engineering process.


Review, edit, modify, and regenerate prompt data at a large scale to accelerate optimization and fine-tuning.


How it Works

Installing (Docker image)

PromptDesk was built to get started in under 5 minutes. Our 2 line install script can execute a local or remote development install with a domain/sub-domain name (SSL). Our Quickstart Guide provides more information.

Setup

from promptdesk import PromptDesk

# PromptDesk is only available as a self-hosted Docker image
pd = PromptDesk(
    api_key="YOUR_LOCAL_OR_SELF_HOSTED_PROMPTDESK_API_KEY",
    service_url="http://localhost"
)

# Check if the PromptDesk service is up and running!
print(pd.ping())

Prompt Generation

# Generate text immediately
story = pd.generate("short-story", {
    "setting": "dark and stormy night",
    "character": "lonely farmer",
    "plot": "visited by a stranger"
})

print(story)

Classification and Caching

# Built-in Classification
isHappy = pd.generate("is_positive", {
  "text": text
}, classification={
  True: ["positive", "happy", "yes"],
  False: ["negative", "sad", "no"]
}, cache=True)

if isHappy:
  print("I'm happy!")
else:
  print("I'm sad!")


For Your Consideration

If you believe that PromptDesk may enhance your prompt-based development workflow, I invite you to try it at github.com/promptdesk/promptdesk.


Your support as a GitHub star ⭐ would be greatly appreciated!


Thank you,


Justin




In an AI landscape characterized by rapid change and often premature expansion, PromptDesk aims to provide a stable, focused, and truly open-source solution for prompt-based development. Join us in our mission to simplify and accelerate the creation of innovative AI applications.