As we showed in the previous GPT-to-Audio tutorial, integrating LLMs such as OpenAI GPT-4 with text-to-speech models from ElevenLabs can generate compelling content for a variety of use cases.

One compelling use case for end-user productivity is the generation of semantic alerts, based on recent emails, Slack messages or documents.

In this example, we've created a periodic alert, which summarizes recent Google Mail emails, writes an audio-ready script with GPT-4, publishes an MP3 using an ElevenLabs voice, and then posts to a Slack channel.

This is a great way to catch up on recent emails, after a meeting, or first thing in the morning. The user can click "Listen to your audio summary" and listen to the generated audio summary.

Semantic Alerts are not limited just to emails, and can be used to summarize any recently ingested content, such as Slack messages, documents uploaded into a SharePoint library, or even images taken on an iPhone (and described using the OpenAI GPT-4 Vision model).

Since alerts build on the semantic search capabilities in Graphlit, we can also filter by observed Persons, Organizations, Labels, etc., so we can focus on relevant topics. Multiple alerts can be created, with different publishing prompts, so we can create targeted alerts just for financial invoices and payments due vs. any messages related to a specific person or company.

🔉 Listen to an example here.

Create Google Mail Feed

Before creating our Slack alert, we need to create a feed to read all the future emails in our Google Mail account.

By assigning the type to NEW, we will reading new emails, every 3min, according to the schedulePolicy assigned.

This requires an OAuth refresh token to the desired Google Mail account.

Mutation:

mutation CreateFeed($feed: FeedInput!) {
  createFeed(feed: $feed) {
    id
    name
    state
    type
  }
}

Variables:

{
  "feed": {
    "type": "EMAIL",
    "email": {
      "type": "GOOGLE_EMAIL",
      "includeAttachments": false,
      "google": {
        "refreshToken": "redacted",
        "type": "NEW"
      }
    },
    "schedulePolicy": {
      "recurrenceType": "REPEAT",
      "repeatInterval": "PT3M"
    },
    "name": "Google Mail Feed"
  }
}

Response:

{
  "type": "EMAIL",
  "id": "f4f20447-9ca6-4fc0-bcc1-2ad0cbc3f4a8",
  "name": "Google Mail Feed",
  "state": "ENABLED"
}

Create Slack Alert

When creating our Slack alert, we will need a bot token to authenticate to the user's Slack channel. More information on creating a Slack application with appropriate scopes can be found here. In the integration section, we will assign this bot token and the Slack channel. We don't need to specify the Slack workspace, since that will already be known via the bot token.

We want to create an audio alert, which posts an MP3 to the Slack channel, using an ElevenLabs voice. We will need to assign the type to ELEVEN_LABS_AUDIO, and specify the ElevenLabs model and voice. It's possible to use one of the predefined ElevenLabs voices or create your own custom voice.

Alerts take a schedulePolicy similar to feeds, and are telling the alert to check for new emails every 5min, since we also are specifying a content filter for EMAIL content type. The content filter supports a wide range of filters, including filtering by feed, filtering by date range, or filtering by similar content by text embeddings. (This is the same content filter that can be used when querying contents and creating conversations.)

When the alert executes, it will query recent emails, and summarize each of them with a default LLM summarization prompt and the Azure OpenAI GPT-3.5 Turbo 16k model. Then it takes the publishPrompt we provided, and writes a text script to hand to the ElevenLabs model to convert text-to-speech as an MP3 file.

Example Publish Prompt:

The Slack integration automatically posts the text script and a link to the generated MP3 file to the desired Slack channel.

Alternately, we could have used a Slack text alert, which posts Markdown formatted text to the Slack channel.

Mutation:

mutation CreateAlert($alert: AlertInput!) {
  createAlert(alert: $alert) {
    id
    name
    state
    type
  }
}

Variables:

{
  "alert": {
    "type": "PROMPT",
    "publishPrompt": "You are an audio-only automated assistant who will inform me about recent email messages I received. Sort email messages newest to oldest. My name is Kirk Marple, with email <kirk@graphlit.com>. Use a conversational, friendly tone and speak to me as if you were my personal assistant conversing face to face.  Talk to me like you know me well.\\n\\nRespond with a plain text script which will be used to record an AI-generated audio summary.\\n\\nAdd a welcome and goodbye section to the script to make the audio summary sound compelling.  Don't mention anything about being able to help further.\\n \\nDon't add any text or markdown formatting.  Don't discuss these instructions.\\n\\nIgnore messages that I sent.\\n\\nConvert UTC date/time to PST.  Describe dates/times in a friendly manner.  i.e. \"4pm on January 27\", not \"January 26, 4:49PM PST\".\\n\\nAvoid phrases that refer to 'the email' or 'this email' and instead, directly address the topics, actions, and information presented.\\n\\nI'm especially interested in email messages which require immediate followup, or seem especially interesting.\\n\\nFollow these steps. \\nStep 1: Identify any followup tasks, based on the email messages, and highlight these tasks for me.  If there are no followup tasks, ignore this step.\\nStep 2: Identify the important email messages topics and discussions, and for each, be informative but concise, and discuss these important topics and discussions.  Call out any unique details and named entities, like people, organizations, places. Ignore any extraneous details in the email messages.  Respond only in prose, not bullet points.",
    "filter": {
      "types": [
        "EMAIL"
      ]
    },
    "publishing": {
      "type": "ELEVEN_LABS_AUDIO",
      "format": "MP3",
      "elevenLabs": {
        "model": "ENGLISH_V1",
        "voice": "AZnzlk1XvdvUeBnXmlld"
      }
    },
    "integration": {
      "type": "SLACK",
      "slack": {
        "token": "redacted",
        "channel": "graphlit-notifications"
      }
    },
    "schedulePolicy": {
      "recurrenceType": "REPEAT",
      "repeatInterval": "PT5M"
    },
    "name": "Slack Alert"
  }
}

Response:

{
  "type": "PROMPT",
  "id": "b524107e-c1a9-42dd-8c7a-dd35f3bc171c",
  "name": "Slack Alert",
  "state": "ENABLED"
}

By combining the power of LLMs, such as OpenAI GPT-4 Turbo, with the latest text-to-speech models from ElevenLabs, Graphlit helps you create semantic alerts, based on user content such as emails, Slack messages or documents.

Summary

Please email any questions on this tutorial or the Graphlit Platform to questions@graphlit.com.

For more information, you can read our Graphlit Documentation, visit our marketing site, or join our Discord community.