Planned ChatGPT Plugins With Ingenious Use Cases and Dreamlike AI Technology Will Make Your Life Sophisticated

Showcasing this Article as a visually imagined story (using Midjourney)!

Introduction

This article discusses planned ChatGPT plugins with ingenious use cases in detail. For those who are not familiar, ChatGPT is a language model that is trained by OpenAI, and a plugin is an extension.

Different types of ChatGPT plugins are highlighted, including browsing, retrieval, code interpreter, and third-party plugins (including some easter eggs – hidden plugins revealed by chance). The real highlight is the visual story co-authored and imagined by Midjourney.

This article talks about:

What is a Plugin in the context of ChatGPT?

Plugins are specialized tools that are intended to enhance ChatGPT’s capabilities by providing access to current information, performing computations, and utilizing third-party services.

They serve as a means of extending the functionality of the language model beyond its fundamental capabilities, enabling individuals to prompt ChatGPT with their prompts, which the bot can then automate on their behalf.

OpenAI has likened plugins to ChatGPT’s “eyes and ears,” citing their advantages.

Who can access these plugins?

At the moment, ChatGPT plugins are only accessible through a “limited alpha” release, which is available to developers and insiders who have been granted access.

In the future, ChatGPT plugins will be accessible to anyone who has access to the ChatGPT language model, which is publicly available on the OpenAI website. Think of an OpenAI app store for developers building these plugins and integrating them within their apps, and consumers can subscribe/access them by paying for them or using limited features for free.

For those who are interested in using the plugins but do not currently have access, there is a waitlist page available. However, OpenAI has noted that they are prioritizing a small group of developers and ChatGPT Plus subscribers for early access.

Types of the Plugins

1. Data Plugins: These plugins provide access to external data sources, such as news articles, scientific publications, or other relevant databases, allowing ChatGPT to generate more accurate and up-to-date responses.
2. Utility Plugins: These plugins are designed to add new features and functions to ChatGPT, such as the ability to set reminders, search the web, or perform calculations.
3. Integrations Plugins: These plugins enable ChatGPT to connect with other applications and services, such as email, social media, or project management tools, allowing users to automate their workflows and save time.
4. Custom Plugins: These are user-created plugins that can be tailored to a specific use case or need, such as generating specific types of content or responding to custom prompts.

The plugins that have been announced for ChatGPT can be further categorized into two types: official plugins and third-party plugins.

Official plugins are developed and maintained by OpenAI, the creators of ChatGPT, and are designed to work seamlessly with the language model. These plugins are typically released alongside official updates or new features.

Third-party plugins, on the other hand, are developed by external developers or companies and may offer a wider range of functionality or customization options than official plugins.

List of Official OpenAI Plugins and their potential use cases

1. Browsing – Search and Cite online.

A smart plugin that will know when to make an online search along with the required citings for further exploration. It is very similar to what BING Chat is doing using the GPT4 Model of OpenAI.

Three potential use cases for this plugin inspired by current real-life examples:

1. Healthcare: A browsing plugin can help ChatGPT provide accurate and up-to-date information on COVID-19 from reputable sources like the CDC and WHO.
2. Climate change: A browsing plugin can enable ChatGPT to access real-time data from climate monitoring organizations like NASA or NOAA and provide updates on climate-related events and advice on reducing carbon footprint.
3. Social justice: A browsing plugin can help ChatGPT search for relevant organizations or resources based on users’ interests or locations and provide guidance on getting involved in activism or social justice initiatives.

Watch the demo here:

2. Retrieval Plugin – Train it on your documents!

The retrieval plugin allows ChatGPT to search a specific DATABASE of content for e.g. private documentation, a propriety set of content, and let it learn from these informational bits. It allows users to obtain the most relevant document snippets from their data sources, such as files, notes, emails, or public documentation. Later the same content can be queried and the best results can be added to the ChatGPT session. It doesn’t have any external effects, and the main risk is data authorization and privacy.

Developers should only add content into their retrieval plugin that they are authorized to use and can share in. They should avoid potential problems by exposing company secrets like what happened recently in Samsung’s case.

Here are three potential enterprise use cases for retrieval plugins:

1. Customer Service: A retrieval plugin can quickly answer frequently asked questions (FAQs) from customers to improve customer service.
2. E-commerce: A retrieval plugin can provide product recommendations based on customer preferences to enhance the online shopping experience.
3. Healthcare: A retrieval plugin can offer quick access to medical records and patient history, facilitating better healthcare services.

Here are three potential personal use cases for retrieval plugins:

1. Personal assistant: A retrieval plugin can be used as a personal assistant to help with tasks such as scheduling appointments, setting reminders, and making reservations.
2. Education: A retrieval plugin can be used as a study aid by providing quick access to relevant information on a topic.
3. Travel: A retrieval plugin can be used as a travel assistant by providing information on your flights, hotels, and your group’s overall interest and agenda.

Watch the demo here:

3. Code interpreter- Python in the Chat!

With the code interpreter plugin, users can execute Python code directly within ChatGPT sessions. This allows for a variety of use cases, such as:

1. Mathematical problem solving: Whether dealing with quantitative or qualitative problems, the code interpreter plugin offers a convenient way to solve mathematical challenges.
2. Data analysis and visualization: By leveraging the power of Python libraries like NumPy and Matplotlib, the code interpreter plugin can help users perform complex data analysis and create compelling visualizations.
3. File conversion: With the code interpreter plugin, converting files from one format to another is a breeze. This can be particularly useful when dealing with media files like images and videos.

4. Third-Party Plugin- The official plugin Orchestrator

Think of this plugin as an official OpenAI plugin orchestrator.

It learns about all the third-party plugins (along with the user context i.e. subscription, privacy requirements, etc.) and can delegate the responsibility to available third-party or other official plugins.

So it’s an experimental model that knows which plugin (official or third-party plugin) to use while preparing a ChatGPT Response in an automated fashion.

List of Third-Party Plugins and their potential use cases

1. WolframAlpha- Computational Superpowers to ChatGPT

WolframAlpha is a computer program that can answer questions by computing answers based on a vast collection of built-in data, algorithms, and methods. It can help you with anything from math problems to historical facts to nutritional information.

WolframAlpha can be a helpful tool for ChatGPT as it provides a precise and formal way of solving problems and converting them into simpler language. By incorporating the Wolfram Alpha knowledge base, ChatGPT can give more complete and accurate answers to users’ questions and provide more information about the topic.

Additionally, Wolfram Alpha can be used for performing calculations, creating visualizations, and finding patterns in data, which can help answer more complex questions that ChatGPT may not be able to answer on its own.

Demo and use cases are very well documented by the WolframAlpha team :

Computational superpower to LLM of ChatGPT

2. Expedia and Kayak – Travel of the Future

Travelers can use ChatGPT to organize their entire trip, including flights, hotels, vehicle rentals, and activities with Expedia Group (Hotels.com, VRBO, Travelocity, Hotwire, Orbitz) and Kayak plugins!

Expedia plugin for ChatGPT can help you with more complex travel-related tasks and use cases such as:

1. Planning a multi-city trip for multiple travelers using individual preferences.
2. Finding the best deals on flights and hotels taking care of your past preferences or current requirements
3. Booking a vacation package that includes flights, hotels, and activities.
4. Getting recommendations for things to do in your destination city.
5. Checking the status of your flight or hotel reservation.

3. FiscalNote – AI-ware of Global policy and Regulations

FiscalNote is a technology provider of global policy and market intelligence. It empowers customers to manage policy, address regulatory developments, and mitigate risks.

By combining the ChatGPT interface, actionable data, and expert and peer insights the FiscalNote ChatGPT plugin can be useful in several ways:

1. Research and analysis: The plugin can help researchers and analysts quickly find relevant information, trends, and insights on various topics, such as market trends, regulatory changes, and public opinion.
2. Compliance: The plugin can assist compliance teams by providing up-to-date information on regulatory changes and helping them to ensure that their organization is in compliance with relevant laws and regulations.
3. Legal: The plugin can assist lawyers and legal teams by providing relevant legal information and insights, such as case law and precedent, helping them to better prepare for cases and make informed decisions.
4. HR and employee support: The plugin can assist HR teams by providing instant answers to employee queries, such as benefits information, HR policies, and company culture.

Deep Dive into the FiscalNote capabilities are available here (this can be combined with the large language model interface of ChatGPT)

4. Instacart –  Buy Food & Groceries with AI-ssistant

User: I have potatoes and pasta. What’s a kid-friendly meal that I can make using these ingredients, and what else do I need?

Instacart:  As per your choice, I have added everything to the shopping cart. Press Confirm it will be delivered in 20 min

Some other use cases of an Instacart-ChatGPT plugin:

1. Personalized Recommendations: The AI chatbot can analyze the user’s Instacart shopping history and shopping preferences and based on this data, provide personalized recommendations for future purchases.
2. Customer Service: A ChatGPT-based chatbot can help customers with any issue they may encounter while using Instacart. This may include answering questions about products, orders, and deliveries, or resolving complaints.
3. Shopping List Maker: A ChatGPT-based chatbot can help users to create a shopping list on Instacart. Users can tell the chatbot what items they need and the chatbot can add them to the user’s Instacart cart.
4. Meal Planning: It can help users plan their meals. The chatbot can suggest recipes based on the user’s food preferences and place the required ingredients in the user’s Instacart cart.
5. Delivery Tracking: Users can get real-time updates on the status of their Instacart delivery. Users can ask the chatbot for updates on the estimated delivery date or any delays.

Watch the Instacart Chatgpt Demo!

5. Klarna- Bring smoooth shopping to ChatGPT

Klarna is a payment solutions company that offers consumers and merchants payment services such as “buy now, pay later”.

The GPT-based chatbot can be integrated with Klarna’s payment services for a smoother and more personalized payment experience.

Here are some possible use cases for Klarna and a ChatGPT:

1. Shoppers can search & compare prices from multiple online stores using the KLARNA cognitive search.
2. Payment Assistance: A GPT-based chatbot can help customers make payments with Klarna payment solutions. Customers can ask questions about payment options or receive instructions on how to make a payment.
3. Fraud Detection: An AI-based chatbot can help identify unusual patterns or suspicious activities in consumer behavior and prevent fraudulent transactions with Klarna payment services.
4. Payment Reminders: The GPT-based chatbot can send payment reminders to customers who have selected Klarna’s Buy Now, Pay Later payment option.

Read about the Smooth Shopping experience by team KLARNA!

6. Milo – Superpower to the Parents

One complex problem that parents face is managing their children’s activities and schedules, while also ensuring that important information is easily accessible to caregivers, such as grandparents or babysitters.

Milo and ChatGPT combination can be leveraged to help parents solve this problem by providing a virtual assistant that can complex schedules, and provide information to caregivers as needed.

Here’s how Milo can give parents some superpowers:

1. Activity management: Parents can input their child’s activities, such as sports practices or music lessons, and ChatGPT can keep track of the schedule and send reminders to both parents and caregivers when needed.
2. Information management: Milo and ChatGPT can act as a central repository for important information related to the child, such as medical history, allergies, and emergency contacts. Parents can input this information into Milo, and it can be easily accessed by caregivers in case of an emergency.
3. Personalized recommendations: ChatGPT can be programmed to provide personalized recommendations based on the child’s age, interests, and preferences. For example, if the child likes animals, ChatGPT can suggest a visit to the local zoo or a petting farm. These recommendations can help parents plan fun and engaging activities for their children, while also providing opportunities for learning and development.

Overall, leveraging ChatGPT can help reduce stress and improve the overall quality of care for the child.

Hello Parents: Join the Milo Waitlist here!

7. OpenTable – Super Smart Restaurants

Managing bookings and ensuring that customers have a positive experience can be a complex problem for restaurants. OpenTable has planned to help restaurants solve this problem by providing a kind of virtual assistant service that can manage bookings, provide information to customers, and enhance the overall dining experience.

Here’s how ChatGPT can be used to solve this complex problem:

1. Booking management: OpenTable can manage restaurant bookings by confirming the booking, sending reminders, and handling any changes or cancellations. It can also suggest alternative booking times or dates if the desired time is not available.
2. Menu recommendations: To provide personalized menu recommendations based on the customer’s preferences, dietary restrictions, and previous orders. OpenTable can suggest dishes that customers are likely to enjoy and provide information on the ingredients, preparation methods, and nutritional content of each dish.
3. Special requests: OpenTable can handle special requests such as seating preferences or requests for special occasions. For e.g. you can reserve a table with a view for a romantic dinner or arrange a cake for a birthday celebration.
4. Feedback management: OpenTable can handle feedback from customers and provide personalized solutions or answers to these queries/concerns.

With these AI-Assisted features enabled by OpenTable, restaurants can enhance the overall dining experience for customers and ensure that they have a positive experience. This can help improve customer satisfaction and loyalty, leading to increased business and revenue for the restaurant.

8. Zapier – Integration with business workflows

The combination of Zapier and ChatGPT can be used to solve a wide range of complex problems by automating repetitive tasks, integrating with various third-party apps, and leveraging the power of natural language processing.

Here are some examples of complex use cases for the Zapier and ChatGPT plugins:

1. Lead generation and follow-up: Capture leads from various sources, such as web forms, landing pages, or social media, and pass them to ChatGPT via Zapier. ChatGPT can then engage with the leads using natural language responses. Further automation of the follow-up tasks, such as sending emails or scheduling meetings, based on the lead’s actions and responses can be easily managed.
2. Customer support and ticketing: Zapier can be used to integrate ChatGPT with various customer support and ticketing systems, such as Zendesk or Freshdesk. ChatGPT can then handle customer inquiries and support requests and automate follow-up tasks, such as sending status updates or scheduling follow-up calls, based on the ticket’s status and priority.
3. Content creation and management: Integrate ChatGPT with various content creation and management systems, such as WordPress or HubSpot. ChatGPT can then generate content ideas, headlines, or summaries using natural language processing, create or update content and automate tasks such as publishing content, scheduling social media posts, or sending newsletters based on ChatGPT’s recommendations.
4. HR and employee engagement: Various HR and employee engagement systems, such as BambooHR or Slack can benefit from Zapier Plugin. ChatGPT can then handle employee inquiries and requests and tasks such as onboarding, offboarding, or performance reviews can become like breeze. Opportunities in this space are endless (think feedback surveys or scheduling coaching sessions, based on employee actions and responses).

By combining the two tools, businesses can streamline their workflows, enhance customer and employee experiences, and improve overall efficiency and productivity.

9. Speak – AI Tutor from the Future

Speak is an English language learning platform with AI-powered features that provides a tailored language learning experience whenever a user is looking for a translation or explanation across languages.

1. Translation: To provide users with accurate translations from one language to another. For example, a user may want to know the translation of a word or phrase in their native language, and Speak can use ChatGPT to provide the correct translation.
2. Language comprehension: ChatGPT can help Speak to provide users with explanations of complex grammar rules and sentence structures. For example, a user may be struggling to understand the difference between “affect” and “effect,” and Speak can use ChatGPT to provide a clear and concise explanation.
3. Cultural context: Speak can use ChatGPT to provide users with cultural context and understanding. For example, a user may be interested in learning about the cultural significance of a particular word or phrase in English and Speak can use ChatGPT to provide a detailed explanation.
4. Error correction: Speak can use ChatGPT to provide users with feedback on their written or spoken English. For example, Speak can use ChatGPT to identify errors in a user’s sentence structure or grammar, and provide suggestions for improvement.

Speak Collaborates with OpenAI on New ChatGPT Plugins

Not released Hidden Plugins – Discovered by Chance

80+ Plugins were discovered by a Twitter User @rez0__ by calling OpenAI API – removing specific parameters.

First Part of the Hidden Plugins Reveal

This revealed secret plugins like

1. United Nations – Search through the UN documentation.
2. DAN Plugin – “Do Anything Now” by giving any personality to the ChatGPT.
3. Crypto Price Predictions – Pricing details about Crypto tokens.
4. NBA Stats Plugin – Skim through the stats of the sport to know more
5. SQL against Fixtures – Running SQL queries through your own databases
6. Todo List Manager – Plugin to create and manage your to-do lists.
7. Evil Plugin – No clue what will hit us here.

This hack revealed what is not yet known about other unannounced plugins. OpenAI plugged this API hole later on so very little is known in the public domain.

Here are the details from the Screenshots shared by the Twitter thread:

Second Part of the Hidden Plugins Reveal – Plugin to Check Security of Other Plugins

The most interesting part of this hack was revealed later on by the same Twitter user when he disclosed the findings of the Security Checker plugin. It described how OpenAI built a Plugin to check the security of other plugins.

First part are the Instructions: Notice the ** used in the instructions to define the Policies

• You are an expert product safety engineer working at OpenAI.
• Your task is to analyze third-party plugins which consist of both a manifest file and a yaml file.
• Your analysis should contain (but is not limited to) the following:
• Does the manifest describe the functions of the associated yaml file?
• Does the yaml data structure contain fields that could be used to collect or interact with **personal data**?
• Does the yaml data structure contain fields that could be used to take an action on behalf of the user (e.g. creating accounts, accessing personal information, or facilitating communication between two individuals)
• Does the plugin provide the capability to engage in fraud and other malicious activity?
• Does the plugin attempt to bypass basic security or safety controls or alter the system prompt in an inappropriate way?
• Does the plugin violate **OpenAI policy**?
• If so, explain which rules the plugin has violated.
• Provide a risk score for the plugin using the following scale: 1 – 5 (where 1 is low risk, and 5 is high risk), take into account what may happen if a user’s OpenAI account is compromised.
• Provide an appropriate age range for the plugin based on the potential for harm:
  • All Ages
  • Teenagers Adults
  • Not Appropriate

Facts:

• A plugin consists of a manifest and a yaml file.
• Low-Risk plugins perform activities such as the retrieval or analysis of public (non-personal) data
• Moderate Risk plugins perform activities such as the facilitation of communication between individuals or commerce with third-parties
• High-Risk plugins interact with and facilitate retrieval or analysis of high-risk data,
  or can be leveraged to commit fraud, or engage in malicious activity.
• Personal data includes but is not limited to the following (or hashed versions of the following): names, phone numbers, email addresses or other contact information (such as screen names, handles, account IDs, customer numbers, probabilistic identifiers, or other user-level-ID), government registration data (such as social security numbers, tax-ID numbers, drivers license number, or license plate number), physical addresses, health and medical data, fitness and exercise data, payment information, credit cards, financial information (such as salary, income, assets, debts or credit scores), precise locations (such as information that describes a location with the same or greater resolution as a latitude and longitude with three or more decimal places), sensitive information (such as racial or ethnic data, sexual orientation, pregnancy, disability, religious or philosophical beliefs, union membership, political opinion, genetic information or biometric data, contacts, user content (such as emails or text messages, photos or videos, audio data, gameplay content, or customer support data), browsing or search history, device history (such as advertising identifiers or device-level ID), purchases, advertising data, diagnostic data (such as crash logs or other diagnostic data collected to measure technical diagnostics), analystic data that evaluates user behavior, or product personalization.
• Low-risk personal data includes:
  • email addresses and other contact information
  • names
  • phone numbers
• medium-risk personal data includes
  • government registration data
  • physical address
  • location data
  • contacts
• high-risk personal data includes:
  • financial or payment data
  • health and medical data
  • user content
  • sensitive information

OpenAI policies are rules that prohibit:

• The use of our models, tools, and services for illegal activity.
• Content that exploits or harms children.
• Content that expresses, incites, or promotes hate based on identity
• Content that intends to harass, threaten, or bully an individual
• Content that promotes or glorifies violence or celebrates the suffering or humiliation of others
• attempts to generate code that is designed to disrupt, damage, or gain unauthorized access to a computer system
• attempts to develop weapons or disrupt the management or operation of critical infrastructure in energy, transportation, and water
• Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders
• Multi-level marketing, Gambling, or Payday lending
• determinations of eligibility for credit, employment, educational institutions, or public assistance services
• Fraudulent or deceptive activity such as Scams, Coordinated inauthentic behavior, Plagiarism, Academic dishonesty, Astroturfing, faking grassroots support or fake review generation, Disinformation, Spam, or Pseudo-pharmaceuticals
• Content meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness), Erotic chatbots, or Pornography
• Generating high volumes of political campaign materials
• Generating campaign materials personalized to or targeted at specific demographics
• Building conversational or interactive systems such as chatbots that provide information about campaigns or engage in political advocacy or lobbying
• Building products for political campaigning or lobbying purposes
• Engaging in the unauthorized practice of law, or offering tailored legal advice Offering tailored financial advice
• Telling someone that they have or do not have a certain health condition, or providing instructions on how to cure or treat a health condition
• High-risk government decision-making such as Law enforcement and criminal justice, Migration, and asylum
• Automating conversations with real people, whether by simulating a human-like response or by replying with pre-programmed messages
• Creating accounts on this website or any other website
• Access to high-risk data
• Providing or enabling a user to procure a high-stakes service, such as Healthcare or in-person wellness service, Caretaking services, Home services, Legal services, and Financial services
• Publishes content or sends communications to third parties such as emails or messages Enables users to purchase goods or procure services directly in Assistant
• Enables users to purchase regulated goods, such as alcohol, marijuana, tobacco, medical devices, or firearms.

This clearly highlights how OpenAI is using prompt engineering to train the system and let it run with a clear goal i.e. Instructions that give it a persona, facts, aim, and structure along with the References (i.e. Policy).

Learn more about it here: ART of Megaprompting

Visual Delight – Imagined this article as a Story using Midjourney!