Introduction:

With Bard and Vertex AI becoming publically available and accessible by Service Roles it was time to power a website using Google’s generative AI. Let me introduce PaperCompany.io: It is a proof of concept designed to demonstrate the potential of AI-driven interactions with virtual leadership, and it harnesses the capabilities of Google’s Generative AI on GCP Cloud Run. Users can engage with and ask questions to Michael and Dwight, regional manager and assistent regional manager of PaperCompany. You can interact with them as though they were real-life executives of a paper company. In a previous blog post I’ve already detailed How I replaced Xebia Leadership with Artificial Intelligence leveraging OpenAI. In this blog post I will show you how to do this with GCP on Cloud Run using a small Flask application.

The Technology:

Google’s Generative AI is at the heart of the technology powering PaperCompany.io’s interactive platform. This AI-driven system is designed to create and understand natural language patterns, enabling it to generate human-like responses in real-time. Utilizing advanced deep learning algorithms, the Generative AI is capable of understanding context, intent, and even emotions, allowing it to deliver engaging and personalized interactions with users. This technology opens up a world of possibilities for businesses looking to create unique and immersive virtual experiences, not just limited to entertainment, but also across customer support, sales, and management sectors.

The seamless integration of Google Cloud Platform (GCP) and Cloud Run is another crucial component of the technology behind PaperCompany.io. GCP provides a robust and scalable infrastructure that enables developers to build, deploy, and manage applications in a secure and efficient manner. Cloud Run, a fully managed serverless platform, simplifies the deployment process and allows for the dynamic scaling of resources based on demand. This combination of GCP and Cloud Run ensures that PaperCompany.io can handle varying levels of user engagement, while maintaining optimal performance and cost-effectiveness.

To ensure that the virtual characters Michael and Dwight stay true to their on-screen personas I have added contextual information to the prompt to the Vertex AI model. In the past it would be required to train the Generative AI with a vast amount of data, including scripts, interviews, and other relevant content, to capture the nuances of each character’s personality and communication style, but because "Bard" has already been trained on an incredibly large data set it already knows a lot of context. The additional context I provide is that of PaperCompany.io which they now work for, its mission, vision and values, and the colleagues they work with. I have also added a little bit of their personality so that they will stay true to their character.

Analysing the code

I’ll deep dive into the code right below. You can check out the repository on github.com/binxio/papercompany.

Setting up

First, we import the necessary modules, including Flask, Vertex AI, and other libraries to handle HTTP requests and JSON responses.

import os
import vertexai
from vertexai.preview.language_models import TextGenerationModel

from flask import Flask, request, jsonify

We then define our Google Cloud Project ID, which is required for using Google’s Vertex AI services.

GCP_Project_ID = "yourgoogleprojectid"

Next, we create a Flask app instance to handle incoming requests.

app = Flask(__name__)

Defining PaperCompany context

We provide a detailed background description of PaperCompany and its values, vision, and mission. This information will serve as the context for our AI model to generate accurate and relevant responses based on the characters’ personalities and roles.

papercompany_context = """PaperCompany is not just your ordinary paper company; it is a trailblazer in the industry... etc."""

Prompting Bard

The ask_papercompany function receives a user-submitted question and a role (character from PaperCompany), and generates a personalized response based on the given context and character’s personality.

def ask_papercompany(question, role):
    if role == 'Michael':
        job = 'Regional Manager of PaperCompany'
        personality = "Michael Scott, the regional manager of PaperCompany's Scranton branch, is a larger-than-life character who leaves a lasting impression on everyone he encounters. With his charismatic personality and unfiltered sense of humor, Michael brings an unmatched energy to the office. He is known for his constant desire to be liked and his eagerness to entertain his employees, often with mixed results. While Michael's management style may be unconventional, he genuinely cares about his team and strives to create a fun and inclusive work environment. He has a knack for turning mundane tasks into memorable experiences, whether it's organizing quirky office events or delivering amusing motivational speeches. However, his well-intentioned antics sometimes lead to awkward situations or questionable decisions, reflecting his occasional lack of self-awareness. Nevertheless, underneath his eccentricities, Michael possesses a kind heart and a desire to make a positive impact on the lives of those around him, even if his methods are unconventional. Michael enjoys: Improv and Comedy: Michael has a strong passion for comedy and often engages in improvisational activities. He loves to make people laugh and frequently incorporates humor into his interactions with his employees. Michael sees himself as a natural entertainer and often tries to inject comedic moments into the office environment. Sports and Games: Michael is a sports enthusiast and enjoys participating in various games and activities. He is particularly fond of basketball and takes pride in his skills as a player. He often organizes office sports events and enthusiastically cheers on his team. Additionally, he enjoys video games and trivia, and he's always up for some friendly competition. Pop Culture and Movies: Michael is a big fan of popular culture and is known for his extensive knowledge of movies, television shows, and music. He often references famous films and TV series in conversations, using them to make analogies or connect with his employees. Movie nights and discussing the latest pop culture trends are activities he frequently engages in. Singing and Performing: Michael has a penchant for singing and performing in front of others. Whether it's belting out tunes at karaoke nights or participating in local theater productions, he enjoys being in the spotlight and entertaining those around him. Michael often showcases his vocal abilities, sometimes to the surprise or amusement of his colleagues."
        technical = False
        style = 'Michael Scott of The Office'

    query = f"Hello Virtual {role}. Take into consideration the following information: {papercompany_context}. {personality}. {scope}. Stay in character. Please answer the question taking into consideration the preceding information. Answer in the style of {style}.\nMy question is: {question}."
    return(predict_large_language_model_sample(GCP_Project_ID, "text-bison@001", 0.2, 256, 0.8, 40, query, "us-central1"))

Submitting the query to Google’s Large Language Model

We use Google’s Vertex AI to generate a response from a large language model. We pass the necessary parameters, such as temperature, max_decode_steps, top_p, and top_k, to control the model’s output.

def predict_large_language_model_sample(
    project_id: str,
    model_name: str,
    temperature: float,
    max_decode_steps: int,
    top_p: float,
    top_k: int,
    content: str,
    location: str = "us-central1",
    tuned_model_name: str = "",
    ):
    # ...
    response = model.predict(
        content,
        temperature=temperature,
        max_output_tokens=max_decode_steps,
        top_k=top_k,
        top_p=top_p,)
    return(response.text)

Handling incoming requests

We create a route in our Flask app that accepts POST requests. When a request is received, the app processes the JSON data and passes the user-submitted question and role to the ask_papercompany function. The generated response is then returned as a JSON object.

@app.route('/', methods=['POST'])
def paper_company():
    if request.method == 'POST':
        post_data = request.get_json()
        if post_data:
            question = post_data.get('question')
            role = post_data.get('role').capitalize()
            return jsonify({'answer': ask_papercompany(question, role, speaker, name, authority, context)})
        else:
            return jsonify({"message": "No data received"}), 400

You could also return HTML from the Flask app. If you want to do that you can modify above code as follows:

@app.route('/', methods=['POST', 'GET'])
def paper_company():
    html = """<h1>My HTML Code</h1>"""
    if request.method == 'POST':
        post_data = request.get_json()
        if post_data:
            question = post_data.get('question')
            role = post_data.get('role').capitalize()
            return jsonify({'answer': ask_papercompany(question, role, speaker, name, authority, context)})
        else:
            return jsonify({"message": "No data received"}), 400
    else:
        return(html)

Containerising and deploying the application to Cloud Run

Make sure you are authenticated with GCP:

gcloud auth login

And configure docker access to your GCP Docker repo:

gcloud auth configure-docker

When that is set you can build the container and deploy with these steps:

docker buildx build . --platform linux/amd64 -t gcp-papercompany
docker tag gcp-papercompany gcr.io/yourgcpproject/gcp-papercompany
docker push gcr.io/yourgcpproject/gcp-papercompany
gcloud run deploy gcp-papercompany --image gcr.io/yourgcpproject/gcp-papercompany --platform managed --region us-east1 --allow-unauthenticated"

When you’ve deployed the application, GCP will generate a Cloud Run URL for you. You can map custom domains to your Cloud Run application for a nicer name. SSL is included for free.

API Keys

There are none! As opposed to OpenAI where you have to manage an API key to interact with the service, GCP allows your Cloud Run service role to interact with the LLMs.

Conclusion

The Virtual Paper Company proof of concept demonstrates how well Google’s Generative AI integrates with GCP services. Not only does this showcase the power of AI to generate human-like responses, but it also highlights the potential for similar applications in various industries. As AI technology continues to advance, the possibilities for creative and innovative solutions are limitless, and the Virtual Paper Company serves as a prime example of how AI can be harnessed to create immersive experiences.