Creating a Kendra-based Serverless Chatbot For Your Website Application

1. Introduction

In this article, we’ll explore two easy ways to deploy a chatbot with minimal effort, primarily using AWS services. And if that’s not impressive enough, our entire application will be deployed serverlessly! We’ll dive into creating a serverless chatbot for your website using mostly AWS services, including Amazon Kendra. The tutorial is divided into two parts: one focusing on developing the application backend, and the other introducing Amazon Lex as an alternative solution.

Throughout this guide, we’ll keep in mind security considerations and AWS best practices, offering you a comprehensive approach to building a powerful, efficient, and secure chatbot solution.

Creating a Kendra-based Serverless Chatbot

My team’s vision to develop this combination of services emerged from a practical need during Develeap’s Hackathon, where we were tasked to create a chatbot overnight. As the clock ticked away, a pressing question nagged at me: How could we possibly train an AI model for a chatbot in such a short time? After a brief moment of frustration, a sudden realization struck. Surely, a cloud giant like AWS, with its vast array of Machine Learning services, must offer a solution to my dilemma. That’s when I discovered AWS Kendra and its world of possibilities.

Within five hours, we had assembled a Kendra database, a basic S3 frontend website, a Docker image housing our application, and an AWS ECS cluster running containerized tasks. But as the dust settled, another thought took root. Why stop there? With S3 serving as our serverless frontend and Amazon Kendra as our serverless database, transforming our backend application to a serverless model would make our entire application serverless!

The following days saw me working tirelessly on migrating from ECS to Lambda + API Gateway. Just as I neared completion, I stumbled upon Amazon Lex, opening up yet another avenue for chatbot creation. The results were nothing short of amazing, compelling me to document this journey.

In the past, the idea of creating a chatbot for your web server might have seemed daunting, given the time, costs, and expertise required in ML. However, times have changed. If you’ve been considering a chatbot for your application but didn’t know where to start, you’ll be pleased to know that new, simpler methods are now available. These approaches not only streamline the process but also eliminate the need for backend development, leaving you to focus solely on the user interface (UI) for your website.

Today we are going to see two easy ways to deploy a chatbot in almost no effort using key services, mainly using AWS, and our entire application is going to be deployed serverless!

2. Prerequisites

1. 1. AWS Account.
   2. Basic programming knowledge in Python.
   3. OpenAI account.
   4. Basic understanding of front-end development.

Let’s start with the more complex way to make our chatbot and gain the basic understanding of our architecture.

3. Create a serverless chatbot – architecture overview

4. Amazon Kendra

Modern chatbots don’t usually require human training when it comes to their AI models. Instead, in an unsupervised learning model, the software trains the algorithm. Because our chatbot application is serverless, we don’t have infrastructure to train, so we have to find another solution. Therefore, our application should have a database powered by machine learning algorithms in order to work properly and to retrieve relevant data.

Our chatbot application uses Amazon Kendra as a database. Kendra allows you to quickly implement a unified search experience across multiple structured and unstructured content repositories. Use natural language processing (NLP) to get highly accurate answers without the need for machine learning (ML) expertise. Also, it delivers ML-powered instant answers, FAQs, and document ranking as a fully managed service. Our usage with Kendra, is to extract relevant data for the application that will host the chatbot on. For example, if we want to create a chatbot for a ‘Develeap’ website we will load all Develeap’s magazine, articles, etc. and then the app issues a search query to the Amazon Kendra index based on the user request. Then, the index returns search results with excerpts of relevant documents from the ingested data.

4.a Creating Kendra Index

Creating the Kendra index itself is actually a very easy process:

• • Enter the Kendra service in the AWS console.

• • Go to the ‘indexes’ tabs and select ‘create index’.

• • Choose a name for your index and specify the role that your index is going to use. Note that you can select an existing role, but the best and most recommended way is to select the option to create a new role.

• • Then you can configure the user access control to your index. If you don’t have any special modifications to make, just leave it with the defaults.

• • In the ‘Provisioning editions’ section select the ‘Developer edition’ as we don’t need more than that. Note that Kendra is a little bit expensive service, and although you are paying hourly for using the service, it’s better to read the pricing information before creating the index.

4.b Creating Index datasource

1. 1. After creating the index go to the ‘Data sources’ tab and create a new ‘Web Crawler’ data source. We are choosing the web crawler as we want to crawl all of Develeap’s website pages. Give it a name.
   2. Select ‘Source URLs’ as the crawler source, and specify the website sources, e.g –

https://www.develeap.com

https://www.develeap.com/*

we specify the /* path to let our crawler run on sub-URLs too.

3. 3. We don’t need Authentication, Web proxy or VPC. If you want to configure this it is up to you.
      
      
   4. Create a new role for the crawler and give it a name.
      
      
   5. In the ‘Sync Scope’ you can specify how deep you want your crawler to dive in, in my opinion you can choose the ‘Sync everything’ option, and leave the other options as defaults.
      
      
   6. Change the “sync run schedule” option to “run on demand”.
      
      
   7. Add the data source.

After creating the data source you need to select the option “sync now” and then the crawler starts its job. It can take some time to collect all the data, from a few minutes to even a few hours. Be patient, but you can still continue to the next step.

5. OpenAI

OpenAI is an artificial intelligence research laboratory. OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits our users. AGI refers to highly autonomous systems that have the capability to outperform humans at most economically valuable work.

So in our chatbot application, this will be our LLM (Large Language Model). OpenAI will “understand” how to retrieve the Kendra answer to the user.

OpenAI management is very easy, after you’ve created an account, navigate to the ‘API keys’ tab and generate a new API key. Place it in an AWS Secret Manager secret named for example ‘OPENAI_API_KEY’.

Important note: If you use the free tier plan you may face an issue later that denies you making a call from the Lambda function to the OpenAI API. To fix this issue you should place a small budget (5$).

6. Lambda Backend

As our entire application is going to be serverless, we need a solution to handle the backend in an efficient way. Lambda functions serve our goal perfectly! With lambda we can run code on servers without managing them. Lambda supports many languages including Python, Java, .NET, Go, Ruby and NodeJS, so when you trigger the Lambda it will perform the actions that you want to make with an option even to return a response. The function can be triggered manually or by other AWS services, in our use case by API Gateway.

In our application we will get the question from the frontend, our Lambda function will handle the application logic and run queries against the OpenAI engine and Amazon Kendra which are already ready, and then will retrieve the answer back from Kendra back to the user.

6.a Create Lambda Source Code

Our source code will contain the logic for the Question-Answer mechanism. The Lambda function is going to get a question, perform API requests against the Kendra index, and the OpenAI using the API key, and then output the response back. For now let’s say that we are developing our Lambda function in the Python programming language. So in order to create this source code we mostly need three libraries:

• • langchain

• • openai

• • boto3

here is an example of a lambda_function.py source code that you can use:

from langchain.retrievers import AmazonKendraRetriever
from langchain.chains import ConversationalRetrievalChain
from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI
import os


chat_history = []
MAX_HISTORY_LENGTH = 5


def lambda_handler(event, context):
   user_message = event['question']
   user_name ='default'# request.headers['X-User-ID', 'default']
  
   qa = build_chain()
   try:
     if (len(chat_history) == MAX_HISTORY_LENGTH):
       chat_history.pop(0)
   except:
     print("no user")
   result = run_chain(qa, user_message, chat_history)
   chat_history.append((user_message, result["answer"]))
   answer=result['answer']
   if 'source_documents' in result:
     answer+="\nSources:\n"
     for d in result['source_documents']:
       answer+=d.metadata['source']+"\n"
  
   response = {
       'statusCode': 200,
       'body': answer
   }
   return response


region = os.environ["AWS_REGION"]
kendra_index_id = os.environ["KENDRA_INDEX_ID"]
openai_api_key=os.environ["OPENAI_API_KEY"]


def build_chain():


 llm = OpenAI(batch_size=5, temperature=0, max_tokens=100,openai_api_key=openai_api_key)
    
 retriever = AmazonKendraRetriever(index_id=kendra_index_id, region_name=region )


 prompt_template = """
 The following is a friendly conversation between a human and an AI.
 The AI is talkative and provides lots of specific details from its context.
 If the AI does not know the answer to a question, it truthfully says it
 does not know. The AI is helpful and tries to answer the question to the best of its ability.
 {context}
 Instruction: Based on the above documents, provide a detailed answer for, {question} Answer "don't know"
 if not present in the document.


 Solution:"""
 PROMPT = PromptTemplate(
     template=prompt_template, input_variables=["context", "question"]
 )


 condense_qa_template = """
 Given the following conversation and a follow up question, rephrase the follow up question
 to be a standalone question.


 Chat History:
 {chat_history}
 Follow Up Input: {question}
 Standalone question:"""
 standalone_question_prompt = PromptTemplate.from_template(condense_qa_template)


 qa = ConversationalRetrievalChain.from_llm(
       llm=llm,
       retriever=retriever,
       condense_question_prompt=standalone_question_prompt,
       return_source_documents=True,
       combine_docs_chain_kwargs={"prompt":PROMPT})
 return qa


def run_chain(chain, prompt: str, history=[]):
 return chain({"question": prompt, "chat_history": history})

Triggering the Lambda will start the lambda_handler function. At first it will call the build_chain() function that configures the OpenAI and the Kendra index, and create a prompt to generate a response for a question using the ConversationalRetrievalChain function. Then we use the run_chain(chain, prompt:str, history=[ ]) function that sends our user’s question to this prompt and returns the answer to this question. At last the lambda_handler function adds the source documents to the answer and builds the response format. Then the response is returned to the user.

6.b Create Lambda Function Package

After we have the source code of our Lambda function, we need to create a package in order to deploy this function to the AWS console with the function’s dependencies. There are many ways to deploy the function to AWS, so if you know how to use Lambda functions layers you are also allowed to use them. I’m going to show you how to achieve this goal using a Lambda compressed package.

1. 1. Create a folder named (for example) ‘dependencies’.

mkdir dependencies

2. 2. Download the function dependencies into this folder.

pip install langchain openai boto3 -t ./dependencies

3. 3. Enter the folder and create a zip file out of the folder’s content.

cd dependencies && zip -r ../chatbot_lambda_function.zip . && cd – 

4. 4. Add the lambda_function.py file into the zip file.

chatbot_lambda_function.zip lambda_function.py

5. 5. Now you can try to deploy directly to your lambda function in the console but you most likely get an error saying that your file size is too big, so you better upload it to S3 first and then upload the Lambda function from there. You can upload the zip file to S3 directly or by using the AWS CLI command.

aws s3 cp chatbot_lambda_function.zip s3://< your_S3_bucket_name >

6.c Create Lambda Function in AWS Console

1. 1. In AWS Lambda console select “create function”.
      
      
   2. Specify a function name, choose Python runtime, and create the function.
      
      
   3. For “Code source” select “Amazon S3 location” and upload the Lambda package that we’ve created earlier.
      
      
   4. In the configuration tab, go to permissions and open your Lambda role in IAM. Because our Lambda need permissions to access the Kendra service we need to add a new inline policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "kendra:Retrieve",
            "Resource": "arn:aws:kendra:<KENDRA_REGION>:<ACCOUNT_ID>:index/<INDEX_ID>"
        }
    ]
}

5. Now let’s continue in the “Environment variables” tab and add two new variables.

• • KENDRA_INDEX_ID – from the kendra service.

• • OPENAI_API_KEY – OpenAI API Key that we created earlier.

6. Continue to the ‘General Configuration’ tab and set the function’s timeout to 30 seconds.

7. That’s it, we can now test our Lambda function in the “test” tab using the following snippet:

{
    “question”:”What is DevOps?”
}

If the information is available in your Kendra index, you should get back a relevant response.

7. API Gateway

Because our frontend chatbot UI can’t directly trigger the backend Lambda logic, we need one last piece between them. The AWS API Gateway service comes to our help. The frontend will send the request to the API Gateway stage endpoint with the appropriate method and this one will trigger our Lambda.

Only one problem remained unsolved: If we deploy the API Gateway endpoint publicly, we expose our API to other requests that won’t come from our frontend service. It means that if we don’t want someone to create a new frontend website and call our API or actually still the chatbot functionality, we must secure our endpoint so only our frontend will be able to trigger the chatbot mechanism.

In the second section of this article we are going to learn a different solution that resolves this issue, but for now as we don’t want to involve Lambda authorizers or other difficult authentication mechanisms, so we will just create the API publicly.

1. 1. In the API Gateway service page select “Create API” and choose to build a “REST API”.
      
      
   2. Then you will be asked to specify basic API information. Choose a name for your API, and select the endpoint type.
      
      
   3. Create a new method:

Select the “Method type” to be “POST” and the “Integration type” as “Lambda function”. Then select your Lambda function from the list.

4. 4. Return to the “Resources” tab and hit the “Enable CORS” button. under “Access-Control-Allow-Methods” select the “POST” button and save your changes. By doing this you will later enable the S3 website to access your API endpoint with a POST method”
      
      
   5. Go back to the POST method tab and hit the “Deploy API” button that will allow you to create a deployment stage for your API. Give it a name and create the stage.

8. S3 Frontend

So far, we have been working on the application backend (logic) and now we are going to focus on the other part, the user interface (UI). There are many different ways to deploy the final application itself on the internet. Some will prefer building docker images for frontend and run on a containerized environment, others may serve static content using nginx service. But as we are searching for a serverless solution we need to think about something else. Of course we can use the containerized solution with AWS ECS Fargate but there must be a simpler one.

Take a moment to imagine that we have a quite simple website with HTML files for web pages, CSS files for styling, and JS files for logic (of course images are also included as static content). When it comes to such a basic website, there is a better way to deploy it. No need to play around with cloud servers or docker images that can sometimes be a pain.

AWS already solved our problem and came up with this amazing feature called S3 static web hosting, that takes all the files in our bucket and makes a website out of them.

8.a Benefits of using static website hosting

1. The first benefit of using S3 static web hosting is that you as a developer don’t need to manage any servers. actually your entire application can be serverless! For example we can build a website that serves the content from S3, and if any logic is needed we can use Javascript functions to send requests to API gateway that triggers a Lambda function.

So we don’t need to manage the servers and we don’t need to configure any image like Nginx.

2. Because our website is a S3 bucket we are going to pay only for the storage of the bucket, no compute and networking modifications/costs.

3. By enabling bucket versioning we can easily monitor the versions of our files, without any git repository.

8.b Configuring the website

1. 1. Create a new bucket. Select a name for your bucket and select a region. Keep the ACLs disabled (happens by default because it’s recommended) and on the “Block Public Access settings for this bucket” window uncheck the box to allow public access to our bucket. You can specify versioning if you wish and encryption as you decide.
      
      
   2. Upload the website files to the bucket. If you want you can use the sample files stored here. Note that you need to change “YOUR_API_GATEWAY_ENDPOINT” in script.js (line 44) to match your API Gateway invoke URL.
      
      
   3. In the properties tabs, go all the way down and edit the “Static web hosting” configuration. Enable it.
      
      
   4. Select the index document to be “index.html” (if you are following my example) and save the changes.
      
      
   5. Now you can open the link of the s3 website and face an “access denied” error 🙂.

8.c Changing permissions

Although our bucket is not blocking public access we still need to give our users permissions to access the files. This can be done in two ways: The first is by using ACLs, which is not so recommended, the other is by using bucket policy.

Go to “Permissions” and scroll down to the bucket policy.

Use the following bucket policy to enable all users to access your bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
        }
    ]
}

8.d CORS

If you are still facing issues and your website is using files across buckets/regions/accounts it could be a CORS problem. Cross Origin Resource Sharing or CORS is the way to use files from other origins as well. For example, if you are using an index.html file in one bucket and this bucket is calling an image that is stored in another bucket.

To fix this issue you can navigate to the Permissions tab again and scroll all the way down to the CORS section and then edit the file according to your preferences.

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE"
        ],
        "AllowedOrigins": [
            "http://www.example2.com"
        ],
        "ExposeHeaders": []
    }
]

 

9. Create a chatbot using Lex as an alternative solution

As more and more websites start to use chatbots technology, AWS came with a new service called Amazon Lex that we can use as our chatbot engine in the architecture and save a little bit of time and effort in order to develop our application. This solution also solves the authentication problem that we’ve faced before, our chatbot’s endpoint won’t be publicly available for all, but only to our website. While we won’t need to develop a Lambda function ourselves and serve it with API gateway, there are still some modifications to make including adding Amazon Cognito to authenticate with Lex. Let’s check it out.

 

10. Amazon Lex

So Amazon Lex is our new chatbot backend. It is a fully managed artificial intelligence (AI) service with advanced natural language models to design, build, test, and deploy conversational interfaces in applications. In other words, this service comes to replace the Lambda function that handles our request, the OpenAI LLM that makes a response and the API Gateway because it has its own endpoint.

With Amazon Lex we are able to create different types of bots such as: We can create a bot for ordering a pizza or for scheduling a vacation, etc’. This time we are planning to create a Q&A bot.

10.a Create new Lex bot

1. 1. Enter the Amazon Lex service. Go to ‘Create bot’.
      
      
   2. Choose to create a blank bot.
      
      
   3. Enter a name as you want and tell AWS to create a new role for your Lex bot.
      
      
   4. Select ‘No’ in the COPPA, and keep the ‘Idle session timeout’ as the defaults.
      
      
   5. Next, you can modify the language and voice, but I would recommend to you to stick with the defaults (which is English US).
      
      
   6. That’s it you have a new bot! Let’s configure it.

10.b Creating intents

Intents is our way to describe common patterns or questions in our bot discussion. We can create an opening answer for the question “What’s up?” and configure the bot an answer with few variations, we can make a choice between answers. We can even ask the user something like “What kind of icecream do you like the most?” and give it three options to choose. Actually you can configure the bot as you wish for every conversation use case. From an intent we can also trigger a Lambda function that will perform some actions in our AWS account.

In our chatbot we are going to create two intents: The first one is a required intent because Lex won’t let you build the bot without this intent, so it doesn’t really matter, but still I’m going to guide you through it. The second one is the Kendra intent that is going to search the answer asked to the chatbot in the Kendra index, this is the important one. We also need to modify the Fallback intent for errors responding to messages.

Required intent:

1. 1. Add intent -> Add empty intent
      
      
   2. Give it a name, for example ‘required’
      
      
   3. In ‘Sample utterances’ add an example line ‘requiredintent’.
      
      
   4. Save the intent.

FallbackIntent intent:

• • You only need to change the closing response to something like: “Oops! Something went wrong. Please try again.”

Kendra intent:

1. 1. Add intent -> Use built-in intent
      
      
   2. Search for the AMAZON.KendraSearchIntent
      
      
   3. Give the intent a name and select your Kendra index from the available list.
      
      
   4. Add the intent.
      
      
   5. On the Fulfillment section enter “((x-amz-lex:kendra-search-response-answer-1)).” on a successful fulfillment. You can find more information about building the Kendra intent here.
      
      
   6. Save the intent.

Now after creating all the intents, we can build the bot and test it.

10.c Creating version and alias

The last step in the process is to create a bot version and a bot alias in order to deploy the bot. This process can be achieved simply by going to the ‘Bot versions’ tab and creating a new version. Then under ‘Deployments’ search for ‘Aliases’, create a new alias, give it a name and associate the alias with the version you’ve just created.

11. Cognito

Lex doesn’t have a public endpoint, so when we try connecting to the bot we will have to use some authentication method and access AWS API. The most common way to create this kind of connection is using AWS Cognito. Cognito and prevents unauthenticated users from accessing the bot. We can also configure Cognito to automatically load the Cognito login page and change our authentication requirements as we wish.

In our architecture the chatbot UI component requires a configuration object pointing to an existing Lex bot and to an Amazon Cognito Identity Pool to create credentials used to authenticate the Lex API calls from the browser.

11.a Create identity pool

1. 1. Go to Cognito service in the AWS management console and select “Identity pools” from the menu.
      
      
   2. Create a new identity pool, and because in our use case we don’t want to limit access to our chatbot we are going to choose the “Guest access” as the user access authentication.
      
      
   3. Create a new role for this identity pool and specify a name.
      
      
   4. Specify also an identity pool name.
      
      
   5. Create the identity pool.

11.b Configure the Cognito role

Our chatbot users are going to get the Cognito token from the website without any problems, but still they need permission to access the Lex service. So in order to achieve this goal navigate to the ‘User access’ tab and scroll all the way down to ‘Guest access’ section, and open the guest role in IAM.

If your chatbot is the only bot that you have in Lex, you can choose to add the AmazonLexRunBotsOnly directly which gives the users access to run all the bots in your account. This is not a best practice because you may accidentally forget about this access and create other bots. But the best practice is to create an inline policy to your chatbot:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lex:PostContent",
                "lex:PostText",
                "lex:PutSession",
                "lex:GetSession",
                "lex:DeleteSession",
                "lex:RecognizeText",
                "lex:RecognizeUtterance",
                "lex:StartConversation"
            ],
            "Resource": [
                "arn:aws:lex:AWS_REGION:ACCOUNT_ID:bot/BOT_ID",
                "arn:aws:lex:AWS_REGION:ACCOUNT_ID:bot-alias/*/BOT_ALIAS_ID"
            ]

        }
    ]
}

12. S3 Frontend for Lex

Our S3 website is going to look exactly the same, but the source code is a little bit different because we need to authenticate against Amazon Cognito and send the request to Lex instead of API Gateway. You can find the source code here.

Change the variables in script.js according to your own infrastructure:

• • AWS_REGION (line 10) = your Lex chatbot region

• • COGNITO_IDENTITY_POOL_ID (line 13) = your Cognito pool ID

• • BOT_ALIAS_ID (line 24) = your bot alias ID

• • BOT_ID (line 25) = your bot ID

• • SESSION_ID (line 28) = string for session name that you want to create

13. Conclusion

In wrapping up, we’ve journeyed through two game-changing ways to create a serverless chatbot using AWS services, with Amazon Kendra as our secret sauce. What started as a late-night hackathon scramble turned into a full-blown serverless adventure. We’ve seen how to mash up Kendra, OpenAI, Lambda, API Gateway, and S3 into a chatbot powerhouse that’s as flexible as it is impressive. Then we took a detour into Amazon Lex territory, showing how to streamline the whole process while still delivering powerful results. Both routes prove that serverless is the way to go for chatbots that are lean, mean, and ready to scale.

The best part? You don’t need to be an ML guru or have deep pockets to make it happen. Whether you’re a pro or new to this process, this guide has got you covered with practical steps and best practices to keep your bot secure and running smooth. So go ahead, give it a shot – your very own serverless chatbot is just a few AWS services away from becoming reality!