CANA, CRATE, and the chatbot

CANA has always encouraged its team members to think outside the box and test new ideas. A recent CANA CRATE project perfectly exemplifies this spirit. CRATE stands for the CANA Research and Technology Enterprise, and it's our way to foster CANA innovation and encourage technical exploration. Our intended result is the creation of new knowledge, theories, models, or datasets that can be used by others for analysis or further research. 

Not only that, we often find ways to do things better, and improve the process in which we turn ideas into reality.

Structurally, CANA’s CRATE projects are either externally or internally focused. An external example might be exploring energy-efficient technologies for military installations and equipment, while a recent CRATE - the BizOps Chatbot - represents an internal project. No matter what, these are good repetitions -  the more we do it, the better at it we become. 

CANA Data Science Analyst, Kiko Whiteley, recently presented "BizOps," a custom-built chatbot designed to streamline workflows and answer frequently asked CANA Business Operations questions. This isn't another off-the-shelf solution – it's CANA ingenuity in leveraging existing tools to create cost-effective and highly tailored AI solutions. The project goals were to address frequently asked questions and free up the team from repetitive inquiries; offer a centralized, editable knowledge base; provide accessibility; and explore in-house AI development by gaining insights into the process of building custom AI tools.

A Sneak Peek: BizOps in Action

At one of our regular CANA Analytics Roundtables, Kiko demonstrated the chatbot’s capabilities. When asked, "When does my Paid-Time-Off accrue?", the chatbot not only provided a summary answer, but also cited relevant company documents as its source. It also effortlessly answered more general CANA-specific questions demonstrating its ability to pull information from arbitrary text within its knowledge base. A key design element is how the chatbot presents its answers: a concise summary followed by clear assertions or claims directly backed up by evidence.

The Secret Sauce: Clever Engineering & Cost-Efficiency

Kiko highlighted the strategic approach taken in building BizOps. While managed AI services exist, they often come with a significant cost and less flexibility. CANA's solution combines several Google Cloud functionalities with open-source software to achieve a powerful yet economical outcome. The architecture leverages:

• Google's Gemini for natural language understanding and response generation.

• Google Chat API that enables seamless integration and accessibility across various devices.

• Open-source software forms the backbone of the chatbot's logic and knowledge retrieval.

• Containerization allows for efficient and low-cost cloud deployment, minimizing continuous running costs.

  
  

Decoding the Dialogue: The Power of Prompt Engineering

Kiko offered a carefully crafted prompt structure that guides the chatbot’s responses. The most effective approach involved clear instructions that guide the AI on how to interpret and answer; contextual information to provide relevant snippets from the knowledge base; the specific query from the user/team member, and final reminders to reinforce desired output formats and behaviors. The process of fetching the most relevant information is crucial. The BizOps chatbot employs:

• Document Chunking: Automatically splitting documents into manageable paragraph-sized segments.

• Vector Store: Transforming these text chunks into numerical representations.

• Semantic Search: Using machine learning to identify the text chunks most semantically similar to the user's question.

  
  

Interestingly, the Gemini model's generous context window allowed for the entire knowledge base to be included in the prompt as a safety net. The prompts were also specifically designed to ensure responses included direct quotes and clickable links to the source material, even linking to other Google Chats for announcements.

Key Takeaways & Future Potential

The team had some powerful takeaways:

• Combining Existing Tools: There is power in integrating readily available technologies for cost-effective solutions compared to expensive managed services.

• Strategic Pre-processing: It is important to handle as much processing as possible outside the Large Language Model (LLM) with clear instructions and examples for higher-quality responses.

• Navigating Platform Nuances: The project enabled CANA to acknowledge and document learnings specific to Google Cloud for future projects.

• Continuous Evolution: There is value in awareness of the constant stream of new tools and features within Google Cloud.

  
  

Looking Ahead

Excitingly, the Business Operations project has laid the groundwork for creating other internal chatbots within CANA. Furthermore, the team has identified potential pathways to leverage this knowledge and the developed tools for external, enterprise-level applications.

CANA continues to encourage team members to consider proposing their own innovative CRATE projects, emphasizing that even seemingly small experiments can yield valuable insights and advancements. This project showcases CANA's commitment to innovation and its ability to develop practical, cost-effective AI solutions by creatively combining existing technologies. The future of internal communication and efficiency at CANA looks brighter – and potentially more conversational!