How do you bring the power of the ontology to your organization’s frontline workers using a voice agent?

Here’s a how we went about building a working version in under a week. We used the Model Context Protocol (MCP) framework to connect LLM clients to backend services and data sources, and set up voice to voice (V2V) agents that can provide business context and actionable insights to employees at the edge. In this way, MCP + Ontology architecture democratizes access to Business Intelligence in a secure and scalable manner.

Components:

1. AIP Ontology: Data model and Business Logic implemented in Palantir AIP that securely connects to enterprise data systems. This can be exposed to external applications using OSDK.
2. Node JS MCP Server: Typescript application that defines MCP tools that utilize OSDK to retrieve data, run actions, and write data. This is where we provide detailed prompt templates that help LLMs gain context on the structure and functionality of our ontology.
3. Voice to Voice Client Application: React web app built using ElevenLabs Client SDK, that connects to its Conversational AI model and interacts with our MCP Server to gain context and run MCP tools.

While searching for the perfect office space for our team, we came face to face with the challenges that forward-deployed staff in real estate companies encounter. Property managers and brokers, spread across multiple locations, spend an inordinate amount of time keeping track of leads, customers and inventory, alongside showing properties, negotiating and renewing leases, and preparing all the requisite paperwork.

Data on inventory availability, pricing, promotions and financing options is usually accessible only through desktop-accessible proprietary web apps, making it hard to access on-site, where it is most needed. As a result, brokers and property managers are dependent on back-office support leading to delays in communicating with customers and closing deals.

Using our voice-to-voice Sales Assist Agent, running on a smartphone browser, our broker can now share the following compound question & task list with our Sales Assist Agent in any language:

Our customer Atlas Solutions Inc. is looking to expand to a second office that has a 100 person capacity, they would like to be in close proximity to public transportation and their budget is similar to their current space. Additionally, they would like to be close to their main client Beacon Inc.’s main campus. Could you suggest inventory that satisfies this need and set up meetings with property managers in the top three locations for Tuesday morning.

zz final video 1.m4v