Integrate OpenAI into Your Code with Semantic Kernel

Who Am I? What Am I Doing Here?

	Catallaxy Services
	Curated SQL
	Finding Ghosts in Your Data

@feaselkl

Motivation

My goals in this talk:

• Review requirements for Azure OpenAI (AOAI)
• Gain an introduction to Semantic Kernel
• Build a simple Python application that uses Semantic Kernel
• Extend an existing C# Web API to use Semantic Kernel

What We'll Do

What We'll Do

What We'll Do

What We'll Do

Danger, Will Robinson!

Semantic Kernel is not 100% stable:

• Semantic Kernel is still fairly new and under heavy development
• Expect frequent API changes--there have been multiple complete API rewrites
• Some samples exist but make sure to check the date/version used!

Agenda

1. The Basics of Azure OpenAI
2. An Overview of Semantic Kernel
3. Trying out Semantic Kernel
4. Semantic Kernel in Practice

Azure OpenAI

Azure OpenAI (AOAI) is an Azure service that makes use of large language models from OpenAI. AOAI features and models typically come 3-6 months after OpenAI releases them.

If you have OpenAI credentials and use one of their services, that will not work for Azure OpenAI. These are entirely different services.

You will need a valid Azure subscription (not a free one) to use Azure OpenAI.

Choosing a Model

The Azure AI Foundry (https://ai.azure.com) includes a variety of models.

Choosing a Model - Cost

Prices can vary drastically between models. Simple pricing in East US (as of 2025-03-04) using basic models:

Choosing a Model - Other Considerations

In addition to cost, there are other considerations:

1. Regional availability - Not all models are always available in all regions
2. Model availability - Not all models are always available to your subscription. Azure provides a breakdown
3. Response time - There is a distinct trade-off between speed, accuracy, and cost

Create an Azure OpenAI Service

Create an Azure OpenAI Service

Using the Azure OpenAI Service

Using the Azure OpenAI Service

Embedding vs GPT Models

You'll need both an embedding model and a generative pre-trained transformer (GPT) model for Semantic Kernel.

Embedding models allow us to vectorize text. The most common are text-embedding-ada-002 and text-embedding-3 (large or small).

GPT models allow us to generate output. GPT models need input text that is already vectorized.

Tokens, Vectors, and Embeddings

Language models deal in tokens, unique parts of words.

An example:

Rule of thumb: in English, the average token is ~4 characters long.

Tokens, Vectors, and Embeddings

Each token corresponds to a vector. A vector is a numeric representation of text. GPT models have vectors of 1536 floats, meaning there are 1536 dimensions for comparison.

The main benefit to vectorization is that it retains much of the semantic context of words (or collections of tokens): similar vectors relate to similar concepts.

Tokens, Vectors, and Embeddings

One common algorithm for comparison is cosine similarity. It compares the angle of two vectors. A simplified presentation of this idea:

Tokens, Vectors, and Embeddings

The embedding is a list of tokens converted into numbers. Each token has a token ID, and we can create a list (or array) of these token IDs.

Each token ID ties back to a specific vector in multi-dimensional space.

This is what the large language model uses to perform its work.

Retrieval Augmented Generation

Retrieval Augmented Generation (RAG) is a pattern for sending local information to a language model for analysis.

Agenda

1. The Basics of Azure OpenAI
2. An Overview of Semantic Kernel
3. Trying out Semantic Kernel
4. Semantic Kernel in Practice

Semantic Kernel

Semantic Kernel is an open-source library designed to build AI agents and integrate them into your applications.

Semantic Kernel supports .NET (quite well), Python (fairly well), and Java (not that well). It is a competitor to LangChain and serves some of the same use cases.

Semantic Kernel - The Kernel

The kernel is the central focus, gaining access to inputs and outputs at each step of the process.

Semantic Kernel

The architectural motif of Semantic Kernel is a plug-in model.

Semantic Kernel

Plugins let you encapsulate existing APIs and code, to allow an AI engine to know what code to execute in order to satisfy your request.

Semantic Kernel - The Planner

Semantic Kernel uses function calling as the primary mechanism for planning and execution of tasks.

Semantic Kernel - An Example

Semantic Kernel can chain together a set of commands in order to satisfy a user request.

Agenda

1. The Basics of Azure OpenAI
2. An Overview of Semantic Kernel
3. Trying out Semantic Kernel
4. Semantic Kernel in Practice

What We'll Do

1. Review a .NET console application
2. Perform simple function calling using Semantic Kernel

Demo Time

Agenda

1. The Basics of Azure OpenAI
2. An Overview of Semantic Kernel
3. Trying out Semantic Kernel
4. Semantic Kernel in Practice

The Use Case

Contoso Suites is a luxury hotel chain with properties in seven countries. They have worked with Azure OpenAI before and have created a simple "chat with your data" application using the RAG pattern.

They want to extend their existing C#-based Web API service to become "smarter" and would like to see what Semantic Kernel can do to help.

What We'll Do

1. Review the existing Streamlit dashboard and chat with data
2. Configure function calling in Web API
3. Perform function calling in Streamlit
4. Develop a maintenance copilot

Demo Time

Wrapping Up

Over the course of this talk, we have learned what Semantic Kernel is, how it ties in with Azure OpenAI, and how we can extend an existing .NET Web API using Semantic Kernel.

Wrapping Up

To learn more, go here:
https://csmore.info/on/sk

And for help, contact me:
feasel@catallaxyservices.com | @feaselkl

Catallaxy Services consulting:
https://CSmore.info/contact