Up next in 10
Architectural Design Patterns for Applications Leveraging LLMs | Software Architecture Conference
0 views
Aug 6, 2025
Come see a demo-heavy exploration into application architecture and design patterns that leverage Large Language Models (LLMs), transforming application design, development, and deployment. The session begins by showing the foundational technique of leveraging LLMs through direct invocation. This part focuses on the art of prompt engineering, crucial for tailoring LLM responses to the nuanced needs of applications. Following this, the session talks about bridging natural language queries into structured queries. This demonstrates how to enable powerful searches across both structured and semi-structured data repositories, by using LLMs to interpret and translate human queries into actionable database commands. Next, the session looks at the vector databases and how these work, allowing for nuanced searches that go beyond keyword matching and mere Boolean operations. All of these lead up to RAG apps -- A style of application that creates powerful apps for searching and analyzing vast quantities of data in natural language. The final part of the session looks at how LLMs work with other AI models. This shows how layering AI technologies to achieve a depth of interaction and functionality previously unattainable with single-model approaches, building applications that are richer and more engaging. š Conference Website: https://softwarearchitecture.live šŗ CSharp TV - Dev Streaming Destination http://csharp.tv š C# Corner - Community of Software and Data Developers https://www.c-sharpcorner.com
View Video Transcript
0:03
so my talk today is on llms um
0:07
so my talk today is on llms um
0:07
so my talk today is on llms um particularly like how to leverage llms
0:10
particularly like how to leverage llms
0:10
particularly like how to leverage llms in applications so I'm mostly concerned
0:12
in applications so I'm mostly concerned
0:13
in applications so I'm mostly concerned about the patterns that go into that
0:15
about the patterns that go into that
0:15
about the patterns that go into that okay I cannot tell you everything you'll
0:18
okay I cannot tell you everything you'll
0:18
okay I cannot tell you everything you'll ever want to know about llms in 30
0:21
ever want to know about llms in 30
0:21
ever want to know about llms in 30 minutes or so uh I do want to try to
0:23
minutes or so uh I do want to try to
0:23
minutes or so uh I do want to try to give you an overview of the the kind of
0:26
give you an overview of the the kind of
0:26
give you an overview of the the kind of components you know you need to think
0:28
components you know you need to think
0:28
components you know you need to think about hopefully in a in a bite-sized
0:30
about hopefully in a in a bite-sized
0:30
about hopefully in a in a bite-sized chunk so that you can kind of kind of
0:32
chunk so that you can kind of kind of
0:32
chunk so that you can kind of kind of put all the pieces together and see how
0:34
put all the pieces together and see how
0:34
put all the pieces together and see how all this stuff kind of fits together so
0:36
all this stuff kind of fits together so
0:36
all this stuff kind of fits together so that you can use it in applications so
0:39
that you can use it in applications so
0:39
that you can use it in applications so that's my goal today and so the patterns
0:41
that's my goal today and so the patterns
0:41
that's my goal today and so the patterns themselves would apply regardless of
0:43
themselves would apply regardless of
0:43
themselves would apply regardless of what llms you're using you could be
0:45
what llms you're using you could be
0:45
what llms you're using you could be using chat GPT um like applications
0:48
using chat GPT um like applications
0:48
using chat GPT um like applications using GPT 4 35 you could be using
0:51
using GPT 4 35 you could be using
0:51
using GPT 4 35 you could be using something off of hugging face where
0:53
something off of hugging face where
0:53
something off of hugging face where there's a lot of hosted models there you
0:56
there's a lot of hosted models there you
0:56
there's a lot of hosted models there you might be using something from Facebook
0:58
might be using something from Facebook
0:58
might be using something from Facebook or Google or something that you that's
1:00
or Google or something that you that's
1:00
or Google or something that you that's homegrown uh llms really are just a
1:03
homegrown uh llms really are just a
1:03
homegrown uh llms really are just a powerful tool that give you all kinds of
1:06
powerful tool that give you all kinds of
1:06
powerful tool that give you all kinds of different options for working with
1:08
different options for working with
1:09
different options for working with natural language so an llm itself is
1:11
natural language so an llm itself is
1:12
natural language so an llm itself is just a large AI model that's been built
1:14
just a large AI model that's been built
1:14
just a large AI model that's been built off of large collections of documents
1:18
off of large collections of documents
1:18
off of large collections of documents and data that basically tries to respond
1:22
and data that basically tries to respond
1:22
and data that basically tries to respond to natural language input and return
1:25
to natural language input and return
1:25
to natural language input and return natural language output so it's
1:27
natural language output so it's
1:27
natural language output so it's attempting to respond in the way that a
1:30
attempting to respond in the way that a
1:30
attempting to respond in the way that a human being would by looking at tons and
1:33
human being would by looking at tons and
1:33
human being would by looking at tons and tons of data that were written by humans
1:36
tons of data that were written by humans
1:36
tons of data that were written by humans and it's trying to understand context
1:38
and it's trying to understand context
1:38
and it's trying to understand context and all of the day ways that words are
1:40
and all of the day ways that words are
1:40
and all of the day ways that words are used and how they're how they relate to
1:43
used and how they're how they relate to
1:43
used and how they're how they relate to one another and particularly around the
1:46
one another and particularly around the
1:46
one another and particularly around the subject matters that you're asking it
1:48
subject matters that you're asking it
1:48
subject matters that you're asking it about and so many other things so that
1:50
about and so many other things so that
1:50
about and so many other things so that it can respond in an intelligent way so
1:52
it can respond in an intelligent way so
1:53
it can respond in an intelligent way so let's just Dive Right In and go with my
1:54
let's just Dive Right In and go with my
1:54
let's just Dive Right In and go with my first demo my first demo is around just
1:57
first demo my first demo is around just
1:57
first demo my first demo is around just you know just hey let's let just intro a
2:00
you know just hey let's let just intro a
2:00
you know just hey let's let just intro a um llm what's like one of the most basic
2:03
um llm what's like one of the most basic
2:03
um llm what's like one of the most basic things you can do with an llm and that's
2:05
things you can do with an llm and that's
2:05
things you can do with an llm and that's just ask you to do something now there's
2:07
just ask you to do something now there's
2:07
just ask you to do something now there's nothing about this that's special it's
2:09
nothing about this that's special it's
2:09
nothing about this that's special it's called zero shot prompts and basically
2:11
called zero shot prompts and basically
2:11
called zero shot prompts and basically what a zero shot prompt is is just do
2:14
what a zero shot prompt is is just do
2:14
what a zero shot prompt is is just do something and you ask it a question you
2:17
something and you ask it a question you
2:17
something and you ask it a question you tell it to do something with no
2:19
tell it to do something with no
2:19
tell it to do something with no additional you know context you just
2:21
additional you know context you just
2:21
additional you know context you just describe what you're looking for and
2:23
describe what you're looking for and
2:23
describe what you're looking for and what a zero shot prompt is attempting to
2:24
what a zero shot prompt is attempting to
2:24
what a zero shot prompt is attempting to do is just use what's baked into the
2:27
do is just use what's baked into the
2:27
do is just use what's baked into the model to produce some kind of result so
2:30
model to produce some kind of result so
2:30
model to produce some kind of result so um in this case I'm just telling it to
2:31
um in this case I'm just telling it to
2:31
um in this case I'm just telling it to generate me a quote generate me an
2:33
generate me a quote generate me an
2:33
generate me a quote generate me an inspirational quote I say tell me all
2:37
inspirational quote I say tell me all
2:37
inspirational quote I say tell me all about um llms or something like that and
2:41
about um llms or something like that and
2:41
about um llms or something like that and I was like get inspired and behind the
2:43
I was like get inspired and behind the
2:43
I was like get inspired and behind the scenes what this is doing is
2:45
scenes what this is doing is
2:45
scenes what this is doing is generating a prompt and I have the code
2:48
generating a prompt and I have the code
2:48
generating a prompt and I have the code sending it off to um open a I'm using uh
2:54
sending it off to um open a I'm using uh
2:54
sending it off to um open a I'm using uh U open AI models on Azure so I'm using
2:57
U open AI models on Azure so I'm using
2:57
U open AI models on Azure so I'm using uh hosted open AI on azure to do this
3:00
uh hosted open AI on azure to do this
3:00
uh hosted open AI on azure to do this and this is using GPT 35 behind the hood
3:03
and this is using GPT 35 behind the hood
3:03
and this is using GPT 35 behind the hood and I'm just packaging it up as a as a
3:06
and I'm just packaging it up as a as a
3:06
and I'm just packaging it up as a as a as a prompt sending that over to the API
3:09
as a prompt sending that over to the API
3:09
as a prompt sending that over to the API and saying give me back a quote and it
3:11
and saying give me back a quote and it
3:11
and saying give me back a quote and it gives me some back you know some some
3:14
gives me some back you know some some
3:14
gives me some back you know some some inspirational quote then I'll let you
3:16
inspirational quote then I'll let you
3:16
inspirational quote then I'll let you read that but it's you know talking
3:17
read that but it's you know talking
3:17
read that but it's you know talking about the journey and love of Mastery
3:19
about the journey and love of Mastery
3:19
about the journey and love of Mastery and flickering flames it's supposed to
3:20
and flickering flames it's supposed to
3:20
and flickering flames it's supposed to be poetic and philosophical at the same
3:22
be poetic and philosophical at the same
3:22
be poetic and philosophical at the same time among many other things and
3:25
time among many other things and
3:25
time among many other things and so the idea here is it's just everything
3:28
so the idea here is it's just everything
3:28
so the idea here is it's just everything that's baked into the L m is what it's
3:30
that's baked into the L m is what it's
3:30
that's baked into the L m is what it's using there's no additional context and
3:32
using there's no additional context and
3:32
using there's no additional context and it's just very simple back and forth
3:34
it's just very simple back and forth
3:34
it's just very simple back and forth request response for zero shot prompt
3:37
request response for zero shot prompt
3:37
request response for zero shot prompt and the the trick to these um is is just
3:41
and the the trick to these um is is just
3:41
and the the trick to these um is is just trying to get used to how a prompt will
3:45
trying to get used to how a prompt will
3:45
trying to get used to how a prompt will influence an llm there's nothing special
3:48
influence an llm there's nothing special
3:48
influence an llm there's nothing special about these they're they're limited
3:49
about these they're they're limited
3:49
about these they're they're limited youth they have very limited use in and
3:52
youth they have very limited use in and
3:52
youth they have very limited use in and real world applications but it's a good
3:54
real world applications but it's a good
3:54
real world applications but it's a good way to just kind of learn how prompts
3:55
way to just kind of learn how prompts
3:56
way to just kind of learn how prompts work and just get a feel for what you
3:57
work and just get a feel for what you
3:57
work and just get a feel for what you can do with these kinds of things so
3:59
can do with these kinds of things so
3:59
can do with these kinds of things so that's why included as kind of just an
4:00
that's why included as kind of just an
4:00
that's why included as kind of just an intro to this because yeah it's kind of
4:03
intro to this because yeah it's kind of
4:03
intro to this because yeah it's kind of cool okay you can interact with this and
4:05
cool okay you can interact with this and
4:05
cool okay you can interact with this and I can ask it a question tell it to do
4:06
I can ask it a question tell it to do
4:06
I can ask it a question tell it to do something and it gives me back a
4:08
something and it gives me back a
4:08
something and it gives me back a response it's very grammatically correct
4:11
response it's very grammatically correct
4:11
response it's very grammatically correct it means something it's got content
4:14
it means something it's got content
4:14
it means something it's got content that's specific to U what I'm looking
4:17
that's specific to U what I'm looking
4:17
that's specific to U what I'm looking for and so on so there's H there's a lot
4:18
for and so on so there's H there's a lot
4:18
for and so on so there's H there's a lot of stuff in this the next one is where
4:21
of stuff in this the next one is where
4:21
of stuff in this the next one is where things get a little bit more in the
4:22
things get a little bit more in the
4:22
things get a little bit more in the weeds and that's where we're going to
4:24
weeds and that's where we're going to
4:24
weeds and that's where we're going to talk about prompt engineering prompt
4:26
talk about prompt engineering prompt
4:26
talk about prompt engineering prompt engineering is another set of patterns
4:30
engineering is another set of patterns
4:30
engineering is another set of patterns that is really the heart and soul of
4:33
that is really the heart and soul of
4:33
that is really the heart and soul of everything that you do when you're
4:35
everything that you do when you're
4:35
everything that you do when you're developing applications using LMS so
4:38
developing applications using LMS so
4:38
developing applications using LMS so prompt engineering is the art of
4:40
prompt engineering is the art of
4:40
prompt engineering is the art of crafting the prompts that you send to
4:43
crafting the prompts that you send to
4:43
crafting the prompts that you send to the llm just like I did before except
4:45
the llm just like I did before except
4:46
the llm just like I did before except here there's a set of patterns that you
4:47
here there's a set of patterns that you
4:47
here there's a set of patterns that you can use and there's a ton of patterns I
4:49
can use and there's a ton of patterns I
4:49
can use and there's a ton of patterns I could talk about um but the basic look
4:53
could talk about um but the basic look
4:53
could talk about um but the basic look it looks like this you have an llm you
4:55
it looks like this you have an llm you
4:55
it looks like this you have an llm you have some kind of Client app uh and then
4:58
have some kind of Client app uh and then
4:58
have some kind of Client app uh and then you have you might or might not have
5:00
you have you might or might not have
5:00
you have you might or might not have external content or external context
5:03
external content or external context
5:03
external content or external context that you bring into the The Prompt most
5:06
that you bring into the The Prompt most
5:06
that you bring into the The Prompt most of the time when you're dealing with LMS
5:08
of the time when you're dealing with LMS
5:08
of the time when you're dealing with LMS of some kind you're going to bring in
5:09
of some kind you're going to bring in
5:09
of some kind you're going to bring in some external context so what I mean by
5:11
some external context so what I mean by
5:11
some external context so what I mean by that is I'm going to bring in some kind
5:14
that is I'm going to bring in some kind
5:14
that is I'm going to bring in some kind of data that I want to I want the llm to
5:17
of data that I want to I want the llm to
5:17
of data that I want to I want the llm to work with I'm going to tell it what I
5:19
work with I'm going to tell it what I
5:19
work with I'm going to tell it what I want it to do and I'm going to tell it
5:21
want it to do and I'm going to tell it
5:21
want it to do and I'm going to tell it uh here's the data I want you to work
5:22
uh here's the data I want you to work
5:23
uh here's the data I want you to work with and so I take the the instructions
5:25
with and so I take the the instructions
5:25
with and so I take the the instructions essentially and I take that data and I
5:27
essentially and I take that data and I
5:27
essentially and I take that data and I combine it and that's what I use to
5:29
combine it and that's what I use to
5:29
combine it and that's what I use to create my prompt I pass that back to the
5:31
create my prompt I pass that back to the
5:31
create my prompt I pass that back to the llm and then it returns back something
5:34
llm and then it returns back something
5:34
llm and then it returns back something to me now there's a lot of different
5:36
to me now there's a lot of different
5:36
to me now there's a lot of different patterns you can use with this and there
5:37
patterns you can use with this and there
5:38
patterns you can use with this and there I I could go into each one of these and
5:39
I I could go into each one of these and
5:39
I I could go into each one of these and spend a lot of time on each one of those
5:42
spend a lot of time on each one of those
5:42
spend a lot of time on each one of those but ones that I'm I'm going to call out
5:45
but ones that I'm I'm going to call out
5:45
but ones that I'm I'm going to call out here are are instruction prompting um
5:48
here are are instruction prompting um
5:48
here are are instruction prompting um instruction prompting is where you tell
5:49
instruction prompting is where you tell
5:49
instruction prompting is where you tell it to do something with the data you're
5:51
it to do something with the data you're
5:51
it to do something with the data you're sending and this is a very useful one
5:53
sending and this is a very useful one
5:53
sending and this is a very useful one now typically instruction prompting is
5:55
now typically instruction prompting is
5:55
now typically instruction prompting is just a set of instructions that you
5:57
just a set of instructions that you
5:57
just a set of instructions that you describe in Pros you can give it like a
5:59
describe in Pros you can give it like a
5:59
describe in Pros you can give it like a step
6:00
step
6:00
step step-by-step instructions if you want to
6:02
step-by-step instructions if you want to
6:02
step-by-step instructions if you want to do that but um it's very useful you just
6:05
do that but um it's very useful you just
6:05
do that but um it's very useful you just say here's my data here's what I want
6:06
say here's my data here's what I want
6:06
say here's my data here's what I want you to do with it and then the llm will
6:08
you to do with it and then the llm will
6:08
you to do with it and then the llm will try to understand what exactly you're
6:09
try to understand what exactly you're
6:09
try to understand what exactly you're trying to do another one that's very
6:12
trying to do another one that's very
6:12
trying to do another one that's very similar to that is called fuse shot
6:13
similar to that is called fuse shot
6:13
similar to that is called fuse shot prompting and fuse shot prompting is
6:17
prompting and fuse shot prompting is
6:17
prompting and fuse shot prompting is where you take something like
6:18
where you take something like
6:18
where you take something like instructions and data and you provide
6:20
instructions and data and you provide
6:20
instructions and data and you provide examples to go along with that so the
6:23
examples to go along with that so the
6:23
examples to go along with that so the examples that you would have would be
6:25
examples that you would have would be
6:25
examples that you would have would be the external context that I'm talking
6:26
the external context that I'm talking
6:26
the external context that I'm talking about here so in this case I would give
6:29
about here so in this case I would give
6:29
about here so in this case I would give it some You Know sample data uh that I
6:32
it some You Know sample data uh that I
6:32
it some You Know sample data uh that I want to use and I can use that sample
6:34
want to use and I can use that sample
6:34
want to use and I can use that sample data to say here's what I kind of want
6:36
data to say here's what I kind of want
6:36
data to say here's what I kind of want you to do for output here's my data
6:38
you to do for output here's my data
6:38
you to do for output here's my data here's here's here's my context I might
6:41
here's here's here's my context I might
6:41
here's here's here's my context I might say here's some pros here's some sample
6:43
say here's some pros here's some sample
6:43
say here's some pros here's some sample outputs and I want you to uh analyze my
6:48
outputs and I want you to uh analyze my
6:48
outputs and I want you to uh analyze my pros and make it look like whatever I'm
6:51
pros and make it look like whatever I'm
6:51
pros and make it look like whatever I'm giving in my sample output so that's a
6:53
giving in my sample output so that's a
6:53
giving in my sample output so that's a very useful pattern there so it's
6:55
very useful pattern there so it's
6:55
very useful pattern there so it's basically providing examples for desired
6:57
basically providing examples for desired
6:57
basically providing examples for desired inputs and output Pairs and instruction
6:59
inputs and output Pairs and instruction
6:59
inputs and output Pairs and instruction prompting is combining that with
7:01
prompting is combining that with
7:01
prompting is combining that with instruction prompting and then you get
7:02
instruction prompting and then you get
7:03
instruction prompting and then you get something that is going to be very
7:05
something that is going to be very
7:05
something that is going to be very useful for exact getting very specific
7:08
useful for exact getting very specific
7:08
useful for exact getting very specific outputs and I'll show you we'll have
7:09
outputs and I'll show you we'll have
7:09
outputs and I'll show you we'll have some examples of this later on um I do
7:12
some examples of this later on um I do
7:12
some examples of this later on um I do want to mention another one right here
7:14
want to mention another one right here
7:14
want to mention another one right here role playing um is another kind of
7:18
role playing um is another kind of
7:18
role playing um is another kind of prompt that is sometimes used with
7:20
prompt that is sometimes used with
7:20
prompt that is sometimes used with instruction prompting and Fus prompting
7:22
instruction prompting and Fus prompting
7:22
instruction prompting and Fus prompting you tell the llm what kind of role you
7:25
you tell the llm what kind of role you
7:25
you tell the llm what kind of role you want the llm to Pro to fulfill so in in
7:29
want the llm to Pro to fulfill so in in
7:29
want the llm to Pro to fulfill so in in the case of of an example I'm going to
7:30
the case of of an example I'm going to
7:30
the case of of an example I'm going to give later it's like you're a research
7:32
give later it's like you're a research
7:32
give later it's like you're a research assistant and so that kind of gives a
7:35
assistant and so that kind of gives a
7:35
assistant and so that kind of gives a color to The Prompt that the lolm will
7:37
color to The Prompt that the lolm will
7:37
color to The Prompt that the lolm will be able to uh use it's a okay I'm going
7:40
be able to uh use it's a okay I'm going
7:40
be able to uh use it's a okay I'm going to be doing some kind of research and
7:41
to be doing some kind of research and
7:41
to be doing some kind of research and I'm going to reply to the prompt in the
7:44
I'm going to reply to the prompt in the
7:44
I'm going to reply to the prompt in the same way that a research assistant would
7:46
same way that a research assistant would
7:46
same way that a research assistant would and based on the instructions and then
7:48
and based on the instructions and then
7:48
and based on the instructions and then maybe some examples I would then get
7:51
maybe some examples I would then get
7:51
maybe some examples I would then get back a response that's very similar to
7:53
back a response that's very similar to
7:53
back a response that's very similar to what a research assistant might do and I
7:56
what a research assistant might do and I
7:56
what a research assistant might do and I want to I I do want to mention uh
7:59
want to I I do want to mention uh
7:59
want to I I do want to mention uh another one here I'm not going to be
8:00
another one here I'm not going to be
8:00
another one here I'm not going to be able to get into all these down here but
8:01
able to get into all these down here but
8:01
able to get into all these down here but I do want to say U Chain of Thought
8:04
I do want to say U Chain of Thought
8:04
I do want to say U Chain of Thought prompting is another very powerful
8:05
prompting is another very powerful
8:05
prompting is another very powerful prompt pattern that I could spend a
8:07
prompt pattern that I could spend a
8:08
prompt pattern that I could spend a whole like half an hour talking about
8:10
whole like half an hour talking about
8:10
whole like half an hour talking about but this one basically is where you you
8:12
but this one basically is where you you
8:12
but this one basically is where you you have uh combinations of of instructions
8:16
have uh combinations of of instructions
8:16
have uh combinations of of instructions but it's multiple steps in the process
8:19
but it's multiple steps in the process
8:19
but it's multiple steps in the process so step one do this step two from the
8:21
so step one do this step two from the
8:21
so step one do this step two from the output from Step One do this step three
8:24
output from Step One do this step three
8:24
output from Step One do this step three from the output from step three do this
8:26
from the output from step three do this
8:26
from the output from step three do this and it gives the llm the ability to
8:28
and it gives the llm the ability to
8:28
and it gives the llm the ability to break down each section of your prompt
8:31
break down each section of your prompt
8:31
break down each section of your prompt and analyze each section individually
8:34
and analyze each section individually
8:34
and analyze each section individually and that way you have a so-called Chain
8:36
and that way you have a so-called Chain
8:36
and that way you have a so-called Chain of Thought that is giving some context
8:40
of Thought that is giving some context
8:40
of Thought that is giving some context and some some some kind of taxonomy to
8:43
and some some some kind of taxonomy to
8:43
and some some some kind of taxonomy to what you're trying to get out of the
8:44
what you're trying to get out of the
8:44
what you're trying to get out of the prompt and so that one's very useful for
8:46
prompt and so that one's very useful for
8:46
prompt and so that one's very useful for complex things and so I've used that one
8:49
complex things and so I've used that one
8:49
complex things and so I've used that one for solving uh complex problems that
8:52
for solving uh complex problems that
8:52
for solving uh complex problems that need multiple steps in their iteration
8:54
need multiple steps in their iteration
8:54
need multiple steps in their iteration so I'm not going to be able to demo that
8:55
so I'm not going to be able to demo that
8:55
so I'm not going to be able to demo that one as much today but I do want to talk
8:57
one as much today but I do want to talk
8:57
one as much today but I do want to talk I do want to demo uh you shot prompting
9:00
I do want to demo uh you shot prompting
9:00
I do want to demo uh you shot prompting instructions and uh also role playing
9:02
instructions and uh also role playing
9:02
instructions and uh also role playing because those are the three that I in my
9:04
because those are the three that I in my
9:04
because those are the three that I in my experience have been the most commonly
9:07
experience have been the most commonly
9:07
experience have been the most commonly used ones um uh in the my my experience
9:12
used ones um uh in the my my experience
9:12
used ones um uh in the my my experience here now uh some of these other ones are
9:16
here now uh some of these other ones are
9:16
here now uh some of these other ones are definitely uh why something that you
9:18
definitely uh why something that you
9:18
definitely uh why something that you might want to consider but for our
9:20
might want to consider but for our
9:20
might want to consider but for our purposes today let's just look at this
9:22
purposes today let's just look at this
9:22
purposes today let's just look at this so I'm going to do a little bit of
9:23
so I'm going to do a little bit of
9:23
so I'm going to do a little bit of prompt engineering for some output here
9:27
prompt engineering for some output here
9:27
prompt engineering for some output here and so what I've got down here is an
9:29
and so what I've got down here is an
9:29
and so what I've got down here is an example application in this one what I
9:32
example application in this one what I
9:32
example application in this one what I want to do is I want to do something
9:34
want to do is I want to do something
9:34
want to do is I want to do something with a a Wikipedia article so what
9:37
with a a Wikipedia article so what
9:37
with a a Wikipedia article so what what's going on behind the scenes for
9:38
what's going on behind the scenes for
9:38
what's going on behind the scenes for this demo is I have uh some code that's
9:40
this demo is I have uh some code that's
9:40
this demo is I have uh some code that's going to go out to Wikipedia it's going
9:42
going to go out to Wikipedia it's going
9:42
going to go out to Wikipedia it's going to scrape the content from Wikipedia and
9:45
to scrape the content from Wikipedia and
9:45
to scrape the content from Wikipedia and it's going to use that as my contact so
9:46
it's going to use that as my contact so
9:46
it's going to use that as my contact so that's just my just some data and I'm
9:48
that's just my just some data and I'm
9:49
that's just my just some data and I'm GNA tell my my llm to do something with
9:52
GNA tell my my llm to do something with
9:52
GNA tell my my llm to do something with that data now behind the scenes I'm I'm
9:55
that data now behind the scenes I'm I'm
9:55
that data now behind the scenes I'm I'm I'm combining the the Wikipedia article
9:57
I'm combining the the Wikipedia article
9:57
I'm combining the the Wikipedia article and then what I'm trying to describe
10:00
and then what I'm trying to describe
10:00
and then what I'm trying to describe um in my prompt here uh what I wanted to
10:02
um in my prompt here uh what I wanted to
10:02
um in my prompt here uh what I wanted to do I'm taking those and then passing
10:04
do I'm taking those and then passing
10:04
do I'm taking those and then passing that to the llm and it's going to return
10:05
that to the llm and it's going to return
10:05
that to the llm and it's going to return back some kind of results so how do I
10:07
back some kind of results so how do I
10:07
back some kind of results so how do I want to summarize this so I'm gonna use
10:09
want to summarize this so I'm gonna use
10:09
want to summarize this so I'm gonna use this article about a Tyrannosaurus Rex
10:12
this article about a Tyrannosaurus Rex
10:12
this article about a Tyrannosaurus Rex and I'm going to come down here and I'm
10:14
and I'm going to come down here and I'm
10:14
and I'm going to come down here and I'm gonna paste that URL in I'm gonna say
10:15
gonna paste that URL in I'm gonna say
10:15
gonna paste that URL in I'm gonna say how I want to summarize this U I'm gonna
10:18
how I want to summarize this U I'm gonna
10:18
how I want to summarize this U I'm gonna say can you summarize this um this
10:24
say can you summarize this um this
10:24
say can you summarize this um this article
10:26
article
10:26
article um using pirate speak and so I'm I'm
10:29
um using pirate speak and so I'm I'm
10:29
um using pirate speak and so I'm I'm going to use pirate speak you know like
10:31
going to use pirate speak you know like
10:31
going to use pirate speak you know like I might like I can't even talk like a
10:33
I might like I can't even talk like a
10:33
I might like I can't even talk like a pirate today um and let's just see what
10:36
pirate today um and let's just see what
10:36
pirate today um and let's just see what this does I'm gonna see if it will
10:37
this does I'm gonna see if it will
10:37
this does I'm gonna see if it will actually do this maybe it will summarize
10:39
actually do this maybe it will summarize
10:39
actually do this maybe it will summarize the article and um give me back some
10:42
the article and um give me back some
10:42
the article and um give me back some results that look like a pirate talk if
10:44
results that look like a pirate talk if
10:45
results that look like a pirate talk if they if all goes well now this one might
10:46
they if all goes well now this one might
10:46
they if all goes well now this one might take a minute to do um because it's got
10:49
take a minute to do um because it's got
10:49
take a minute to do um because it's got It's going to have to scrape a lot of
10:51
It's going to have to scrape a lot of
10:51
It's going to have to scrape a lot of content and analyze it so this is what's
10:53
content and analyze it so this is what's
10:53
content and analyze it so this is what's going on uh in the background here and
10:56
going on uh in the background here and
10:56
going on uh in the background here and so that's the scrape content and let's
10:58
so that's the scrape content and let's
10:58
so that's the scrape content and let's see what it's doing there's my results
11:02
see what it's doing there's my results
11:02
see what it's doing there's my results um it did summarize it let's where's my
11:05
um it did summarize it let's where's my
11:05
um it did summarize it let's where's my Pirates it doesn't look like it gave me
11:06
Pirates it doesn't look like it gave me
11:06
Pirates it doesn't look like it gave me Pirates gep uh describes the
11:08
Pirates gep uh describes the
11:08
Pirates gep uh describes the habitat um let's try it one more time
11:11
habitat um let's try it one more time
11:11
habitat um let's try it one more time let's see if it'll work sometimes if you
11:12
let's see if it'll work sometimes if you
11:12
let's see if it'll work sometimes if you run something again um it did summarize
11:15
run something again um it did summarize
11:15
run something again um it did summarize the article but it didn't do the pirate
11:16
the article but it didn't do the pirate
11:16
the article but it didn't do the pirate speak part
11:20
speak part
11:20
speak part um H Nice Shot nice try but it did it at
11:24
um H Nice Shot nice try but it did it at
11:24
um H Nice Shot nice try but it did it at least did summarize it for me um I
11:26
least did summarize it for me um I
11:26
least did summarize it for me um I thought that might work but it didn't so
11:29
thought that might work but it didn't so
11:29
thought that might work but it didn't so um maybe if I change this to pirate talk
11:32
um maybe if I change this to pirate talk
11:32
um maybe if I change this to pirate talk or using pirate talk or similar
11:35
or using pirate talk or similar
11:36
or using pirate talk or similar dialect and be more specific in my
11:38
dialect and be more specific in my
11:38
dialect and be more specific in my prompt um it will actually be able to uh
11:42
prompt um it will actually be able to uh
11:42
prompt um it will actually be able to uh give me something
11:44
give me something
11:44
give me something about that I don't know if this is going
11:46
about that I don't know if this is going
11:46
about that I don't know if this is going to work or not but it's worth a shot and
11:48
to work or not but it's worth a shot and
11:48
to work or not but it's worth a shot and you'll see that I'm getting back
11:49
you'll see that I'm getting back
11:49
you'll see that I'm getting back different results each time and that's
11:51
different results each time and that's
11:51
different results each time and that's because llms are non-deterministic oh
11:54
because llms are non-deterministic oh
11:54
because llms are non-deterministic oh that time it worked uh so here's the
11:56
that time it worked uh so here's the
11:56
that time it worked uh so here's the pirate talk right here okay um R miar me
12:00
pirate talk right here okay um R miar me
12:00
pirate talk right here okay um R miar me article be about fearsome Beast known as
12:02
article be about fearsome Beast known as
12:02
article be about fearsome Beast known as the T-Rex and so on so yeah that that
12:04
the T-Rex and so on so yeah that that
12:04
the T-Rex and so on so yeah that that that adding that little extra piece of
12:06
that adding that little extra piece of
12:06
that adding that little extra piece of context there uh did uh enable me to do
12:10
context there uh did uh enable me to do
12:10
context there uh did uh enable me to do a summary based on pirate talk and
12:12
a summary based on pirate talk and
12:12
a summary based on pirate talk and similar so on so that's the kind of I
12:14
similar so on so that's the kind of I
12:14
similar so on so that's the kind of I mean this is silly but you could
12:16
mean this is silly but you could
12:16
mean this is silly but you could definitely use this for other kinds of
12:17
definitely use this for other kinds of
12:17
definitely use this for other kinds of analysis or output depending on what you
12:20
analysis or output depending on what you
12:20
analysis or output depending on what you put into your prompt but this one I'm
12:22
put into your prompt but this one I'm
12:22
put into your prompt but this one I'm giving it instructions and I am giving
12:24
giving it instructions and I am giving
12:24
giving it instructions and I am giving it context so I'm combining like um
12:27
it context so I'm combining like um
12:27
it context so I'm combining like um instruction based prompting with context
12:29
instruction based prompting with context
12:29
instruction based prompting with context I'm not giving it examples so I'm not
12:30
I'm not giving it examples so I'm not
12:30
I'm not giving it examples so I'm not doing any kind of uh multi-shot
12:32
doing any kind of uh multi-shot
12:32
doing any kind of uh multi-shot prompting here but that is one of the
12:34
prompting here but that is one of the
12:34
prompting here but that is one of the patterns that I could definitely use for
12:36
patterns that I could definitely use for
12:36
patterns that I could definitely use for it so moving on from that there's
12:40
it so moving on from that there's
12:40
it so moving on from that there's another kind of pattern that you might
12:42
another kind of pattern that you might
12:42
another kind of pattern that you might that you can use with this kind of thing
12:44
that you can use with this kind of thing
12:44
that you can use with this kind of thing and this is natural language to code and
12:47
and this is natural language to code and
12:47
and this is natural language to code and this is a very powerful way to use llms
12:50
this is a very powerful way to use llms
12:50
this is a very powerful way to use llms now there's a lot of different ways that
12:52
now there's a lot of different ways that
12:52
now there's a lot of different ways that this actually can be leveraged now the
12:54
this actually can be leveraged now the
12:54
this actually can be leveraged now the way I'm going to demo it today is using
12:56
way I'm going to demo it today is using
12:56
way I'm going to demo it today is using natural language to SQL but one thing
12:59
natural language to SQL but one thing
12:59
natural language to SQL but one thing that you can do with this pattern is you
13:02
that you can do with this pattern is you
13:02
that you can do with this pattern is you can use natural language to describe
13:04
can use natural language to describe
13:04
can use natural language to describe algorithms and it will it has the
13:07
algorithms and it will it has the
13:07
algorithms and it will it has the ability to create code run that well not
13:09
ability to create code run that well not
13:09
ability to create code run that well not chat um you can have it create code you
13:12
chat um you can have it create code you
13:12
chat um you can have it create code you can then execute that code get the
13:14
can then execute that code get the
13:14
can then execute that code get the output from that code and then interact
13:16
output from that code and then interact
13:16
output from that code and then interact with the result sum and that's kind of
13:18
with the result sum and that's kind of
13:19
with the result sum and that's kind of what this is going to do but in this
13:20
what this is going to do but in this
13:20
what this is going to do but in this case I'm basically going to use the um
13:23
case I'm basically going to use the um
13:23
case I'm basically going to use the um llm to write SQL statements based on
13:26
llm to write SQL statements based on
13:26
llm to write SQL statements based on natural language prompts and so for this
13:29
natural language prompts and so for this
13:29
natural language prompts and so for this example I have a client application and
13:31
example I have a client application and
13:31
example I have a client application and I have a database uh execution
13:34
I have a database uh execution
13:34
I have a database uh execution environment for which is my database and
13:36
environment for which is my database and
13:36
environment for which is my database and I have an API and I have an llm so I'm
13:38
I have an API and I have an llm so I'm
13:38
I have an API and I have an llm so I'm going to describe what I'm going to look
13:40
going to describe what I'm going to look
13:40
going to describe what I'm going to look for and then the API is going to receive
13:43
for and then the API is going to receive
13:43
for and then the API is going to receive that it's then going to go all the AP
13:44
that it's then going to go all the AP
13:44
that it's then going to go all the AP the llm it's going to take that natural
13:47
the llm it's going to take that natural
13:47
the llm it's going to take that natural language request and translate that into
13:49
language request and translate that into
13:49
language request and translate that into some kind of code and then it's going to
13:51
some kind of code and then it's going to
13:51
some kind of code and then it's going to execute that code so this is exactly
13:53
execute that code so this is exactly
13:53
execute that code so this is exactly what I'm doing down here I've ingested
13:54
what I'm doing down here I've ingested
13:54
what I'm doing down here I've ingested some data into a search database and so
13:56
some data into a search database and so
13:56
some data into a search database and so my client application is going to call
13:58
my client application is going to call
13:58
my client application is going to call the llm get some SQL code and then take
14:01
the llm get some SQL code and then take
14:01
the llm get some SQL code and then take that back to my search database execute
14:03
that back to my search database execute
14:03
that back to my search database execute the SQL and then it's going to pass it
14:05
the SQL and then it's going to pass it
14:05
the SQL and then it's going to pass it back to my client application so in this
14:07
back to my client application so in this
14:07
back to my client application so in this execution what I did is I have a
14:12
execution what I did is I have a
14:12
execution what I did is I have a database of Jeopardy questions so what I
14:14
database of Jeopardy questions so what I
14:14
database of Jeopardy questions so what I did is I took the this data set and I
14:16
did is I took the this data set and I
14:16
did is I took the this data set and I put into a sqlite database that's
14:19
put into a sqlite database that's
14:19
put into a sqlite database that's running and I wired up an API and the
14:22
running and I wired up an API and the
14:22
running and I wired up an API and the API is basically receiving the call I'm
14:24
API is basically receiving the call I'm
14:24
API is basically receiving the call I'm gonna I'm gonna say enter my my question
14:27
gonna I'm gonna say enter my my question
14:27
gonna I'm gonna say enter my my question about Jeopardy and I then it's going to
14:31
about Jeopardy and I then it's going to
14:31
about Jeopardy and I then it's going to try to understand what I'm asking it and
14:34
try to understand what I'm asking it and
14:34
try to understand what I'm asking it and then generate a SQL query so behind the
14:36
then generate a SQL query so behind the
14:36
then generate a SQL query so behind the scenes what this is doing is I'm
14:38
scenes what this is doing is I'm
14:38
scenes what this is doing is I'm basically describing the schema like I
14:40
basically describing the schema like I
14:40
basically describing the schema like I have here in my prompt so behind the
14:43
have here in my prompt so behind the
14:43
have here in my prompt so behind the scenes I'm taking the question and I'm
14:45
scenes I'm taking the question and I'm
14:45
scenes I'm taking the question and I'm describing my schema which is basically
14:48
describing my schema which is basically
14:48
describing my schema which is basically what we might call um inshot prompting
14:51
what we might call um inshot prompting
14:51
what we might call um inshot prompting so I'm describing the output and the
14:53
so I'm describing the output and the
14:53
so I'm describing the output and the input using an example I'm describing
14:55
input using an example I'm describing
14:55
input using an example I'm describing this the exact schema of my database
14:57
this the exact schema of my database
14:57
this the exact schema of my database here's my question and then I'm telling
14:59
here's my question and then I'm telling
14:59
here's my question and then I'm telling it based on instructions what my output
15:01
it based on instructions what my output
15:01
it based on instructions what my output should be so in that case I'm using an
15:03
should be so in that case I'm using an
15:04
should be so in that case I'm using an inshot prompt with instructions to get
15:06
inshot prompt with instructions to get
15:06
inshot prompt with instructions to get back a SQL query and then I'm going to
15:08
back a SQL query and then I'm going to
15:08
back a SQL query and then I'm going to take that SQL query and execute it
15:10
take that SQL query and execute it
15:10
take that SQL query and execute it against the database and then that
15:11
against the database and then that
15:11
against the database and then that should return results so let's ask us a
15:14
should return results so let's ask us a
15:14
should return results so let's ask us a question here um can you um give me
15:20
question here um can you um give me
15:20
question here um can you um give me questions uh that mention Star
15:25
questions uh that mention Star
15:25
questions uh that mention Star Trek or Star Wars and that that's a
15:28
Trek or Star Wars and that that's a
15:28
Trek or Star Wars and that that's a pretty popular category on um Jeopardy
15:31
pretty popular category on um Jeopardy
15:31
pretty popular category on um Jeopardy so uh let's see let's submit the
15:32
so uh let's see let's submit the
15:32
so uh let's see let's submit the question and behind the scenes I should
15:34
question and behind the scenes I should
15:35
question and behind the scenes I should get back a query that looks like this so
15:37
get back a query that looks like this so
15:37
get back a query that looks like this so the llm generated this um so from the
15:41
the llm generated this um so from the
15:41
the llm generated this um so from the tables uh SQL query select from um
15:45
tables uh SQL query select from um
15:45
tables uh SQL query select from um questions and answers where question
15:47
questions and answers where question
15:47
questions and answers where question like Star Trek or question like Star
15:49
like Star Trek or question like Star
15:50
like Star Trek or question like Star Wars limit to 500 so that's my query
15:53
Wars limit to 500 so that's my query
15:53
Wars limit to 500 so that's my query that it generated based on this question
15:55
that it generated based on this question
15:55
that it generated based on this question right here and the the context I
15:57
right here and the the context I
15:57
right here and the the context I provided to it and here's the result so
16:00
provided to it and here's the result so
16:00
provided to it and here's the result so uh the question
16:02
uh the question
16:02
uh the question has so many questions about it they're
16:04
has so many questions about it they're
16:04
has so many questions about it they're all Star Trek or Star Wars related
16:05
all Star Trek or Star Wars related
16:05
all Star Trek or Star Wars related questions so you get a long list of
16:07
questions so you get a long list of
16:07
questions so you get a long list of these these questions related to that
16:10
these these questions related to that
16:10
these these questions related to that and so this is a very useful prep
16:12
and so this is a very useful prep
16:12
and so this is a very useful prep pattern if you need to do data analytics
16:14
pattern if you need to do data analytics
16:14
pattern if you need to do data analytics using llms so what you can tell it to do
16:18
using llms so what you can tell it to do
16:19
using llms so what you can tell it to do kind of as a second step is once you
16:21
kind of as a second step is once you
16:21
kind of as a second step is once you have the results back you can return
16:23
have the results back you can return
16:23
have the results back you can return those results back then you can pass the
16:24
those results back then you can pass the
16:24
those results back then you can pass the results back um to the LM and have it
16:28
results back um to the LM and have it
16:28
results back um to the LM and have it analyzed that in some way like provide
16:30
analyzed that in some way like provide
16:30
analyzed that in some way like provide me a summary of these questions um give
16:34
me a summary of these questions um give
16:34
me a summary of these questions um give me some some kind of output from it if
16:36
me some some kind of output from it if
16:36
me some some kind of output from it if you're generating some kind of code with
16:38
you're generating some kind of code with
16:38
you're generating some kind of code with this it doesn't have to be SQL code you
16:39
this it doesn't have to be SQL code you
16:39
this it doesn't have to be SQL code you can generate python code and execute the
16:41
can generate python code and execute the
16:41
can generate python code and execute the python code you can generate uh like any
16:44
python code you can generate uh like any
16:44
python code you can generate uh like any kind of scripted language that you can
16:46
kind of scripted language that you can
16:46
kind of scripted language that you can execute in kind of like a closed
16:48
execute in kind of like a closed
16:48
execute in kind of like a closed environment uh you could definitely do
16:50
environment uh you could definitely do
16:50
environment uh you could definitely do that so a database is one such
16:52
that so a database is one such
16:52
that so a database is one such environment like a uh a containerized
16:55
environment like a uh a containerized
16:55
environment like a uh a containerized environment to execute little code
16:57
environment to execute little code
16:57
environment to execute little code Snippets would be another such
16:58
Snippets would be another such
16:58
Snippets would be another such environment you generally want to do
17:00
environment you generally want to do
17:00
environment you generally want to do this though in an environment that is
17:02
this though in an environment that is
17:02
this though in an environment that is secure uh you don't want to do this in
17:04
secure uh you don't want to do this in
17:04
secure uh you don't want to do this in an environment that has wide or broad
17:06
an environment that has wide or broad
17:06
an environment that has wide or broad permissions because it could generate
17:08
permissions because it could generate
17:08
permissions because it could generate code that could be harmful so if I was
17:09
code that could be harmful so if I was
17:10
code that could be harmful so if I was going to do this against a database I
17:11
going to do this against a database I
17:11
going to do this against a database I wouldn't want to make sure that it's got
17:12
wouldn't want to make sure that it's got
17:12
wouldn't want to make sure that it's got only read access it can't do things like
17:15
only read access it can't do things like
17:15
only read access it can't do things like delete or update or any kind of data
17:17
delete or update or any kind of data
17:17
delete or update or any kind of data manipulation with the prompt I'm going
17:19
manipulation with the prompt I'm going
17:19
manipulation with the prompt I'm going to be running or at least the user I'm
17:20
to be running or at least the user I'm
17:21
to be running or at least the user I'm going to be running this execution of
17:22
going to be running this execution of
17:22
going to be running this execution of the query under so it can only read data
17:25
the query under so it can only read data
17:25
the query under so it can only read data then and I would generally if I was
17:28
then and I would generally if I was
17:28
then and I would generally if I was going to be doing this I would probably
17:29
going to be doing this I would probably
17:29
going to be doing this I would probably even create a separate database for the
17:31
even create a separate database for the
17:31
even create a separate database for the purpose of doing this kind of
17:33
purpose of doing this kind of
17:33
purpose of doing this kind of application just to isolate it not even
17:36
application just to isolate it not even
17:36
application just to isolate it not even have it run in the same database as my
17:38
have it run in the same database as my
17:38
have it run in the same database as my production data that's feeding into this
17:40
production data that's feeding into this
17:40
production data that's feeding into this and which is why I would just create a
17:42
and which is why I would just create a
17:42
and which is why I would just create a readon database that only um a single
17:46
readon database that only um a single
17:46
readon database that only um a single user the user executing the code could
17:48
user the user executing the code could
17:48
user the user executing the code could run that way they can do no no harm to
17:50
run that way they can do no no harm to
17:50
run that way they can do no no harm to the original data and they can really do
17:52
the original data and they can really do
17:52
the original data and they can really do no harm to the environment if they were
17:54
no harm to the environment if they were
17:54
no harm to the environment if they were a if they were somehow able to produce a
17:57
a if they were somehow able to produce a
17:57
a if they were somehow able to produce a query or some kind of EX extion that
17:59
query or some kind of EX extion that
17:59
query or some kind of EX extion that would violate some kind of security
18:01
would violate some kind of security
18:01
would violate some kind of security parameters so just a caveat there when
18:03
parameters so just a caveat there when
18:03
parameters so just a caveat there when you're doing this but it's a very
18:04
you're doing this but it's a very
18:04
you're doing this but it's a very powerful tool to enable non-technical
18:07
powerful tool to enable non-technical
18:07
powerful tool to enable non-technical people to uh ask questions about data
18:10
people to uh ask questions about data
18:10
people to uh ask questions about data that's in a database without having to
18:12
that's in a database without having to
18:12
that's in a database without having to know any SQL at all so that's just a
18:14
know any SQL at all so that's just a
18:14
know any SQL at all so that's just a it's just a really uh neat way to do
18:16
it's just a really uh neat way to do
18:16
it's just a really uh neat way to do that um Vector searches are another
18:19
that um Vector searches are another
18:19
that um Vector searches are another similar way and this is a very similar
18:21
similar way and this is a very similar
18:21
similar way and this is a very similar uh pattern and this is utilizing Vector
18:24
uh pattern and this is utilizing Vector
18:24
uh pattern and this is utilizing Vector searching in in a database so in this
18:26
searching in in a database so in this
18:26
searching in in a database so in this one where we have uh vectors that
18:29
one where we have uh vectors that
18:29
one where we have uh vectors that have uh a very similar pattern where
18:32
have uh a very similar pattern where
18:32
have uh a very similar pattern where we're ingesting data but we're putting
18:33
we're ingesting data but we're putting
18:33
we're ingesting data but we're putting into a vector database um so in a vector
18:36
into a vector database um so in a vector
18:36
into a vector database um so in a vector database we're basically representing
18:38
database we're basically representing
18:38
database we're basically representing stuff in in multiple Dimensions we're
18:39
stuff in in multiple Dimensions we're
18:39
stuff in in multiple Dimensions we're looking for similarities or semantic
18:41
looking for similarities or semantic
18:41
looking for similarities or semantic similarities so I basically took that
18:43
similarities so I basically took that
18:43
similarities so I basically took that same data set and when this one instead
18:45
same data set and when this one instead
18:45
same data set and when this one instead of doing uh generating query I'm going
18:48
of doing uh generating query I'm going
18:48
of doing uh generating query I'm going to generate what's called an embedding
18:50
to generate what's called an embedding
18:50
to generate what's called an embedding do a SQL search and that's going to
18:51
do a SQL search and that's going to
18:51
do a SQL search and that's going to return results this way this should use
18:53
return results this way this should use
18:53
return results this way this should use as an embedding model not so much a data
18:55
as an embedding model not so much a data
18:55
as an embedding model not so much a data uh this doesn't use so much a um an llm
18:58
uh this doesn't use so much a um an llm
18:58
uh this doesn't use so much a um an llm but it's a kind of llm where it's going
19:00
but it's a kind of llm where it's going
19:00
but it's a kind of llm where it's going to return an embedding and then I'm
19:02
to return an embedding and then I'm
19:02
to return an embedding and then I'm going to pass it off to a vector
19:03
going to pass it off to a vector
19:03
going to pass it off to a vector database and it's going to search on
19:04
database and it's going to search on
19:04
database and it's going to search on that one so and on this one I'm gonna
19:07
that one so and on this one I'm gonna
19:07
that one so and on this one I'm gonna ask can you give
19:09
ask can you give
19:09
ask can you give me questions about Star Trek and let's
19:13
me questions about Star Trek and let's
19:13
me questions about Star Trek and let's see what this does uh or sci-fi let's
19:16
see what this does uh or sci-fi let's
19:16
see what this does uh or sci-fi let's just say that just so I get some kind of
19:18
just say that just so I get some kind of
19:18
just say that just so I get some kind of context here
19:21
context here
19:21
context here um and
19:23
um and
19:23
um and search and it's going to give me results
19:26
search and it's going to give me results
19:26
search and it's going to give me results back using Vector searches so this is
19:28
back using Vector searches so this is
19:28
back using Vector searches so this is based on similarity searches so in this
19:30
based on similarity searches so in this
19:30
based on similarity searches so in this case I'm not going to get back stuff
19:32
case I'm not going to get back stuff
19:32
case I'm not going to get back stuff that's mentioning Star Trek I'm going to
19:33
that's mentioning Star Trek I'm going to
19:33
that's mentioning Star Trek I'm going to get stuff that's in the vein of Star
19:35
get stuff that's in the vein of Star
19:35
get stuff that's in the vein of Star Trek so science fiction would be in the
19:37
Trek so science fiction would be in the
19:37
Trek so science fiction would be in the vein so you can see stuff about Star
19:38
vein so you can see stuff about Star
19:38
vein so you can see stuff about Star Trek but we see some stuff um about
19:42
Trek but we see some stuff um about
19:42
Trek but we see some stuff um about Battle Star galactico about some
19:43
Battle Star galactico about some
19:44
Battle Star galactico about some hitchhikers gu of the Galaxy and other
19:46
hitchhikers gu of the Galaxy and other
19:46
hitchhikers gu of the Galaxy and other other kinds of things like that that are
19:49
other kinds of things like that that are
19:49
other kinds of things like that that are related to science fiction because of
19:51
related to science fiction because of
19:51
related to science fiction because of its similar context so this is about
19:54
its similar context so this is about
19:54
its similar context so this is about context searching not about matching
19:56
context searching not about matching
19:56
context searching not about matching Things based on Boolean searches but
19:58
Things based on Boolean searches but
19:58
Things based on Boolean searches but things that are semantically similar to
19:59
things that are semantically similar to
19:59
things that are semantically similar to it and this goes through a whole process
20:01
it and this goes through a whole process
20:01
it and this goes through a whole process of embedding that data in the database
20:03
of embedding that data in the database
20:03
of embedding that data in the database and being able to search on it this is a
20:04
and being able to search on it this is a
20:05
and being able to search on it this is a whole another topic but it's very useful
20:07
whole another topic but it's very useful
20:07
whole another topic but it's very useful and it's it really leads into the next
20:10
and it's it really leads into the next
20:10
and it's it really leads into the next kind of retrieval which is rag apps so
20:13
kind of retrieval which is rag apps so
20:13
kind of retrieval which is rag apps so rag apps are probably I wouldn't say the
20:15
rag apps are probably I wouldn't say the
20:15
rag apps are probably I wouldn't say the penultimate solution but they're really
20:17
penultimate solution but they're really
20:17
penultimate solution but they're really where all of this kind of comes together
20:19
where all of this kind of comes together
20:19
where all of this kind of comes together where you're combining searching with
20:21
where you're combining searching with
20:21
where you're combining searching with prompt engineering and and your and data
20:25
prompt engineering and and your and data
20:25
prompt engineering and and your and data analytics to provide a more holistic
20:27
analytics to provide a more holistic
20:27
analytics to provide a more holistic application that can provide you uh the
20:30
application that can provide you uh the
20:30
application that can provide you uh the ability to provide natural language
20:32
ability to provide natural language
20:32
ability to provide natural language queries into this stuff retrieve data
20:34
queries into this stuff retrieve data
20:34
queries into this stuff retrieve data and then analyze the results as a uh
20:37
and then analyze the results as a uh
20:37
and then analyze the results as a uh from that so in this particular example
20:40
from that so in this particular example
20:40
from that so in this particular example I have an embedding model that I used in
20:42
I have an embedding model that I used in
20:42
I have an embedding model that I used in my Vector database I have the general of
20:44
my Vector database I have the general of
20:44
my Vector database I have the general of AI to analyze that I also have a client
20:46
AI to analyze that I also have a client
20:46
AI to analyze that I also have a client app and the API that orchestrates all
20:48
app and the API that orchestrates all
20:48
app and the API that orchestrates all that so I basically for this example I'm
20:50
that so I basically for this example I'm
20:50
that so I basically for this example I'm gonna I embedded a bunch of resumƩs in
20:53
gonna I embedded a bunch of resumƩs in
20:53
gonna I embedded a bunch of resumƩs in this database right here and I'm going
20:55
this database right here and I'm going
20:55
this database right here and I'm going to then query that database using a
20:57
to then query that database using a
20:57
to then query that database using a vector search and this is using that
20:59
vector search and this is using that
20:59
vector search and this is using that similarity search not the SQL search
21:01
similarity search not the SQL search
21:01
similarity search not the SQL search however you could use SQL searches if
21:03
however you could use SQL searches if
21:03
however you could use SQL searches if you wanted to uh to do your retrieval
21:06
you wanted to uh to do your retrieval
21:06
you wanted to uh to do your retrieval rather than doing Vector searches Vector
21:07
rather than doing Vector searches Vector
21:07
rather than doing Vector searches Vector searches are typically the most commonly
21:09
searches are typically the most commonly
21:09
searches are typically the most commonly used in rag apps but you can use other
21:11
used in rag apps but you can use other
21:11
used in rag apps but you can use other kinds of retrieval like SQL retrieval
21:13
kinds of retrieval like SQL retrieval
21:13
kinds of retrieval like SQL retrieval like we just saw and I'm getting back
21:15
like we just saw and I'm getting back
21:15
like we just saw and I'm getting back something from a data stat this is not
21:17
something from a data stat this is not
21:17
something from a data stat this is not data stored in the llm this is data
21:19
data stored in the llm this is data
21:19
data stored in the llm this is data stored in the database so I pull back
21:21
stored in the database so I pull back
21:21
stored in the database so I pull back that and then I G the the model to
21:23
that and then I G the the model to
21:23
that and then I G the the model to analyze it so in this particular example
21:26
analyze it so in this particular example
21:26
analyze it so in this particular example I have ingested some redacted rums and
21:29
I have ingested some redacted rums and
21:29
I have ingested some redacted rums and I'm gonna say can you give me
21:32
I'm gonna say can you give me
21:32
I'm gonna say can you give me resumƩs about
21:36
resumƩs about
21:36
resumƩs about candidates who
21:38
candidates who
21:38
candidates who know
21:41
know
21:41
know JavaScript and um have management
21:46
JavaScript and um have management
21:46
JavaScript and um have management experience and let's see what this does
21:48
experience and let's see what this does
21:48
experience and let's see what this does this will generate an embedding uh from
21:52
this will generate an embedding uh from
21:53
this will generate an embedding uh from this question right here and it's going
21:55
this question right here and it's going
21:55
this question right here and it's going to then ask questions uh to this Vector
21:58
to then ask questions uh to this Vector
21:58
to then ask questions uh to this Vector database base and it's going to try to
21:59
database base and it's going to try to
21:59
database base and it's going to try to find things that match that data and
22:01
find things that match that data and
22:01
find things that match that data and this takes a little bit to do once it's
22:03
this takes a little bit to do once it's
22:03
this takes a little bit to do once it's got the results actually that was pretty
22:04
got the results actually that was pretty
22:04
got the results actually that was pretty quick and then it's going to pull back
22:06
quick and then it's going to pull back
22:06
quick and then it's going to pull back some results from the vector database
22:07
some results from the vector database
22:07
some results from the vector database and it's going to send those results
22:09
and it's going to send those results
22:09
and it's going to send those results back to the llm to an analyze them and
22:11
back to the llm to an analyze them and
22:11
back to the llm to an analyze them and then it's going to give me a link to the
22:13
then it's going to give me a link to the
22:13
then it's going to give me a link to the actual the actual uh resume right here
22:16
actual the actual uh resume right here
22:16
actual the actual uh resume right here and so it's quoting this so This Is
22:18
and so it's quoting this so This Is
22:18
and so it's quoting this so This Is Telling Me here's some result results
22:20
Telling Me here's some result results
22:20
Telling Me here's some result results from uh from the data in fact this
22:23
from uh from the data in fact this
22:23
from uh from the data in fact this doesn't look exactly right but it says
22:25
doesn't look exactly right but it says
22:25
doesn't look exactly right but it says uh software developer has an entire
22:27
uh software developer has an entire
22:27
uh software developer has an entire workflow develop
22:29
workflow develop
22:29
workflow develop you know doing a lot of different things
22:30
you know doing a lot of different things
22:30
you know doing a lot of different things with uh uh different kinds of things web
22:33
with uh uh different kinds of things web
22:33
with uh uh different kinds of things web developer senior developer and so on
22:35
developer senior developer and so on
22:35
developer senior developer and so on it's got some different
22:38
it's got some different
22:38
it's got some different um uh things right here this these it
22:40
um uh things right here this these it
22:41
um uh things right here this these it says it couldn't find any experience in
22:42
says it couldn't find any experience in
22:42
says it couldn't find any experience in these right here I apologize but
22:43
these right here I apologize but
22:43
these right here I apologize but couldn't find any rumes or the for
22:45
couldn't find any rumes or the for
22:45
couldn't find any rumes or the for JavaScript experience this one came out
22:47
JavaScript experience this one came out
22:47
JavaScript experience this one came out of the banking industry right here
22:49
of the banking industry right here
22:49
of the banking industry right here that's interesting um so it says
22:51
that's interesting um so it says
22:51
that's interesting um so it says somebody that's got some qualifications
22:53
somebody that's got some qualifications
22:53
somebody that's got some qualifications around JavaScript HTML and so on but
22:55
around JavaScript HTML and so on but
22:55
around JavaScript HTML and so on but they also have work experience
22:56
they also have work experience
22:56
they also have work experience developing applications and management
22:58
developing applications and management
22:58
developing applications and management experience right here so this one might
22:59
experience right here so this one might
22:59
experience right here so this one might be a good candidate and so I'd click on
23:01
be a good candidate and so I'd click on
23:01
be a good candidate and so I'd click on that resume right there and I would see
23:03
that resume right there and I would see
23:03
that resume right there and I would see you know kind of what it's looking for
23:05
you know kind of what it's looking for
23:05
you know kind of what it's looking for and so I can see software engineer and
23:06
and so I can see software engineer and
23:06
and so I can see software engineer and say it's 3 to n some experience here and
23:09
say it's 3 to n some experience here and
23:09
say it's 3 to n some experience here and I can actually review the management
23:11
I can actually review the management
23:11
I can actually review the management experience and see that they're web
23:12
experience and see that they're web
23:12
experience and see that they're web developer as well so this might be a
23:14
developer as well so this might be a
23:14
developer as well so this might be a candidate based on the kinds of
23:16
candidate based on the kinds of
23:16
candidate based on the kinds of Criterion that I gave it so uh this is a
23:19
Criterion that I gave it so uh this is a
23:19
Criterion that I gave it so uh this is a rag app using a lot of these patterns
23:20
rag app using a lot of these patterns
23:20
rag app using a lot of these patterns that we've talked about so this is a
23:22
that we've talked about so this is a
23:22
that we've talked about so this is a very common one right here and of all
23:24
very common one right here and of all
23:24
very common one right here and of all the ones I talk about it just brings
23:26
the ones I talk about it just brings
23:26
the ones I talk about it just brings together prompt engineering natural
23:27
together prompt engineering natural
23:27
together prompt engineering natural language processing and searching into a
23:30
language processing and searching into a
23:30
language processing and searching into a single application where you then U
23:33
single application where you then U
23:33
single application where you then U analyze data and then get back results
23:35
analyze data and then get back results
23:35
analyze data and then get back results and then you can use that now one last
23:37
and then you can use that now one last
23:37
and then you can use that now one last thing that you can do with this kind of
23:39
thing that you can do with this kind of
23:39
thing that you can do with this kind of thing is you can actually combine
23:40
thing is you can actually combine
23:40
thing is you can actually combine different kinds of input for your
23:42
different kinds of input for your
23:42
different kinds of input for your multimodal input into uh application so
23:45
multimodal input into uh application so
23:45
multimodal input into uh application so if I wanted to build an application that
23:47
if I wanted to build an application that
23:47
if I wanted to build an application that was kind of like a rag app or I wanted
23:49
was kind of like a rag app or I wanted
23:49
was kind of like a rag app or I wanted to use prompt engineering combine it
23:51
to use prompt engineering combine it
23:51
to use prompt engineering combine it with this I can use multimodal input and
23:54
with this I can use multimodal input and
23:54
with this I can use multimodal input and this is where I can use other models or
23:56
this is where I can use other models or
23:56
this is where I can use other models or I maybe I can use image process
23:58
I maybe I can use image process
23:58
I maybe I can use image process processing or whatever to generate
24:00
processing or whatever to generate
24:00
processing or whatever to generate context that I can then pass into llms
24:03
context that I can then pass into llms
24:03
context that I can then pass into llms well in this case right here I have an
24:05
well in this case right here I have an
24:05
well in this case right here I have an AI model that can do Vision speech
24:07
AI model that can do Vision speech
24:07
AI model that can do Vision speech recognitions and so on and I can then
24:10
recognitions and so on and I can then
24:10
recognitions and so on and I can then have it generate context which would be
24:13
have it generate context which would be
24:13
have it generate context which would be some kind of text that I can then use as
24:16
some kind of text that I can then use as
24:16
some kind of text that I can then use as part of a prompt engineering and then
24:18
part of a prompt engineering and then
24:18
part of a prompt engineering and then return the results back from the llm in
24:21
return the results back from the llm in
24:21
return the results back from the llm in this case so this is a very useful
24:23
this case so this is a very useful
24:23
this case so this is a very useful pattern if you want to do applications
24:25
pattern if you want to do applications
24:25
pattern if you want to do applications like Alexa or similar things where
24:27
like Alexa or similar things where
24:27
like Alexa or similar things where you're doing trans cribe text or you're
24:29
you're doing trans cribe text or you're
24:29
you're doing trans cribe text or you're trying to have some kind of intelligent
24:31
trying to have some kind of intelligent
24:31
trying to have some kind of intelligent application where you're doing voice
24:33
application where you're doing voice
24:33
application where you're doing voice recognition or maybe a video processing
24:35
recognition or maybe a video processing
24:35
recognition or maybe a video processing and so on so in this case right here I'm
24:37
and so on so in this case right here I'm
24:37
and so on so in this case right here I'm going to record myself talking into my
24:40
going to record myself talking into my
24:40
going to record myself talking into my microphone and it's I hope it's going to
24:42
microphone and it's I hope it's going to
24:42
microphone and it's I hope it's going to transcribe the text in this case it's
24:45
transcribe the text in this case it's
24:45
transcribe the text in this case it's gonna I'm asking it um to do something
24:48
gonna I'm asking it um to do something
24:48
gonna I'm asking it um to do something with it so I I really um and so this is
24:51
with it so I I really um and so this is
24:52
with it so I I really um and so this is my prompt here just um want you to make
24:56
my prompt here just um want you to make
24:56
my prompt here just um want you to make the output
24:58
the output
24:58
the output sound more professional whenever I'm
25:01
sound more professional whenever I'm
25:01
sound more professional whenever I'm speaking for my
25:05
speaking for my
25:05
speaking for my transcription and that's my um prompt
25:08
transcription and that's my um prompt
25:08
transcription and that's my um prompt right there and so I'm GNA say start
25:10
right there and so I'm GNA say start
25:10
right there and so I'm GNA say start recording right here and I'm gonna say
25:12
recording right here and I'm gonna say
25:12
recording right here and I'm gonna say this is me I'm gonna say well this is me
25:15
this is me I'm gonna say well this is me
25:15
this is me I'm gonna say well this is me talking into the microphone and I'm
25:17
talking into the microphone and I'm
25:17
talking into the microphone and I'm telling it to do something so behind the
25:19
telling it to do something so behind the
25:19
telling it to do something so behind the scenes it's recording something and
25:21
scenes it's recording something and
25:21
scenes it's recording something and while it's recording I want it to
25:23
while it's recording I want it to
25:23
while it's recording I want it to transcribe this into text and then once
25:25
transcribe this into text and then once
25:25
transcribe this into text and then once it's done I want to see the adaptation
25:26
it's done I want to see the adaptation
25:26
it's done I want to see the adaptation so I'm hit stop recording now
25:29
so I'm hit stop recording now
25:29
so I'm hit stop recording now and it's going to then hopefully
25:30
and it's going to then hopefully
25:31
and it's going to then hopefully transcribe that and this my raw
25:33
transcribe that and this my raw
25:33
transcribe that and this my raw transcription right here this is me
25:34
transcription right here this is me
25:34
transcription right here this is me talking in the microphone that's exactly
25:35
talking in the microphone that's exactly
25:35
talking in the microphone that's exactly what I said and this is the adaptation
25:37
what I said and this is the adaptation
25:38
what I said and this is the adaptation that it gave right here I'm speaking
25:39
that it gave right here I'm speaking
25:39
that it gave right here I'm speaking into the microphone um it gave
25:42
into the microphone um it gave
25:42
into the microphone um it gave me uh behind the scenes it's processing
25:45
me uh behind the scenes it's processing
25:45
me uh behind the scenes it's processing and it's just smoothing out what I said
25:47
and it's just smoothing out what I said
25:47
and it's just smoothing out what I said uh right here but it's using an
25:50
uh right here but it's using an
25:50
uh right here but it's using an adaptation right here that will be more
25:53
adaptation right here that will be more
25:53
adaptation right here that will be more professional sounding Bic on my input
25:55
professional sounding Bic on my input
25:55
professional sounding Bic on my input right here so using that prompt
25:58
right here so using that prompt
25:58
right here so using that prompt engineering uh I I'm using uh
26:01
engineering uh I I'm using uh
26:01
engineering uh I I'm using uh instructions I'm using context and then
26:03
instructions I'm using context and then
26:03
instructions I'm using context and then I'm expecting some kind of results I
26:06
I'm expecting some kind of results I
26:06
I'm expecting some kind of results I could tell I could give it examples if I
26:07
could tell I could give it examples if I
26:07
could tell I could give it examples if I wanted to and so on but this is just a
26:09
wanted to and so on but this is just a
26:09
wanted to and so on but this is just a way that you can use multimodal uh
26:12
way that you can use multimodal uh
26:12
way that you can use multimodal uh applications of AI models to generate
26:13
applications of AI models to generate
26:14
applications of AI models to generate context for your llms and then get uh
26:16
context for your llms and then get uh
26:16
context for your llms and then get uh data back from your llms so that's all
26:18
data back from your llms so that's all
26:19
data back from your llms so that's all my demos um
26:24
[Music]