Up next in 10
An Introduction to Terraform, Infrastructure as Code by John Martin || Lightup Conference
5K views
Nov 16, 2023
With ever increasing complexity in data platform and application solutions, it is becoming ever more important to take people out of the loop when it comes to system provisioning. Infrastructure as code is the way forward with Azure Resource Manager Templares, Google Deployment Manager, or AWS Cloud Formation. But what if you need to use multiple cloud vendors, or even build a hybrid environment? This is where Terraform from Hashicorp can step in, one solution that has a provider model that will interact with Azure, AWS, Google, and others. Meaning that you only need to learn one syntax. Add that to the automation potential and now we have something that can really help get us down the road to infratructure as code. This session will take an introductory look at how infrastructure can be defined as code and be shipped to standardise the deployment process and minimise the chance of mistakes creeping in when deployed by different memebers of Development or Operations. Conference Website: https://www.2020twenty.net/lightup #lightup #2020twenty
View Video Transcript
0:00
Thank you very much. Really appreciate it. Thanks very much for having me. So as I said
0:04
the session is going to be here about going to talk about introduction to Terraform, which is a
0:08
technology for managing resources for deployment on cloud. My name is John Martin. I'm a data
0:14
platform MVP, chartered IT professional. My pronouns are here and him. So my contact details
0:20
there are SQL diplomat on Twitter and LinkedIn's there as well if you can get in touch after the
0:26
session. So obviously for a great cause, appreciate any and all donations in support of the organization
0:35
It's a great conference, some great speakers and some fantastic content here and anything
0:41
that people give is much appreciated. Big thank you to sponsors obviously who helped
0:46
put this on for us. Feedback, so speaker feedback really important. If you've got anything for
0:52
me sort of some suggestions on how I can improve or things I'm doing well please let me know
0:56
and obviously event feedback is fantastic the response sorry the organizers really looking
1:00
forward to that so that they can make sure that they they put on the the great events again so
1:07
before we dive in and look at Terraform and understand exactly what Terraform is we need
1:11
to understand the concept of infrastructure as code and this is something that's really sort
1:16
of come to the fore in the last couple of years and as we've moved from a traditional operational
1:23
infrastructure on-premises that type of thing we started branching out we started using
1:27
either sort of cloud first technologies or we're starting to use sort of hybrid scenarios but even
1:32
then filtering back to on-premises with the advancements that you'll see in things like
1:36
VMware hyper-v and other platforms that you can deploy using code taking everything from either
1:43
running it manually what we used to do so we'd run an installer or we manually go through uis and
1:50
what have you to deploy resources then we move on to some scripting sort of tasks so a bit of
1:54
powershell here and there that type of thing and we would have scripts that may be quite complex
2:01
maybe we would have to pull everything down and remove things before we could redeploy those
2:06
scripts things like that moving over to an infrastructure as code mindset means that
2:10
that we're adopting some of the development techniques that exist from say .NET, Java
2:17
development, that type of thing, which if we bring those into the way that we manage infrastructure
2:23
so deploying servers or cloud resources or anything like that, it gives us a little bit
2:30
more versatility. And one of the key things that this gives us is traceability. There is always
2:36
going to be a case of who made what change, where and when. I know that when I'm troubleshooting
2:40
infrastructure myself, trying to understand okay well what's changed because that's one of the
2:44
things that might clue me in on exactly why I may be seeing a particular issue or particular behavior
2:51
that type of thing there. Now traceability obviously if we're committing everything as code
2:55
we're creating source control when I make a change I commit that I put a comment in it
3:01
then people know that I made a change to the way something may be defined why I made that change
3:06
what it's linked to maybe issues that type of thing and we end up with that history that makes
3:10
life a lot easier for people coming in there that also means that we can line it up with our
3:14
infrastructure with our application code for example okay so by again leveraging that source
3:20
control capabilities we end up with a lot more documentation it's it's i don't like the term
3:25
self-documenting but at the same time it gives us that documentation is there for us to use and
3:31
maybe put into another form. The other thing as well about infrastructure code it allows us to
3:36
sort of take encapsulation and really bring that through to to simplify our deployment scripts to
3:43
understand how we're going to manage our infrastructure. So back when I started using
3:48
Hyper-V and VMware and things like that I could end up writing very very long PowerShell scripts
3:53
either for PowerCLI or for native PowerShell to use with the Hyper-V clusters, System Center
3:59
the virtual machine manager, things like that. Trying to write them in such a way that I can mask a lot of that complexity
4:08
from some of my colleagues who may not necessarily have all of the experience
4:12
for running these things or fully understand the code. I basically want to be able to say
4:16
you've developed this piece of code which will deploy this infrastructure in a specific way, and all you need to do is feed it the variables
4:24
feed it the values that it needs, system names, IP addresses, firewall rules
4:29
things like that. If we can encapsulate all of that complexity and move it to the background
4:36
it allows people to focus on delivering results, delivering value. I want to deploy servers
4:41
It becomes something very, very quick and easy to do is I take that piece of code off the shelf
4:46
I put in some parameter values, and I've come to some system
4:50
Again, by doing that and masking and removing that complexity, it makes for a lot easier pathway
4:57
It also means that we can start looking at automating more effectively as well
5:01
Versioning, I mentioned when I spoke about traceability, that you've obviously got the source code where you've got
5:07
the commits and you got the messages and everything that goes with that
5:12
The other thing we can look at is really sort of linking about our infrastructure as code, as versions
5:18
So if you think of version one of your infrastructure, for example, we're deploying a brand new platform
5:23
We've got our application developers who develop some code, or we've bought applications off the shelf
5:28
And this is what our infrastructure needs to look like. You know, deploy effectively. That's version one
5:34
Six months, nine months down the line, we've got a new release. We've got a patch upgrade, things like that
5:37
We want to make some potential changes to our infrastructure. That's 1.1.2 potentially if it's a radical change, version two of our infrastructure
5:45
And then you've got the corresponding version of the application. So, you know, the version three of your application runs on version two of your
5:53
infrastructure, for example, so you've got that. that linkage so you can understand what the components are that go into all of it as well
5:59
and it means it's repeatable that's the key thing as well means that I can deploy the same chunk of
6:05
infrastructure repeatedly for those applications whether it's in development whether it's in
6:10
test whether it's in production whether I'm using a multi-tenant system where I've got many customers
6:15
and I want to stand up in exactly the same piece of infrastructure for the application for that
6:19
particular customer things like that so I can take that and I've got a lot more control over it
6:23
Then there's the idempotence. Now, this again comes down to historically, the scripts we'd have to write were
6:31
I want to deploy some resources, that's not too bad. But if I want to be able to rerun that script over and over and over
6:37
and not have it cause any damage or overwrite values or things like that
6:44
I then have to add a lot more complexity to my code
6:47
Essentially, I want to go in, I want to double-check, okay, is the value set to the current value
6:53
If so, right, leave it, move on. If it's not set to the current value
6:57
okay, do I need to update it? Can I update it? Do I have to destroy the resource and then do I have to deploy a new one
7:04
What are the interdependencies? All that sort of stuff there. Whereas if I can write my code in such a way that it's idempotent
7:11
I can rerun it over and over and over and it's not going to make, I can get the same thing out every single time
7:16
That becomes very important. Now, if we're doing all of this manually
7:19
It can be very, very complex. Like I say, when you build your own PowerShell modules
7:23
you build your own libraries of code that will do a lot of this But then there a lot of stuff that comes off the shelves The tools that are available to us in the cloud world right now the things that some of the vendors are bringing out it a lot easier
7:36
OK, so we start thinking about why we want to do this
7:42
We take that approach of infrastructure as code. What it means is we can get a greater level of consistency
7:47
Now, I've been in the industry long enough to remember when we were building things by hand
7:52
A lot of the time we'd give something to the junior DBA as part of their learning cycle
7:58
They would be going and deploying new servers and then have a build sheet and then they'd have an installer and they'd have various other things
8:03
They'd be setting everything manually. And even with those build lists, you'd end up with something coming out the other end that may not necessarily look exactly the same as what came before it, what comes after it
8:13
So by moving to an infrastructure as code perspective, it gives us increased consistency
8:19
It reduces the chance of error. people are fallible that's how I've learned over the years I've made mistakes and if we remove that
8:27
from our infrastructure deployments are the way that we're managing things that makes our lives
8:32
a lot easier and we can concentrate again on delivering value that's the key thing here
8:35
by removing and eliminating this complexity we're not taking anything away from the people we're
8:40
just letting them actually spend more time on delivering true value to the businesses and our
8:45
customers that we're running these systems for. That repeatability as well. So consistency leads
8:50
on to repeatability, which means I can repeatedly deploy the same configurations for production
8:55
and then feed it back into test. I can feed it back into development. So if I need to make changes
9:00
I don't necessarily have to have an environment there all of the time. So when you start thinking
9:04
about on-premises, you may provision large VMware farms or Hyper-V farms or what have you
9:10
And then we'll have them sat there sometimes up working, sometimes they won't
9:15
We may spin up an environment. It's got 20 different servers in it
9:19
Now, there'll be a reluctance to tear that down because to build deploy 20 servers might take some time
9:25
But again, moving to infrastructure as code, consistently, consistently and repeatedly redeploy that environment
9:33
Then we can look at saving resources. Now, you take that to the cloud. What that means is you're going to spend less money as well
9:39
so you go to Microsoft Azure, you go to Amazon AWS, you go to Google Cloud Platform
9:45
If you've not got resources deployed and running, you're not going to be spending money on them
9:49
So again, by moving that, you've got some real tangible benefits to the business that you work
9:53
for by saying, well, if we adopt this approach, we can actually save some money on the bottom line
9:57
by how much we're spending on these resources and we can be more reactive. So if I need five
10:03
environments, I can spin them up very quickly in a day as opposed to taking several days to do them
10:07
potentially. And that leads on to the automation piece. When we've got everything written down as
10:12
code in that sort of way we can easily then plummet into the likes of Jenkins, Azure DevOps
10:18
Bamboo, any of these build systems we can then start leveraging things like Ansible, Chef, Puppet
10:24
all of these platforms which will help us coordinate the execution of our code to spit out
10:31
an infrastructure it looks how we want it and when you start thinking about lining that up and
10:36
bringing in the application deployment pipelines and everything like that as well you can build
10:41
automated testing systems very very quickly and easily so deploy the infrastructure as you're
10:45
expecting it to that can get tested then it will deploy the applications again you'll get all the
10:51
tests running through there if you've got the test automation in place as well and then you get all
10:55
the reports at the end so you can be confident if you make a change to your infrastructure you
10:59
essentially say what if so if i take that version of my application and deploy it on this new version
11:03
my infrastructure does it come out the other end and look the right thing do i get the right results
11:08
do all the tests pass so again when you start thinking about wanting to be able to do things
11:13
more reliably by putting it into this sort of scenario it means that if you're in any industry
11:19
that maybe has some regulatory compliance around how you coordinate change or if you've got change
11:24
advisory boards and change control in place by getting everything lined up and scripted in such
11:30
a way it means that you could be confident when you raise that change and say well here's the test
11:33
cheap from all the work we've done to say but it works fine so now we can move forward and deploy
11:38
production so this session is about terraform but what other options exist at the moment as well
11:45
for us well this is where it depends on which platform you're using so if you're using microsoft
11:50
azure platform then obviously you've got azure resource manager um so that's a sort of json-esque
11:56
type scenario where you can build out and define your resources and your infrastructure and deploy
12:01
that to Azure. If you're on AWS, cloud formation, Google Cloud deployment manager and then there's good old PowerShell as
12:09
well if you want to wrap it up and do things yourself that way. So if you're doing some on-premises deployments to say
12:13
VMware, you've got PowerShell, you can use that PowerShell for Azure as well. So there's a lot of different things that you can
12:21
do now. That's great if you're in a business where you're only using one particular cloud vendor that might or technology vendor
12:29
then what that gives you is you don't have to look at that full range of things but if you're working
12:34
in the industry you're a consultant you're a contractor you work for an msp or something
12:39
like that you you potentially deal with multiple platforms multiple technology platforms
12:44
at which point having to learn and try and remember okay well do i use bash for this
12:48
do i use powershell for that or use armed or use cloud formation it can get a little bit more
12:52
complex and a little bit overwhelming and again it sort of introduces inefficiencies into the way
12:56
we operate and this is where terraform really comes into its own because terraform is a consistent
13:05
way of doing things through a single interface and what that means is that i can define my
13:12
infrastructure in a consistent syntax um so for for much larger for aws for google vmware i can
13:20
use the same syntax constructs to define my infrastructure the exact parameters and values
13:28
and the way that we do that around defining resources will be different on every platform
13:31
because what terraform does it actually interacts with the apis okay so you've got a provider for
13:39
the various different technologies in place we'll discuss what providers are in a little while
13:45
but what it means is that everything's got a similar structure similar sort of language that
13:50
that they're writing the definitions in but for example a virtual machine definition for Microsoft
13:58
Azure will not be the same as a virtual machine image definition for AWS but the structure and
14:05
the language and the concepts that we use to define those resources will be the same so it
14:11
dramatically reduces that sort of need to learn multiple platforms in a lot of
14:15
depth when it comes to the code used to deploy things as opposed to
14:24
just having one cloud agnostic deployment system that's not what this is for this is more about
14:29
giving you an interface to be able to work with all of these other technologies so what exactly is terraform how does it hang together how do you actually make use of this
14:39
okay the first thing to understand is terraform is a multi-platform command line tool and
14:43
anything else in case it runs on Linux, Unix, Mac OS, Windows. Just a little executable
14:48
little binary that you drop in and you can start calling. So when you drop it into your
14:53
I going to show you later on I run it here in Windows and I use VS Code as my interface for running everything Just stick the path to the executable in my path variable and then I can call Terraform from the
15:05
command line to do whatever I need to do. What that also means is that it's really useful for
15:09
plugging in like I said to Azure DevOps, to Jenkins, to Bamboo and things like that
15:13
as part of a deployment pipeline. Again it's just a command line tool so if you put on your build
15:18
servers things like that you can call it as you need to the language used as the syntax is hashicorpe
15:25
configuration language hashicorpe is the company that produces terraform one thing to note is that
15:29
terraform itself is free um there are some additional capabilities that hashicorpe have
15:34
around it which is paid for offerings but everything that i'm going to show you today can be done
15:39
so long as you've got an azure an aws type subscription the actual terraform stuff there's
15:45
no additional cost so I use VS code and I use Terraform I can define my infrastructure and it
15:49
doesn't cost me anything okay one of the really interesting things here as well that makes my
15:55
life a lot easier is the way that Terraform handles the interdependency between resources
16:00
that I'm defining so if back when I used to start off with Azure I was a big advocate and fan of
16:06
using PowerShell rather than anything else but I had to make sure that I defined my scripts and
16:11
and wrote my script in this particular order to make sure all the dependencies lined up correctly
16:16
so I need to make sure that I deploy the resource group then I deployed the various different
16:21
resources in the right order otherwise my scripts would fail and that took a lot of brain power
16:27
I'm inherently lazy and therefore that's why I'm a huge fan of automation Terraform I like because
16:32
it will figure that out for me I can define my infrastructure resources and I can reference so
16:39
I can either have implicit or explicit dependencies. We'll look at those in a demo later. But essentially what it means is that Terraform knows which order to deploy things in. What that
16:48
means is when it comes to do the deployments, it will do them either in parallel or serial
16:52
depending on those relationships. So if I'm deploying five VMs, I can do that in parallel
16:59
There'll be various components within those VM deployments that need to be run serially
17:03
but it means that you can end up with a much faster deployment type. I mean, how you've got to handle all that parallelism yourself it's all written into the platform for us
17:13
and it's the key thing here is about state management so i want to be able to run and then
17:19
rerun my um my deployments i want to be able to run a plan and what it does terraform gives me is
17:25
the ability to say okay well what if okay so i've got my existing state uh for the infrastructure
17:32
I've deployed if I bring my next one along and say right this is now what I want to deploy I'm
17:37
going to plan and it will actually give me that information back and say well I'm going to create
17:40
these I'm going to destroy these items and I'm going to update these items it all comes back in
17:45
textual format so it becomes very very easy for us to start planning okay is this going to cause an
17:50
outage do we need to take down time to do this deployment can we do it online because it's all
17:54
updates that type of thing so it's very very powerful tool now when we think about how Terraform
18:02
is is constructed and the various different components that we have. The first thing we need to understand is aside from the Terraform application itself
18:13
Is the providers now these providers are essentially the interconnect between the
18:20
Terraform platform and the target technology platform. So the Azure RM provider, for example
18:27
is the bit that sits between Terraform executable. My definitions of what
18:32
the infrastructure needs to look like and actually deploying it up to Azure because that's the bit
18:36
that will talk to the APIs okay same for AWS same for Google same for VMware same for
18:42
GitHub there's many many tens of different providers for various different technology
18:49
platforms that you may be using so you can go and have a look now if there isn't one there for
18:55
a technology you're using right now it's all open source so you can write your own if you don't if
19:02
you think something's missing for example from a provider you can fork it and you can add that
19:06
functionality yourself it's all written in go um so it's a relatively lightweight i mean obviously
19:13
the the learning curve there if you don't know if you don't know go something we need to learn um
19:18
so it's not i'm not gonna use the term easy but the the learning curve is there um and it's maybe
19:25
not as steep as something say written in c or c plus plus okay and once we've got the providers
19:31
we have the resources that we want to deploy. Now the providers will essentially define the
19:36
structure that we have for our resources. So in this instance this is where I want to deploy
19:41
resource group to Microsoft Azure. Then what are the key values that I need to give it? Location
19:48
name, tags, things like that. If I'm deploying an Azure SQL database, what do I need to deploy
19:53
my server, the name of it, the location, all of those things there. I'm deploying an AWS
20:02
machine image. Okay, what do I need to there? What parameters do I need to give it essentially
20:07
to allow me to have that thing come out the other side? And those are the resources that
20:11
we have. Now, obviously, that can be very prescriptive. If I've got a virtual machine
20:16
I may have to put 10, 15 different values in there. That's going to be an interesting
20:22
for us to look at them let's figure out how we can do that a little bit more efficiently later on now in addition to resources we've got data sources the job of data sources essentially is
20:31
if i've got x resources that are external to my project that i want to reference i can reference
20:36
them through a data source now terraform does give us the ability to do as a state management
20:41
and you can reference another state file but if the resources potentially were not deployed through
20:47
terraform then what it means you can essentially do a look up against that technology platform
20:52
to pull information in to allow you to reference that data. And those data sources are essentially a read-only version of some existing resources
21:01
So I can go and gather information about existing resource groups, existing virtual networks
21:06
So for example, if I'm running a hub and spoke architecture, I can go and look up the core sort of hub VNet, for example
21:14
and all the information that goes with that, so I can then plug into all the components that need to that
21:19
As I mentioned earlier, we've got resources where I need to define a lot of things
21:25
This is where variables come into their own. I can actually define variables and then I can put them into all of the resources and
21:34
such like that I'm using. So if I want to use a data source, I can define a variable with the name of the subscription
21:40
and the resource group and things like that that I want to look up. And then I can pass that in at runtime
21:45
I can set defaults. So again, this is where we start looking at taking that encapsulation concept and simplification
21:51
We can define all of this. We can put everything with variables, which means that all I need to do is then supply values for those variables
21:58
And I can either supply them in line at command line or I can provide a variables file
22:03
which essentially has a map. It's a key value pair mapping for this is my variable name
22:07
This is the value I want to give it. And then I just pass that into the command line. And again, I'll show you that shortly
22:15
Outputs. now this essentially is a mechanism for saying okay i want you to actually output some information
22:21
to the console screen it not a create file for example that where you need to wrap it in something like say bash to take the outputs that come to the console and directly somewhere if you using Azure DevOps Again there an option to automate deployments There are pipeline components there that will consume the output from the console and pipe that into something else that we need to do
22:44
But essentially I can say if I'm deploying some, for example, database or machine images, things like that, and I want to get the name or I want to get the IP address, I want to get the name or the configuration details for a particular endpoint
22:58
For example, if I'm going to SSH into something, I can have that come as an output after everything's run, output that to the screen, and then I've got that information
23:09
If I've got it output into the pipeline, then I can store it somewhere. For example, put it into one of the Azure Key Vault for secret management, that type of thing
23:18
So there's a lot we can do there about taking information that's generated when we do the build and the deployment and actually then do something with that output
23:28
But it doesn't create an output artifact or file. It's more it comes to the screen and then we've got to do something from ourselves
23:35
Now, again, this is where we're thinking about the complexity. We've got quite a lot of resources we may deploy as part of an infrastructure to break that down
23:44
There's various different things. But what we can do is we can wrap it all up in a module
23:49
And a module in Terraform terms is essentially a folder structure. Okay, so there's a wealth of them available for free online in the Terraform GitHub repositories
24:01
But additionally, if you want to build your own, for example, you can define a subsection of your infrastructure and you can modularize it, put it into the folders and nest it
24:11
And what that means is that you've got your own internal infrastructure component, which is something you just pull down off the shelf and plug it together with some other bits and pieces to give you your overall infrastructure
24:22
again you're encapsulating and masking a lot of that complexity that goes into deploying and
24:27
defining a particular infrastructure so it is as simple a case as defining all of your resources
24:34
the providers you need things like that putting it into a separate folder structure and then
24:38
referencing it in this particular way we're not going to cover that today because that's a little
24:42
bit more complex than the introduction um we have got a whole raft of demos that we're going to start
24:47
looking at in a moment about building up and understanding how we connect and go through from
24:52
there and all of these components together combined give us the terraform solution terraform platform
25:00
and again one of the things here is not like a view studio solution project where you've got
25:04
different project types it's essentially collection of files okay and we'll talk
25:09
through that when we jump into the file system at a moment so with that we'll go into our demo
25:17
So here I have Visual Studio Code. I've got this set up so that it will talk to my Azure subscription
25:28
Now there's a couple ways when we start thinking about setting up our Terraform folder structure
25:37
if you go over here you see I've got yeah I've got the demos that we're going to go through today
25:46
okay and they're all just different folder structures okay underneath the mark so if
25:51
we start thinking about um the first demo the first most important thing you understand is how
25:56
we're going to actually authenticate against what we're talking to okay now what I've got here
26:03
essentially is the um the most basic folder structure you're gonna look at for a terraform
26:09
deployment i've got the folder and i've got main.tf tf is the file extension of choice
26:22
so that doesn't know that essentially this is terraform files when i run terraform
26:26
what it will do i'll point it in the directory i'll look for all the tf files
26:30
will basically then merge all of those into its deployment and it will work out those dependencies
26:34
that we spoke about earlier now the main one is what's going to have our resource in it
26:39
and what we want to do is look at deploying a resource group i'm going to call it this and
26:49
i'm going to put uk south as your reason so how do we do this so this is where we've got a couple
26:58
of different options now from an interactive perspective when i'm doing development i can use
27:02
the the azure cli to authenticate so if i go inside login let's just see yeah
27:15
see if that's logged in yet yes there we go so i'm already logged in which is great
27:22
okay so what that means is now that if i start running terraform interactively
27:26
I am going to be able to connect to my subscription. Now, what does that mean
27:32
We've gone to providers. I'll just rename this to TF a second
27:38
So now I'm defining my provider here. I'm going to say, right, I want to use the Azure RM provider
27:45
Okay. If I've got a specific version that I want to use
27:49
then this is what I'm going to reference. Now, one thing you can basically say
27:53
pull the latest version. I would recommend against that when you start looking at doing anything with production infrastructure because you want that consistency
28:01
And as this is an open source, it's constantly evolving, new versions coming along may introduce something or change
28:07
introduce breaking change, which you don't necessarily want to have to go back and work out when you've got that coming straight in
28:13
So what I would look to do is say, well, version three of my infrastructure use this particular version of the providers that we're using here
28:19
And then I'm specifying which subscription I want it to do. I've got three or four ones that are available to me
28:26
So if you come back to this, we see main.tf. If I right click
28:34
and let's just open that in the terminal for a second, save me to type anything
28:40
If I type in terraform plan, what you should do is then go off and say
28:45
okay, well, what do I want to do? But before we can do that, we've actually got to initialize our directory, if you will
28:53
so terraform init and what this does is it goes off and it will just look and understand okay
28:58
what providers do i need i'll pull all that information down and you'll notice that i now
29:02
have a dot terraform file that's a folder and within that folder itself i've got the provider
29:10
information and the lock information things like that okay so now i'm in a position to go
29:16
Terraform plan. And what this will do is it will go off
29:24
It will have a quick look at things for me and say, OK, I want to see what's existing
29:29
at the moment versus what needs to be deployed. And we can see here that I want to create this set of resources
29:41
I'm going to create a resource group. The ID is going to be known after it's applied
29:46
Now the Terraform apply command essentially allows us to say, right, make it happen
29:51
Okay, it's gonna be in the UK South, that's what I'm passing in. The name is known because I'm passing that in
29:55
And again, tags, because I'm not passing in discrete value, it's gonna say, okay, well, I'll know
29:59
After we've done this, OK. Now, the other option, if we want to start with the automation
30:05
is use things like either managed entities or service principles. Now, one thing we can do is we can create a service principle, for example
30:15
So here I'm just going to use the Azure CLI to create a very basic service principle, which would be good
30:23
So what this means is that it will go and create one for me
30:27
and then it will spit out some information which is useful for me. So essentially what I've done is create a service principle said right it's a
30:37
contributor on my subscription here's the password tenant app id okay that because what we can do is if we look at the providers too this is where we can specify
30:55
additional information so i'd say which subscription which client password the tenant now ordinarily
31:02
i would put this in here for some development work if i'm looking at deploying this via
31:08
automation i would not put those values into this file um okay if i'm going to commit this
31:15
to source control i would put placeholders in here and then use something like azure devops
31:20
or Jenkins to do token replacement. So I have a common token value here and I would pull that
31:27
information from a secret vault management system. So Azure key vault or Terraform vault or hashcorp vault or something like that which means I can pull those secrets
31:35
out of somewhere secure and I can basically put them in a runtime and I haven't got to persist
31:40
that information somewhere where other people may or may not get access to it as a privileged scenario
31:46
okay so if we come back to this one here and if we go to
31:57
and then we go terra form apply much like when it does the plan it will go and say okay what's there at the moment what am i going
32:10
to do and what does that actually mean um how does that manifest itself okay so it's going to
32:16
going to create these and I can say yes now there is a command line value that I can put in here as
32:21
well and say just do it don't bother asking for this and that's something you would look to do
32:26
when you use an automation so what that means is I've now deployed my resource group to Microsoft
32:32
Azure so where do we go from there um state management this is an interesting one we've
32:38
got the terraform state file don't ever modify this um it's essentially a json file that describes
32:44
everything that you see here we've got the id value now for the subscription and the resource group
32:49
itself um we can see all the information there now that's all well and good but that's on my machine
32:55
um what if i'm working collaboratively in a team that type of thing this is where the ability to
33:00
store state remotely comes in and this is where you can use amazon s3 buckets you can use terraform
33:06
back end or as i'm going to use here you can actually create an azure storage account um which
33:12
will then hold our files so if I do these a second so it's going to run some commands in the azure cli
33:20
so I'm going to set the account I want to work in I'm going to create the resource group I want to
33:26
put this storage account into and then I'm going to create storage account
33:33
helps if I don't highlight just one word okay that's going to create me a storage account
33:39
and then what we're going to do is we're going to create a container within that storage account
33:43
which is going to hold our state files so just give that a moment
33:52
now why would i use remote state versus local state remote state means that if i'm working
33:57
in a collaborative environment when i'm applying something i can lock that state file which
34:00
prevents someone else from doing something so again if we're one single environment it makes life a little bit easier for us to do that so there we go
34:07
Okay, right then. So we've deployed our storage account. What it means is we need to come in and basically define the back-end storage. There's also a couple
34:20
of files we need to just drop in the file system which I've already done and there'll be more
34:23
information in the slide decks that I'll publish for everyone about just all those pieces you need
34:28
to do. Essentially what we're doing is saying this is where we're going to store the information
34:32
this is the resource group storage account container name, we want to do it, and this is
34:36
what we're going to call the state files. It's demo2. Okay. Much like before
34:41
we've got a resource group that we can do. There's our provider
34:45
So if I come down here again, come back to that one
34:53
Come to there. Initialize it. And he sees initializing the backend for us
35:06
okay so it's created the piece it needs to downloaded the various providers and then if i go
35:14
terraform apply what this means it'll do is gets to come through and say yes okay so we're just going to deploy another resource group this
35:30
time it's going to have demo two in the name so we're going to look at my azure subscription
35:34
in a moment we'll see more of them coming in so if i come back to my resource groups here
35:43
so there's demo one which we created earlier sometimes the portal can take a little while to
35:50
come up to speed but here the back end one here the storage account if i come into the containers We can see there the light up and then there the Terraform state file see all the information in there and I can actually hit edit and there we go so that
36:09
our state file stored centrally for demo 2 so if I'm working again with my team then I can take a
36:15
lock on that if I need to to prevent them from doing anything whilst I'm doing the development work again it also means if I'm using multiple development machines that type of thing
36:24
I'm in a much better position to really just sort of make sure I can make sure my product
36:31
is as high as it's going to be. So let's see if demo2 resource group has shown up
36:40
Which is nice. So on to demo3. Okay so So variables. So you've seen so far in these ones here. Just close those down and look
36:58
in the main.tf. You can see here that I have specified the name and location in the file
37:08
itself. But that means that obviously if I do change this, I've got to come in and change
37:12
every file where variables really come into their own. So there's providers and you can see the
37:18
syntax we're looking at here as I'm saying variable called Azure subscription ID. Okay
37:26
is going to be there. So if I come to this one here. Right so within this one here I define and declare my variable. I can define the type
37:40
I can set a description. I can even set a default value. OK, so the description, if you're doing it at the command line level, will basically prompt you
37:48
So try to put a descriptive thing in there about what it is that you're going to be asking for an input for
37:55
And then when I look at main.that, there's a couple of different options we have available to us now
38:01
This option here is referred to as the interpolation syntax and is deprecated
38:06
Moving forward, we should be looking at using syntax here for location, but just for completeness
38:11
I thought I'd include both of them for you. So if we do this one, three variables
38:23
Now Terraform init, initialize everything. OK. So if I go terraform
38:37
It's just to apply. OK. So asking me for the values I want to put in
38:44
Now this could be quite hard just forgot a large. UM. So if I just exit out of that
38:52
one of the options we got is then to use what's referred to as a TFR files
38:57
This is a file which is going to contain the values we need for our variables
39:07
If I do terraform apply var file equals What I'll do is it means I shouldn't get prompted for anything because the variable
39:34
file will provide all of that information
39:44
So when we go, this is what we're going to do. You can see the interpolation syntax is deprecated
39:50
It's going to add all of this information in. You can see that yes
39:54
Then I should go off and deploy our third resource group. There we go. So nice and easy
40:04
This way I can pass all that information in. Now, when we start to use Cloud resources
40:10
one of the key things that you'll look at is, At least in my experience with Azure
40:15
you want to set up tagging policies. So I want to tag resources so I've got
40:19
more metadata around what they're for. So for example, I may want to put cost centers in there
40:24
If I'm going to be rebuilding to other departments, is it production? Is it development
40:29
Is it something that's going to be automatically cleaned up? So you can see here again
40:34
starting to really build out now the folder structure, more files for Terraform
40:40
So again, main providers come down into tags. Now I tend to break tags out into their own file
40:50
just for ease of maintenance. But essentially what I can do here is I'm using
40:54
a different type of variable called the locals, which allows me to build essentially these hash tables and such like
41:00
So common tags and I can define them all here in one place
41:04
So environment, person to contact for it, cost center is what we're using it for when was it created this is um at least if you're going to use
41:14
much of azure this is a huge one to be looking at putting in as a tag because create that it's not
41:19
actually a property you'll find on pretty much any resource that you create so i tend to create a tag
41:24
with that so i know when something was created so if something magically appears i've got some
41:28
information you create policies that will refuse to allow resources to be deployed unless you do
41:34
that now what that means is when i look at main where i've got my resource group deployed
41:41
i'm using this merge essentially so i'm saying right merge all of the local.com and tags together
42:46
So if I don't put it into the variable file then I'm going to be prompted for it
42:53
One thing you'll notice here is you can see the tags are all called out
43:02
There we go. So that's all created. So if I come out to my resource groups, let's see if we've got demo 4 there yet
43:15
Not quite. OK, so what waiting for that one? If I come back to tags, for example
43:20
And then if I wanted to say. Conference equals. Just put this one in hard code it
43:34
Light up for example, save that. If I then come back to here and
43:39
run another Terraform apply. What we'll see is, once it goes off and does the ysis for us, that what's there at
43:59
the moment and what it needs to do. You can see here it's an update in place and it wants to add a conference tag
44:12
So if I've got something that's built up with 20, 30, 40 different resources, all of which
44:16
are tagged, if by using this sort of construct for using the locals with essentially the
44:21
hash table of values and then using the merge, it means that I only need to go to one file
44:27
and make that update. I can make that update globally. So if I now come down to here, let's see if this is here yet, there it is
44:36
You'll notice all of the tags in here. Okay. Okay. If I then come back to my Terraform and say yes
44:47
That should now go and make the update for us. Okay, so that's been one changed
44:58
So we come back to here. Refresh. Take a couple of moments to come through
45:08
Oh, my nose, you're fakey, fakey. So, let's just..
45:19
Let's see if portals don't pick it up yet. There we go. We've got the extra one on there
45:26
Usage, environment, but there's the conference one, which we just added. So again, taking that infrastructure as code perspective
45:33
we put it all in one place, we reuse that as effectively as we can elsewhere
46:07
Okay. Okay. The keyboard seems to have stopped working. Okay. Right
46:25
I'm having a bit of an issue here with my keyboard, you can see
46:29
It's better. There we go, right? Let's get dependencies going. This will be the last one I can get
47:18
done given the time we've got. But essentially this is where I can create multiple
47:22
Different report various different resources. It's in a resource group. I can create this Azure SQL Server and I'm going to add an Azure SQL database on top of that
47:37
And if I come down to this one here. So, the demo 5 is dependencies. One thing you'll notice is that I'm creating and
47:51
the resource group name I actually referencing in the firewall rule for example I actually referencing the resource group name for another resource and that where I start building up that dependency between them
48:08
Okay so if I go Terraform init pull that information down for us now this is one of the interesting things we can do if we go
48:21
uh terraform graph what it will do is actually pull out a digraph for us
48:41
and what this means is i can essentially come in and say okay what does it look like and I can feed that into something like graph viz for example or whatever
48:55
uh so if I apply now so I can take that I can apply and then that will generate a um
49:07
essentially a graph for me So what we should do is all goes to plan. You can see it's just about an awful lot of
49:48
things that we need to create. All the various different bits and bobs. If I just go yes
49:56
That one I go off and create all of that for us
50:27
environment you've got for your Linux, your Windows, your Unix, your Windows. It doesn't
50:33
matter. The tool there for you, like I say, if you run different build servers. If you
50:39
want to use automation effectively with it, Terraform itself doesn't do automation. It
50:44
facilitates automation. And this is where you want to use something like Ansible Chef
50:47
Puppet or whatever. But then also you can combine it with things like PowerShell, DSC
50:52
and also other elements of those technologies to actually control configuration inside those
50:57
resources so for example if you're deploying windows with most of virtual machines you're
51:02
deploying vmo virtual machines you're deploying linux boxes windows boxes things like that
51:07
that's where you can use Azure DSC or Ansible such like to actually use Terraform to deploy
51:12
them and then you'll configure them internally using another technology so this is all about
51:16
resource management not resource configuration automation like I say Azure DevOps GitHub
51:22
jenkins bamboo any of those will work on your build servers just make sure you've got the appropriate
51:27
hooks in there for the terraform language and if you've got azure devops for example that has got
51:33
a huge amount of built-in stuff that you can just pull into your pipelines and interact with terraform
51:39
as i mentioned earlier doctors development folks sort of focuses for um how we manage our code
51:46
version your infrastructure version your application code deploy them together as units of work
51:50
deployment pipelines and it just really makes our lives a lot easier when it comes to the
51:56
governance the management of infrastructure and allowing us to deliver value very rapidly
52:04
but without having to do a lot of the work that takes a lot of time but doesn't realize much value
52:11
okay so deploying servers it's great but the value add is the work that you can do around deploying
52:17
capability onto those systems, those applications you build, the database and systems performance
52:22
tuning that you do, providing guidance and governance and everything back to the business
52:27
those are your real value adds performance tuning stuff like that not building spitting
52:30
servers out. Okay that's why we've got infrastructure. So from there have we got any questions
52:37
That's all. We don't have any questions in the forum, John. So for all the attendees, if you have any questions
52:54
that you want to get clarified from John, this is the time to post it in the Q&A
52:59
Yep, feel free or if you think something afterwards, my Twitter is SQL Diplomat
53:06
so please feel free to reach out to me. If there's nothing there
53:13
then I'll look to hand over to the next presenter so they can get ready and get ready to deliver their session
53:20
Sure. I just want to say thanks once again for having me. Really appreciate it
53:25
And I'll quickly pop the QR codes back on there for feedback
53:33
So if you want feedback around my session, please um please feel free to to provide that and like say the event coordinator should value that
53:40
feedback as well um they're doing a great job and um it's good to know what they're doing well
53:47
so with that i'll um i'll look to hand over sure thanks john thanks for the session and that's okay
#Business & Industrial
#Distributed & Cloud Computing
#Enterprise Technology
#Networking
#Programming