Up next in 10
Keynote : SQL Server Virtual Conference by Asad Khan
6K views
Oct 30, 2023
Asad Khan is the Director of Program Management in Azure Data engineering group. Currently he lead the product and program management of SQL Server and the SQL based PaaS services in Azure. This includes SQL Server on-premises business, SQL Server on Linux, Azure SQL DB, Azure SQL MI, as well as the SQL hosting business in other Clouds. In the past he used to run the Big Data Services in Azure, which includes Azure Databricks and the HDInsight service, which are the open source analytics offerings on Azure. He has spent the last few years working on Data technologies and building Cloud Services. He enjoys working with large enterprises to modernize their data pipeline and enable intelligent applications. He is passionate about building new businesses and leading organizations to build great products that impact people and organizations. He has a master’s degree from Stanford University. Conference Website: https://www.2020twenty.net/sql-server... C# Corner - Community of Software and Data Developers https://www.c-sharpcorner.com #SQL #newsqltools #vision #conference #sqlserver #azure
View Video Transcript
0:00
Thank you all for joining me today
0:03
My name is Asad Khan and I work in Microsoft Azure Data
0:07
and I'm responsible for running SQL Server and all the SQL services in Azure
0:13
I'm super excited to be here, though I missed the opportunity to have this talk in person
0:20
I'm happy to join you online and share the work my team has done and where we are heading
0:28
But before we get into that, I want to first acknowledge how difficult this year has been
0:35
In the best of cases, these have been trying months for all of us
0:42
And in the worst, they have been devastating to some. There is no way to frame the hardship that makes them easier to bear
0:52
There is only hope and strong communities like you to help us through
1:01
In this day and age, the ultimate tool is data. Over the past 12 months, we have seen organizations undergo digital transformation at breakneck speeds
1:15
racing forward to achieve two years of transformation in just months. While our global situation may have pushed digital transformation to the forefront
1:29
it is community that has made change at the scale and pace we have seen possible
1:37
We have come a long way together since we started SQL Server over 30 years ago
1:45
Today, 98 of the top 100 Fortune companies run SQL Server. While many organizations still run SQL Server at the heart of their business
1:57
they also adopted new application scenarios and new type of databases, to which I am proud to say that many of the Azure data services are built on the innovation
2:10
that we continue to do as part of our SQL strategy. At the heart of this strategy, we are extending SQL from edge to the cloud, putting Microsoft SQL engine, your T-SQL skills, and your favorite tools at the heart of where innovation is happening
2:33
We equip you to reimagine the art of what's possible. For example, imagine being able to run your SQL apps
2:43
with AI capabilities right at the edge of an ARM device for real-time updates
2:50
or using SQL to run ytics over your structured and unstructured data stored in HDFS
2:59
How much time could you save if you could scale horizontally in the cloud
3:05
to accommodate your most elastic workloads or bring the cloud security and scaling capabilities to your on-prem deployments
3:15
You can now do all of that and more using SQL Anywhere
3:22
For the rest of my talk, I would like to focus on the latest enhancements
3:27
we have made to these five SQL-based offerings. and more importantly, show how these enhancements are helping companies in their journey of digital transformation
3:41
Azure SQL Edge is our new addition to the family of SQL-based products
3:48
It offers the full power of SQL Server at the edge, along with built-in capabilities for streaming, time series, and AI
3:58
Let's look at how Azure SQL Edge transformed a 3M manufacturing plant, enabling 3M to fix problems before they occurred
4:10
The 3M brand is a familiar household name. It's unusual to find a work desk that doesn't include some of its product, such as scotch tape and post-it notes
4:22
3M produces more than 60,000 products in various categories, including the N95
4:30
which are instrumental in our fight against COVID. My team partnered with one of the U.S. manufacturing plants
4:37
and the 3M Corporate Research Lab to improve predictive anomalies. The plant's network connectivity was limited
4:47
Therefore, it was difficult to gather and transfer and use that data for predictive maintenance
4:54
The team decided to move all the data processing and AI at the edge using Azure SQL Edge
5:03
Imagine the full power of SQL Server with added AI and time series capabilities
5:09
running on tiny edge devices. Azure SQL Edge also resolved the plant's problem
5:15
of limited network capacity. since most data processing and AI was happening
5:22
where the data was getting generated. Today this predictive maintenance lets 3M fix problems before they occur The quote from Mike their technology manager captures it nicely Azure SQL Edge is a great bridge between
5:41
traditional processes and the newest AI and compute features in the Cloud
5:47
This is the type of solution that will drive us towards Industry 4.0
5:55
I'm sure you are very familiar with SQL Server and how we continue to advance this data platform
6:03
In SQL 2019, we enabled data virtualization, intelligent performance, memory-optimized TempDB, and much more
6:12
Big Data Clusters in SQL 2019 was the first of its kind that removed the chasm that exists between operational stores and ytics platform
6:26
It's amazing to see how customers are advancing and modernizing their data environments by taking advantage of these new capabilities
6:37
Like this example with Etao Unibanco. A Tau Unibanco is the largest bank in Brazil and 10th largest bank in the world
6:46
operating in finance and insurance industries. Their concern was that their infrastructure would not keep up with their growth
6:56
and this would affect their SLAs to the business. I'm sure this is something we can all relate to
7:04
A good example is when they had to roll back an operation, it took a long time
7:10
On average, it took four to five hours to do a rollback
7:15
and sometimes that recovery could take as long as 40 hours. Atau was an early adopter of SQL 2019
7:24
to take full advantage of the new intelligent query processing and accelerated database recovery
7:33
With the accelerated database recovery feature in SQL 2019, the rollback times dropped to matter of seconds
7:41
and the intelligent query processing made over 90% of their queries run faster
7:48
The most interesting part was that it required almost zero core changes
7:56
Azure Arc is the cutting edge of technology innovation. It brings the best of SQL, Kubernetes, and Azure
8:05
to run a fully managed cloud data services in customers' data center
8:12
on top of their choice of hardware and operating system. Let's see how KPMG is using ARK to have consistent cloud capabilities
8:23
in their data centers and in their multi-cloud environment. KPMG, a global network of member of firms providing professional services
8:34
with 220,000 employees is always looking to stay on the cutting edge of technology
8:41
In Tokyo, a KPMG development team called Ignition is building out the next generation cloud-native
8:49
application platform to serve their clients. It's called Cloud Next. While wanting to harness
8:56
all the latest cloud innovation, KPMG's clients often require running their application
9:04
and storing their data on-premises or multiple public clouds in order to comply with government regulations and company policy
9:14
Azure Arc extends Azure management capabilities and data platform services to any infrastructure
9:23
on-premises or any public cloud, enabling the KPMG team to build application once
9:30
and run them anywhere and manage them centrally from Azure. The built-in automated management services
9:39
like self-service provision, elastic scalability, backup, restore, automatic updates of their Azure SQL managed instance databases
9:49
saves them time and ensures they are always using the latest, most secure database platform
9:57
Azure SQL is the largest PaaS service we have in Azure. Azure SQL is not just hosting SQL Server in the cloud
10:08
rather, we have built it from the ground up. Especially in SQL Hyperscale
10:14
we took apart the core of SQL Server and we run them as separate services
10:21
As a result, Hyperscale gives customer the performance of SQL Server with unlimited horizontal scaling
10:30
Hi-Res Studio is a leading video game developer. More than 70 million people worldwide
10:36
play their games on PCs, mobile and console. Some of their popular games are Global Agenda
10:42
Smite, Plyden, Realm Royale, and Rogue Company. Rogue Company, also known as Roku, is their latest release
10:51
and they used Azure SQL Hyperscale for it. When gaming companies launch a game
10:57
they don't know how many players they will get. It can start at five players and within days it can get to 5 million Hi was looking for a database that gives them full T surface area with extremely low read latency and most importantly unlimited scale
11:21
They took a big bet on SQL Hyperscale, our flagship cloud-architected relational database
11:28
As a result, they were able to scale the same database from few gigabytes to hundreds of terabytes within minutes
11:41
Azure Synapse is the first and only ytic system to have run all TPCH queries at petabyte scale
11:50
This was made possible because Azure Synapse uses the same query processing engine that ships in all flavors of SQL
12:00
Many customers have selected Synapse as their modern cloud data warehouse platform, due in part to the completeness of data security
12:09
And GrabTexi is just one such example. GRAB are rapidly expanding into new regions, and in order to adhere with local data compliance restrictions, they required fine-grained data access control
12:24
The native row and column security in Synapse, combined with dynamic data masking, enabled Grab to maintain a scalable, centralized repository of data that was secured appropriately for users in different APAC countries
12:43
By implementing security at the data table layer itself, all reports and dashboards are completely portable across entire company and don't require refactoring for different secure views per team, which had been their previous approach
13:02
This resulted in greater agility and productivity for data developers to build new capabilities
13:09
instead of maintaining ongoing cost of refactoring reports for new markets. The reason for the SQL continued success is because we are not afraid of innovating
13:24
both inside and outside the relational engine. Although SQL is a mature product
13:30
but we never constrained ourselves by what already exists. We are willing to break the bounds of existing systems
13:40
and start all over again. These five investment areas will give you a sense
13:47
of where we are heading with SQL. In the next few years, we will totally blur the lines
13:54
between cloud and on-prem. Arc has already delivered on part of that vision
14:02
but we want to go much further. As a customer, you should not worry
14:08
where you will deploy your database. The product, business model, and data manageability capabilities
14:15
should all remain the same. Today, in Azure SQL, all production databases are protected
14:24
using advanced threat protection. which can detect anomalies and raise alerts indicating unusual activity
14:33
But why restrict these advanced features to cloud-only or Azure-only? Why not protect every SQL database in the world using Azure Security Center
14:44
Or have Purview provide a single pane of glass across all deployments of SQL, cloud or on-prem
14:51
This will allow you to centrally govern and share data with full data lineage
14:58
Whether it's about Azure Edit Services or the past nature of SQL services in Azure
15:04
for customers who cannot move to public Azure, we will bring Azure to you
15:10
Today, in Azure SQL, we run millions of customers' databases, processing over 10 trillion queries per day
15:19
But there is more work needed to fully empower the modern developer
15:24
We are improving Azure SQL experience with modern frameworks and languages, such as Django, Node.js, Angular, Spring.Data
15:33
We will enhance the JSON and graph support in SQL Engine. We are also increasing our investment in intelligent data monitoring and insights
15:42
using the power of AI. and we are doing much deeper integration
15:47
with services like Azure Functions, Azure Web Apps, and the Power Platform
15:53
Hyperscale gives the unlimited scale through its page server design, and this helps us deliver the best latency
16:01
without restricting to the machine size. Serverless makes the underlying infrastructure adapt to your workload
16:09
and Elastic Pools are a simple, cost-effective solution for managing and scaling multiple databases
16:18
that have varying and unpredictable usage demands. Elastic Pool is the number one reason
16:24
most ISVs and SaaS solution providers pick Azure SQL today. In Azure SQL, we intend to bring the best of hyperscale
16:34
serverless and Elastic Pool into a single product offering SQL has always been the leader on security In order to maintain that leadership position we are continuously investing and innovating in the space of confidential
16:52
computing. For example, last week, we announced the journal availability of hardware enclaves in
16:58
Azure. A secure enclave is a protected region within the SQL database engine process. It
17:07
It acts as a trusted execution environment for processing sensitive data inside the database engine
17:14
always encrypted with secure enclaves, takes confidential computing to the whole new level
17:22
Azure Purview, which our CEO, Satya, announced in December, is another example where we are empowering data officers
17:31
with advanced data security and data governance. Blockchain had a huge promise
17:37
but the technology never took off because it is too hard to use
17:42
and doesn't integrate well with existing tools and framework. What if SQL can bring the same power
17:51
of making data fully immutable in untrusted environments, but you don't have to give up on the ease of programmability
18:00
The last thing I want to say is SQL Server is very dear to us
18:09
We will never slow down when it comes to the innovation in SQL Server
18:14
In the coming versions, we will continue to invest in the core features of performance, reliability, scalability, and bringing industry-leading innovation
18:27
One such example of industry-leading innovation is the project where we are rewriting
18:32
parts of the core SQL engine to remove the static variables in the SQL code
18:37
with machine learning models that will be able to adapt to your workloads at runtime
18:45
I think it will be transformational. It will change how customers use data platforms
18:52
Let me pause here and show you a short inspirational video that we recently put together
18:59
Imagine a tiny particle drifting at the edge of the Milky Way
19:04
A single element that could solve the next big challenge. Imagine you had the power
19:09
to reach across the stars and grab it. Our world is immersed
19:14
in a galaxy of data. Data that connects us, cures us, enables us to do more
19:20
Data that helps us innovate wherever imagination takes us. In 1969, Apollo 11's guidance computer was limited to 72 kilobytes as it traveled to the moon
19:31
Now, we have exponentially more power in the palm of our hands. This precious resource demands a trusted partner to harness and make every bit of it intuitively accessible at the speed of our ideas
19:44
The speed of SQL Server and Azure SQL. The latest evolution of data management's highly secure, supercharged platform
19:52
enabling you to break performance barriers and boost agility. Azure SQL Database's hyperscale capabilities allow you to autoscale dynamically
20:01
Advanced data security keeps hazards at bay with a single click, while built-in cloud intelligence eliminates worry
20:08
Machine learning services enhance the value of your data by importing adjacent code to create models faster than ever
20:14
And with cross-platform solutions like Azure Arc, SQL Server and Azure SQL products can be leveraged virtually everywhere
20:22
Our team is motivated to consistently improve this new breed of database
20:26
without disrupting the application of our existing technology. As many of the world's critical systems and most influential corporations deploy it
20:34
and agencies on our most important front lines depend on it. Businesses like Balzano, that leverage SQL Server big data clusters
20:41
to help radiologists interpret medical images. Or Zeiss, that use Azure SQL Edge to constantly improve the manufacturing of optical lenses
20:50
and organizations that help us dream bigger, like the Sloan Digital Sky Survey
20:55
that built its ever-expanding database of 200 million galaxies and 260 million stars on SQL Server
21:03
Imagine what your data could dream up. Imagine unparalleled access to that next great innovation
21:10
Together, we can reach it at the speed of SQL Server in Azure SQL. SQL
21:16
SQL started as a relational engine. Over the years, we have transformed it into a full-blown data platform
21:27
As you can see, many of the innovations are not restricted to SQL Server
21:32
We have taken them much beyond SQL Server. Today, SQL Server power some of the largest services in Azure, such as Azure SQL and Synapse
21:44
and they are also part of some key initiatives we are taking in the data space
21:49
such as Arc, Azure Purview and SQL Edge. Even with the best technical innovation
21:56
no product can be successful without a strong community. Our philosophy has always been community-centric
22:05
Please keep your feedback and correspondence coming. We love hearing from you, and we are always looking for new ways to support you
22:14
Thank you
#Business Operations
#Data Management
#Distributed & Cloud Computing
#Engineering & Technology
#Enterprise Technology