Confluent Platform 5.3 | What’s New in This Release

Confluent Platform 5.3 | What’s New in This Release


– Hi, I’m Tim Berglund with Confluent. I’m here in front of this stream to tell you all about what’s new with Confluent Platform 5.3. (country music) The new stuff at 5.3 breaks
down into three categories. Number one we’ve added some features to help your deployments be both more agile, that’s small ‘a’ agile like moving around fast not like stand ups and post it notes,
and we’re to be more agile and also more cloud-native. Second we’re giving you more tools to understand and manage
your event streams, to really get some insight about the data that’s inside your cluster. Third, we’ve got some exciting new security features that help you manage deployments that involve a large number of people with complex permissions. Let me break these things
down one at a time. To remind you Confluent Platform is the event streaming platform built by the original
creators of Apache Kafka. It’s built on that solid foundation of Kafka as that core
event streaming platform. It’s got development and stream processing tools layered on top of that and of course a lot of good things to help you deploy and manage and secure Confluent Platform at scale. All of this can run on
machines in your data center or machines in the public cloud and of course you’ve always got options in the managed service Confluent Cloud. Confluent Platform is always built with the latest release of Apache Kafka in that case this is Kafka 2.3. I’m not going to tell you too much about those features in particular because we already have another video that breaks those down pretty well. There’s also a great
blog post by Colin McCabe that describes those things in print if that’s what you prefer. Now let’s talk more about these agile cloud-native deployments. 33% of Confluent customers plan soon to be running Confluent
Platform on Kubernetes. There’s no denying that Kubernetes is a very important trend
and I think for good reasons. Kubernetes really was originally created to help manage work loads like lots of stateless microservices, we have all these
containers we need some way to orchestrate them, but once people started doing that they found that their stateful workloads, their databases and their event streaming platforms like Confluent Platform
made a lot of sense to run on Kubernetes also. There’s kind of an economy of scale in getting all of your work into a Kubernetes cluster once you dip your toe into that water a little bit, but to make a stateful work load like Confluent Platform work well on Kubernetes you need a
thing called an operator. Now an operator is a Kubernetes component that helps take the configuration that you give it and turn that into real life inside the cluster and to do that with something stateful like Kakfa and the other components of Confluent Platform
takes a little bit of work. You need a custom operator to do that that’s what the Confluent
Operator is all about. This lets you deploy Confluent Platform in a cloud-native way to Kubernetes and that’s any of the components of the platform: brokers,
your ZooKeeper cluster, Kafka Connect, KSQL, Schema Registry, Control Center, Replicator,
all of that stuff and it doesn’t matter
where you’re deploying it there are a lot of places
you can run Kubernetes. There’s Pivotal Container Service. There’s Google Kubernetes Engine. There’s your own Kubernetes cluster that you might manage on hardware in your own data center. It doesn’t matter any of those is a place where Confluent
Operator can help you. The key life cycle operations
like scaling things, you know we’ve always said
we have elastic scalability in Kafka and Connect and
KSOL and all these things. Well now that’s a configuration change in Kubernetes and Confluent
Operator makes those happen. Also rolling upgrades,
you don’t want to go back to the bad old days of hey it works we never have to version this again, with cloud-native deployments we expect things to be kept
up to the current version and with Confluent Operator you get support for rolling upgrades. Now if you have engineering resources and you know Kubernetes
well this is a thing you could build yourself,
but what we’ve done is to take our growing body of experience operating Confluent Platform in the cloud through Confluent Cloud
our own hosted service and we’ve literally encoded that in the code of Confluent Operator. So this is a way of drawing on the most extensive body
of Confluent Platform experience in the world running a Kubernetes and making that work for you. Now what if you don’t run Kubernetes and you’re deploying the components of Confluent Platform on servers again either on Prim or hosted servers in the cloud, but you’re managing each one of those yourself. Well for some time we’ve
had on a trial basis Ansible playbooks available, those who use Ansible to deploy your configurations in an automated way into those servers that you run. In Confluent Platform 5.3 these are now production ready and fully supported. And we didn’t just flip a switch saying okay they’re ready you can use them we’ll support them. There’s actually some
new security features as you see here we have proper certificate authority based TLS and Kerberos support baked in if that’s a cake you’ve ever tried to bake yourself you know it’s nice to have another chef helping you there and again these cover all the
components of the platform: ZooKeeper, Kafka itself, Connect, KSQL, Schema Registry, Control
Center whatever it is they’re supported by the new production ready Ansible playbooks. Now let’s talk about
features that help you understand and manage your event streams. Since about Confluent Platform 5.0, you may have noticed that Control Center has been growing some new features that are helping it dip its toe into the waters of being
a development tool. Now Control Center was conceived as an ops tool to help
you manage and monitor your Confluent Platform cluster, but increasingly its getting
some dev support also, this is a great thing, it’s grown features like topic inspector,
schema registry support, the KSQL UI and so forth. So continuing this trend we’re giving you an extensively redesigned
Control Center interface. All kinds of new things to help you visualize the data in your cluster better. It’s also normalized with the interface that we see in Confluent Cloud, so whether you’re managing your own deployment of Confluent Platform or your using Confluent Cloud those tools are gonna look more and more the same. Which is a great thing for switching back and forth between
those two platforms. This is a part of our plan of
giving you freedom of choice to be able to run Confluent
Platform wherever you want. Fully managed in Confluent Cloud completely on Prim in actual metal servers that you rack yourself
or in managed servers in the cloud wherever you do that we want it to look the same and feel the same so all those skills transfer across your teams. And here’s that new Control Center UI, the home page now gives you this card view of available clusters this lets you manage multiple Confluent Platform clusters from one instance of Control Center. Right here we see just one cluster if you have a large number of clusters you can search by name or hide clusters that are healthy and might not need your attention right now. When I click on our one cluster we get the new overview screen. We have reworked this to get you a quick view of all Kafka metrics, total number of brokers,
cluster controllers and whether ZooKeeper is healthy. We also have an overview
of the number of partitions and how many are
under-replicated or out of sync. Next let’s look at the new view of topics. The topic inspector gives us a summary of the data in the topic. There’s a table view and what we call the card view of the messages where each message is displayed
as a discrete card. You can navigate through the topic by scrolling to see the older
messages and newer messages. We can also inspect the
schema of this topic this is retrieved from the
Confluent schema registry and we have improved KSQL integration. Here we’re registering a topic as a KSQL stream so we’ll be able to preform queries against it. We select what we want
the key for this stream to be, what field contains the event time and we indicate that we want the stream to be encoded as avro-data. The rest we leave with the defaults. With that stream created we can run a continuous query on it, that’s what this select is to view the data in that stream as it arrives. You can see we’re getting new messages and those messages are available in a tabular view and card view just like in the topic inspector. Finally here’s the new
view of consumer groups. We can drill down into one group and see information about the
consumer lag for that group. The most recently committed offset is clearly indicated and the display gives you a sense of where the offset is relative to the newest
message in the topic. For about the past two
years before this release Confluent has had a
CLI, the Confluent CLI. This has been a development tool it’s been super handy for
me it’s a nice little way to spin up a local environment if I don’t want to use Docker at the moment I can just have this local environment do things with it, bring it up, bring it down, blow away all the data, bring up a new one, great for development and demos and all that kinda thing. Confluent CLI though
in 5.3 has just gotten a major upgrade, it’s now
a production ready tool and all the things that
you might have gotten used to in the previous CLI those things are still supported in the new one plus new features particularly role based access control, which I’ll talk about
more in just a minute. The Confluent CLI is your gateway to that functionality and you should keep an eye on this it’s likely to grow a lot more features in the near future. Now finally let’s talk
about granular secure access to your Confluent Platform cluster. Imagine you have some number of people developers, roles, you know principles something like that accessing your Confluent Platform cluster. Each of them is gonna
have different levels of access like some developers might need to be able to
read and write these topics. Some of them might be able to deploy new connectors to the
development connect cluster. Then you’ve got your administrators and they’re the only ones able to deploy say new connectors to the connect cluster or new queries to the
KSQL clustering production or something like that right it’s clear that you’re going to have these different roles and
responsibilities across the group and maybe with a small
number of principles you can manage that on your own. Alice is an administrator she can touch the production stuff. Bob is a developer he can
touch the development stuff. Now imagine there are 250 Alices and 250 Bobs suddenly your
life is a lot different. Role based access control is a way of managing granular access
to all cluster resources with a large group of
people it allows you to assign those people to roles and then permissions to those roles. So, Alice is an Alice. Alice is an administrator
and as an administrator she gets to access the production cluster. Bob isn’t just Bob,
he’s really a developer and as a developer he gets to access the development resources
it’s that kind of thing. So this is a resource that makes that kind of management much much easier. This is configured with
that new Confluent CLI that I told you about just a minute ago and I need to stress this is in preview as a 5.3, it’s a huge new feature, touches literally every
component of the platform. We would love for you to try it out and let us know what you think. If you’re already a customer of course you know, you’ve got those channels to be able to talk to us. If you’re not and you’ve
just downloaded this and you’ve played around with it and you want to talk to us always, always join us in community slack. You can talk there. You can comment on this video and you can reach out to us on
Twitter at @confluentinc. Any of those channels would be great and we would love to hear more. So that is 5.3 in a nutshell. You’ve got more agile, more
cloud-native deployments. You’ve got the better
ability to understand and manage your event
streams and you got a preview of role based access control. Download it now, check it out and I hope to hear from you soon. (upbeat bluegrass music)

3 thoughts on “Confluent Platform 5.3 | What’s New in This Release

Leave a Reply

Your email address will not be published. Required fields are marked *