Okay.
Hello lagers.
And welcome to this lecture and this lecture we're going to learn all about canisters.
Now before we start on what Carney says is we should start to learn what streaming data is and streaming
data is data that is generated continuously by thousands of data sources which typically send in the
data records simultaneously and in small sizes and we're talking about kilobytes here.
So let's have a look at some examples of streaming data.
So this could be purchases from an online store such as Amazon.com.
So every time you purchase a book or a Kindle or whatever you get the transaction I.D. you'll get the
actual you know device description you might get the unique I.D. for the individual item et cetera et
cetera.
And this is all just transactional data that is generated when you're purchasing something on an online
store.
Another example could just be stock prices.
So could be Tesla stock or it could be stock in anything Microsoft Amazon etc and stock prices move
around all the time.
And as soon as there's a change in the stock price that is then streamed in to something that needs
to process it.
We also have gaming data.
So maybe it's someone playing angry birds for example where they're up to a particular level and what
particular you know puzzle they've just gone through and solved could be social network data a really
good example of this is Twitter.
So it might be a particular hashtag.
How many tweets are against it how many re tweets how many comments et cetera et cetera.
And then geospatial data.
So think of things like Uber for example so when you're catching an Uber where you are on a map is always
streamed in to a server or a central source so they can then go through and calculate the fastest way
for you to get from your source to your destination and it'll take into account things like traffic
which it gets from other data sources as well.
And then finally we have IO T sensor data.
So you could have an iota garden you could be a farmer using IoT devices across your farm and all of
this data is being streamed into a central source.
So that's all streaming data is its data that's generated continuously by thousands of different sources.
And typically this will be done simultaneously and in small sizes and we're talking about kilobytes.
So now that you know what's streaming data is or what is Genesis sickness this is a platform on AWB
to send your streaming data to.
That's all it is it's a place to send all your streaming data to and Canisius makes it easy to load
and analyze streaming data and also providing the ability for you to build your own custom applications
for your business needs.
And we have three different types of Genesis.
We've got canister streams we've got Unisys firehose and we've got Unisys analytics and for the exam
you are going to need to understand the three different types of Genesis and what you should use where.
So let's start with cornices streams here we've got all our devices.
So it could be our mobile phone using Uber.
It could be our laptop tweeting something.
It could be easy to monitoring stock prices could be IO T.
You know you might have a farm or something and you've got all this farming data coming in.
So these are all your data producers and what they do is they stream the data to Kansas and Kansas streams
is a place to store that data by default it's going to store that data for 24 hours.
However can store it up to seven days.
So it's just a way of putting all that data or place to put all that data to your data is contained
in shards and we'll look at what a shard is in the next slide but you have might have a shard for different
purposes you might have a shard for your geospatial data you might have a shard for your stock data
you might have a shard for your social media data you might have a shard for IO T data etc. and the
data stored in shards which easy to instances sometimes known as data consumers can go in and analyze
that data inside those shards you might be wanting to you know have Ruby running an algorithm that predicts
the stock market well you might be doing sentiment analysis on Twitter on a particular hashtag do people
feel good about this particular hashtag or do people feel bad about a particular hashtag you could be
looking at.
It could be movers servers for example calculating the best way for you to get to your destination and
then you're see two instances once they've analyzed this data and done something with it they can then
store it in a number of different places soon dynamo D.B. they could store it in S3 they could store
it in EMR they could store it in redshift they can store it basically in places where the easy two instances
could access so RDX for example.
So that's all canister streams is it allows you to persistently store your data for 24 to 7 24 hours
to 7 days while your data consumers go through and do something with that data and then your data consumers
once they've analyzed that data they might the analytics somewhere in like dynamo D.B. or S3 or EMR
or redshift etc..
That's all Kansas streams is so cornices streams consists of shards shards are basically five transactions
per seconds for reeds and up to a maximum.
Total data read rate of 2 mega 2 megabytes per second and up to thousand records per second for rights
and up to a maximum total data write rate of 1 megabyte per second.
So that's what an individual shard consists of and then the data capacity of your stream is a function
of the number of shards that you specify for a stream and the total capacity of this stream is the sum
of the capacities of shards.
Now don't worry if that makes not a lot of sense you're not gonna be quizzed on any of this in the exam.
The thing that you just have to remember is that shards are stored in canister streams canister streams.
Your data when you put your data into an ISA streams it's all stored in individual shards.
That's all you need to remember so if you see shards come up in the exam I want you to think straight
away of canister streams because canister streams is the only form of canisters that has shards canisters
firehose.
Let's have a look at that.
So we've got our data producers so generously to instances our smartphones our laptops variety etc.
and this is stored what's sent to canisters firehose now inside canisters firehose.
There's no persistent storage.
The data has to be analysed as it comes in.
You have to do something with your data so it's optional to have lambda functions inside your canisters
firehose so soon as the data comes in it triggers a lambda function lambda function runs a particular
set of code for that data and then outputs it somewhere safe.
It could output it to S3 you could output it to redshift it can't directly output it to read if you
have to output it to S3 and then go.
Basically import the data into redshift or it could be going on to elastic search cluster etc..
So the key thing to remember the difference between consensus firehose and canister streams can ease
the streams has shards and your data has persistence by default.
It's stored for 24 hours so this has all kinds of streams as whereas canisters firehose there's no data
persistence you need to do something with that data as soon as it comes in to firehose Kansas.
Analytics works with canisters streams and with canisters firehose and essentially it can analyze the
data on the fly inside either service and then it goes in and stores this data either on S3 redshift
or elastic search cluster.
So onto my exam tips hopefully that made sense but essentially you just need to know the difference
between canister streams canisters firehose hose and you're gonna be given scenario questions where
you must choose the most relevant service so it can easily streams if you see mentions of shards or
even you just need to understand what cornices is at a high level so if you see something about using
streaming data think instantly cornices.
If it then asks you which can a service to use and it's talking about shards you're going to be using
canister streams if you need data persistence.
So if you need 24 hours or up to 7 days again you want to be using canister streams.
You don't care about data persistence.
You want to analyze it on the fly automatically using lambda.
Then you're going to be using Genesis firehose and then of course if you want to analyze your data inside
cornices you can use Genesis analytics and can use its analytics analyzes the data inside both kinds
of streams and Kansas firehose.
So that is it for this lecture everyone you've done really really well.
I know this was a very theoretical section of the course.
A lot of the application services have started to move away from the solutions architect exam to the
developer exam.
That being said you still need to understand escudos S.A. Kansas and definitely API Gateway API gateway
will come up an awful lot in your exam it's probably worth 5 or 10 marks.
So in the next lecture we're going to look at web identity federation and Amazon's Web identity Federation
service cognitive.
It's very very short lecture and then we'll move on to summarize everything we've learned in the application
section of the course.
So if you've got the time please join me in the next lecture.
Thank you.