😁Serverless Solution Architecture Discussions

Mobile application: MyTodoList

• We want to create a mobile application with the following requirements

• Expose as REST API with HTTPS

• Serverless architecture

• Users should be able to directly interact with their own folder in S3

• Users should authenticate through a managed serverless service

• The users can write and read to-dos, but they mostly read them

• The database should scale, and have some high read throughput

Mobile app: REST API layer

Mobile app: giving users access to S3

Mobile app: high read throughput, static data

Mobile app: caching at the API Gateway

In this lecture

• Serverless REST API: HTTPS, API Gateway, Lambda, DynamoDB

• Using Cognito to generate temporary credentials with STS to access S3 bucket with restricted policy. App users can directly access AWS resources this way. Pattern can be applied to DynamoDB, Lambda…

• Caching the reads on DynamoDB using DAX

• Caching the REST requests at the API Gateway level

• Security for authentication and authorization with Cognito, STS

Serverless hosted website: MyBlog.com

• This website should scale globally

• Blogs are rarely written, but often read

• Some of the website is purely static files, the rest is a dynamic REST API

• Caching must be implement where possible

• Any new users that subscribes should receive a welcome email

• Any photo uploaded to the blog should have a thumbnail generated

Adding a public serverless REST API

AWS Hosted Website Summary

• We’ve seen static content being distributed using CloudFront with S3

• The REST API was serverless, didn’t need Cognito because public

• We leveraged a Global DynamoDB table to serve the data globally

• (we could have used Aurora Global Database)

• We enabled DynamoDB streams to trigger a Lambda function

• The lambda function had an IAM role which could use SES

• SES (Simple Email Service) was used to send emails in a serverless way

• S3 can trigger SQS / SNS / Lambda to notify of events

Micro Services architecture

• We want to switch to a micro service architecture

• Many services interact with each other directly using a REST API

• Each architecture for each micro service may vary in form and shape

• We want a micro-service architecture so we can have a leaner development lifecycle for each service

Discussions on Micro Services

• You are free to design each micro-service the way you want

• Synchronous patterns: API Gateway, Load Balancers

• Asynchronous patterns: SQS, Kinesis, SNS, Lambda triggers (S3)

• Challenges with micro-services:

• repeated overhead for creating each new microservice,

• issues with optimizing server density/utilization

• complexity of running multiple versions of multiple microservices simultaneously

• proliferation of client-side code requirements to integrate with many separate services.

• Some of the challenges are solved by Serverless patterns:

• API Gateway, Lambda scale automatically and you pay per usage

• You can easily clone API, reproduce environments

• Generated client SDK through Swagger integration for the API Gateway

Distributing paid content

• We sell videos online and users have to paid to buy videos

• Each videos can be bought by many different customers

• We only want to distribute videos to users who are premium users

• We have a database of premium users

• Links we send to premium users should be short lived

• Our application is global

• We want to be fully serverless

Start simple, premium user service

Add authentication

Add Videos Storage Service

Premium User Video service

• We have implemented a fully serverless solution:

• Cognito for authentication

• DynamoDB for storing users that are premium

• 2 serverless applications

• Premium User registration

• CloudFront Signed URL generator

• Content is stored in S3 (serverless and scalable)

• Integrated with CloudFront with OAI for security (users can’t bypass)

• CloudFront can only be used using Signed URLs to prevent unauthorized users

• What about S3 Signed URL? They’re not efficient for global access

Software updates offloading

• We have an application running on EC2, that distributes software updates once in a while

• When a new software update is out, we get a lot of request and the content is distributed in mass over the network. It’s very costly

• We don’t want to change our application, but want to optimize our cost and CPU, how can we do it?

Why CloudFront?

• No changes to architecture

• Will cache software update files at the edge

• Software update files are not dynamic, they’re static (never changing)

• Our EC2 instances aren’t serverless

• But CloudFront is, and will scale for us

• Our ASG will not scale as much, and we’ll save tremendously in EC2

• We’ll also save in availability, network bandwidth cost, etc

• Easy way to make an existing application more scalable and cheaper

Big Data Ingestion Pipeline

• We want the ingestion pipeline to be fully serverless

• We want to collect data in real time

• We want to transform the data

• We want to query the transformed data using SQL

• The reports created using the queries should be in S3

• We want to load that data into a warehouse and create dashboards

Big Data Ingestion Pipeline discussion

• IoT Core allows you to harvest data from IoT devices

• Kinesis is great for real-time data collection

• Firehose helps with data delivery to S3 in near real-time (1 minute)

• Lambda can help Firehose with data transformations

• Amazon S3 can trigger notifications to SQS

• Lambda can subscribe to SQS (we could have connecter S3 to Lambda)

• Athena is a serverless SQL service and results are stored in S3

• The reporting bucket contains analyzed data and can be used by reporting tool such as AWS QuickSight, Redshift, etc…