---
title: re:Invent 2022 day 1 - A serverless streaming & event-driven ETL flow
description: CloudNation's AWS consultant Luuk Rutten writes live report from day one at re:Invent 2022 in Las Vegas!
image: https://www.cloudnation.nl/hubfs/k5Ng-9m4r.webp
---

 English

[ Nederlands](https://www.cloudnation.nl/inspiratie/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow)

an IG&H company

[![_CN_Monogram_Color_White_rgb](https://www.cloudnation.nl/hubfs/_CN_Monogram_Color_White_rgb.png "Back to home") ![preview__CN_Logo_Left_Color_Black_rgb](https://www.cloudnation.nl/hubfs/preview__CN_Logo_Left_Color_Black_rgb.png "Back to home") ](https://www.cloudnation.nl/en/)

 Services

 Migrate

[ Cloud Migration

 Realize a seamless transition to the cloud 

](https://www.cloudnation.nl/en/services/cloud-migration) [ Cloud Readiness Assessment

 Before you start your journey on the cloud. 

](https://www.cloudnation.nl/en/services/cloud-readiness-assessment) [ Cloud Landing Zone Accelerator

 The start of a cloud journey. 

](https://www.cloudnation.nl/en/services/cloud-landing-zone-accelerator) [ Advice & Strategy

 Use the knowhow of our best experts. 

](https://www.cloudnation.nl/en/services/strategy) [ Application Modernization

 Modernize, simplify and transform applications. 

](https://www.cloudnation.nl/en/services/applicationmodernization)

 Improve & Modernize

[ Cloud Security & Compliance

 Security & compliance 

](https://www.cloudnation.nl/en/services/cloud-security/old) [ Cloud Competence Center

 Accelerate development. 

](https://www.cloudnation.nl/en/cloud-center-of-excellence-cloudnation) [ FinOps

 Turn cloud spend into business value 

](https://www.cloudnation.nl/en/cloudnation-finops)

 Integrate

[ Hybrid cloud

 Combine the best of private and public cloud. 

](https://www.cloudnation.nl/en/services/hybrid-cloud) [ Network & connectivity

 Use the cloud optimally and create value. 

](https://www.cloudnation.nl/en/services/networking-connectivity) [ Managed Cloud Services

](https://www.cloudnation.nl/en/services/managed-cloud-services) [ Managed Kubernetes Services

](https://www.cloudnation.nl/en/en/services/managed-kubernetes-services)

 Data & AI

[ AI Solutions

 Harness the power of artificial intelligence 

](https://www.cloudnation.nl/en/artificial-intelligence) [ Data Landing Zone

 A secure and scalable foundation for your data operations. 

](https://www.cloudnation.nl/en/services/data-landing-zone) [ Agent Lake

 The governed foundation for enterprise AI agents 

](https://www.cloudnation.nl/en/agent-lake)

[![aws-grey](https://www.cloudnation.nl/hubfs/aws-grey.svg) ](https://aws.amazon.com/free/?trk=3c7428a8-c52c-45ed-8072-8e2ba8bf9225&sc_channel=ps&ef_id=Cj0KCQjwwvilBhCFARIsADvYi7InrIXPaSi0tr77kUKV_-nyhN-NviOxvxIBVTUFilC3rM2czBruU-gaAuXlEALw_wcB:G:s&s_kwcid=AL!4422!3!649687334390!e!!g!!aws!19738730142!145148221566&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) [![azure-grey](https://www.cloudnation.nl/hubfs/azure-grey.svg) ](https://azure.microsoft.com/en-us/free/search/?ef_id=_k_Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB_k_&OCID=AIDcmmy4pl1olr_SEM__k_Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB_k_&gclid=Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB) [![google-cloud-grey](https://www.cloudnation.nl/hubfs/google-cloud-grey.svg) ](https://cloud.google.com/gcp/?hl=nl&utm_source=google&utm_medium=cpc&utm_campaign=emea-nl-all-nl-bkws-all-all-trial-e-gcp-1011340&utm_content=text-ad-none-any-DEV_c-CRE_495030366170-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+~+GCP+~+General%23v3-KWID_43700060393215884-kwd-6458750523-userloc_9064924&utm_term=KW_google%20cloud-NET_g-PLAC_&&gad=1&gclid=Cj0KCQjwwvilBhCFARIsADvYi7Kh3LYZSTLyTiCJcfyw5z0iKjKdP0z6hgphICBbRRXr230tD1FvZiIaAh7OEALw_wcB&gclsrc=aw.ds) [![oracle-ci-grey](https://www.cloudnation.nl/hubfs/oracle-ci-grey.svg) ](https://www.oracle.com/) 

[ Services overview ](https://www.cloudnation.nl/en/services)

[ Success stories ](https://www.cloudnation.nl/en/success-stories)

 Industries

 Our **cloud-native specialists** have deep** knowhow** and relevant **experience** in various industries 

[ Branches overview ](https://www.cloudnation.nl/en/industries)

[ FinTech

 Accelerate FinTech innovations 

](https://www.cloudnation.nl/en/industries/finance-fintech) [ ISV

 Remain future proof and transform your applications through the cloud 

](https://www.cloudnation.nl/en/industries/isv) [ Manufacturing

 Improve operational efficiency, security and scalability of your processes. 

](https://www.cloudnation.nl/en/industries/production-industry) [ Digital Natives

 Unlock new possibilities with cloud solutions designed for the digital pioneer. 

](https://www.cloudnation.nl/en/branches/digital-natives)

[ Inspiration ](https://www.cloudnation.nl/en/inspiration)

 About Us

[ About CloudNation ](https://www.cloudnation.nl/en/about-us)

[ Careers ](https://www.cloudnation.nl/en/careers)

[ Our partners ](https://www.cloudnation.nl/en/partners)

[ CloudNation Academy ](https://www.cloudnation.nl/en/cloud-academy)

[ Contact ](https://www.cloudnation.nl/en/contact)

[Plan a call ](https://www.cloudnation.nl/en/schedule-a-meeting)

[![_CN_Monogram_Color_White_rgb](https://www.cloudnation.nl/hubfs/_CN_Monogram_Color_White_rgb.png "Back to home") ![preview__CN_Logo_Left_Color_Black_rgb](https://www.cloudnation.nl/hubfs/preview__CN_Logo_Left_Color_Black_rgb.png "Back to home") ](https://www.cloudnation.nl/en/)

 Services

 Migrate

[ Cloud Migration

 Realize a seamless transition to the cloud 

](https://www.cloudnation.nl/en/services/cloud-migration) [ Cloud Readiness Assessment

 Before you start your journey on the cloud. 

](https://www.cloudnation.nl/en/services/cloud-readiness-assessment) [ Cloud Landing Zone Accelerator

 The start of a cloud journey. 

](https://www.cloudnation.nl/en/services/cloud-landing-zone-accelerator) [ Advice & Strategy

 Use the knowhow of our best experts. 

](https://www.cloudnation.nl/en/services/strategy) [ Application Modernization

 Modernize, simplify and transform applications. 

](https://www.cloudnation.nl/en/services/applicationmodernization)

 Improve & Modernize

[ Cloud Security & Compliance

 Security & compliance 

](https://www.cloudnation.nl/en/services/cloud-security/old) [ Cloud Competence Center

 Accelerate development. 

](https://www.cloudnation.nl/en/cloud-center-of-excellence-cloudnation) [ FinOps

 Turn cloud spend into business value 

](https://www.cloudnation.nl/en/cloudnation-finops)

 Integrate

[ Hybrid cloud

 Combine the best of private and public cloud. 

](https://www.cloudnation.nl/en/services/hybrid-cloud) [ Network & connectivity

 Use the cloud optimally and create value. 

](https://www.cloudnation.nl/en/services/networking-connectivity) [ Managed Cloud Services

](https://www.cloudnation.nl/en/services/managed-cloud-services) [ Managed Kubernetes Services

](https://www.cloudnation.nl/en/en/services/managed-kubernetes-services)

 Data & AI

[ AI Solutions

 Harness the power of artificial intelligence 

](https://www.cloudnation.nl/en/artificial-intelligence) [ Data Landing Zone

 A secure and scalable foundation for your data operations. 

](https://www.cloudnation.nl/en/services/data-landing-zone) [ Agent Lake

 The governed foundation for enterprise AI agents 

](https://www.cloudnation.nl/en/agent-lake)

[![aws-grey](https://www.cloudnation.nl/hubfs/aws-grey.svg) ](https://aws.amazon.com/free/?trk=3c7428a8-c52c-45ed-8072-8e2ba8bf9225&sc_channel=ps&ef_id=Cj0KCQjwwvilBhCFARIsADvYi7InrIXPaSi0tr77kUKV_-nyhN-NviOxvxIBVTUFilC3rM2czBruU-gaAuXlEALw_wcB:G:s&s_kwcid=AL!4422!3!649687334390!e!!g!!aws!19738730142!145148221566&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) [![azure-grey](https://www.cloudnation.nl/hubfs/azure-grey.svg) ](https://azure.microsoft.com/en-us/free/search/?ef_id=_k_Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB_k_&OCID=AIDcmmy4pl1olr_SEM__k_Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB_k_&gclid=Cj0KCQjwwvilBhCFARIsADvYi7IzQ1lpK0myTZJUEGM4La7kZrvaGTatFZ5ommalhQTlkevfIc4qPNsaAl4FEALw_wcB) [![google-cloud-grey](https://www.cloudnation.nl/hubfs/google-cloud-grey.svg) ](https://cloud.google.com/gcp/?hl=nl&utm_source=google&utm_medium=cpc&utm_campaign=emea-nl-all-nl-bkws-all-all-trial-e-gcp-1011340&utm_content=text-ad-none-any-DEV_c-CRE_495030366170-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+~+GCP+~+General%23v3-KWID_43700060393215884-kwd-6458750523-userloc_9064924&utm_term=KW_google%20cloud-NET_g-PLAC_&&gad=1&gclid=Cj0KCQjwwvilBhCFARIsADvYi7Kh3LYZSTLyTiCJcfyw5z0iKjKdP0z6hgphICBbRRXr230tD1FvZiIaAh7OEALw_wcB&gclsrc=aw.ds) [![oracle-ci-grey](https://www.cloudnation.nl/hubfs/oracle-ci-grey.svg) ](https://www.oracle.com/) 

[ Services overview ](https://www.cloudnation.nl/en/services)

[ Success stories ](https://www.cloudnation.nl/en/success-stories)

 Industries

 Our **cloud-native specialists** have deep** knowhow** and relevant **experience** in various industries 

[ Branches overview ](https://www.cloudnation.nl/en/industries)

[ FinTech

 Accelerate FinTech innovations 

](https://www.cloudnation.nl/en/industries/finance-fintech) [ ISV

 Remain future proof and transform your applications through the cloud 

](https://www.cloudnation.nl/en/industries/isv) [ Manufacturing

 Improve operational efficiency, security and scalability of your processes. 

](https://www.cloudnation.nl/en/industries/production-industry) [ Digital Natives

 Unlock new possibilities with cloud solutions designed for the digital pioneer. 

](https://www.cloudnation.nl/en/branches/digital-natives)

[ Inspiration ](https://www.cloudnation.nl/en/inspiration)

 About Us

[ About CloudNation ](https://www.cloudnation.nl/en/about-us)

[ Careers ](https://www.cloudnation.nl/en/careers)

[ Our partners ](https://www.cloudnation.nl/en/partners)

[ CloudNation Academy ](https://www.cloudnation.nl/en/cloud-academy)

[ Contact ](https://www.cloudnation.nl/en/contact)

[Plan a call ](https://www.cloudnation.nl/en/schedule-a-meeting)

[![_CN_Monogram_Color_White_rgb](https://www.cloudnation.nl/hubfs/_CN_Monogram_Color_White_rgb.png "Back to home") ![preview__CN_Logo_Left_Color_Black_rgb](https://www.cloudnation.nl/hubfs/preview__CN_Logo_Left_Color_Black_rgb.png "Back to home") ](https://www.cloudnation.nl/en/)

[ Let's talk ](https://www.cloudnation.nl/en/schedule-a-meeting)

Menu

[![_CN_Monogram_Color_White_rgb](https://www.cloudnation.nl/hubfs/_CN_Monogram_Color_White_rgb.png "Back to home") ![preview__CN_Logo_Left_Color_Black_rgb](https://www.cloudnation.nl/hubfs/preview__CN_Logo_Left_Color_Black_rgb.png "Back to home") ](https://www.cloudnation.nl/en/)

[ Let's talk ](https://www.cloudnation.nl/en/schedule-a-meeting)

Menu

 Services

 Migrate

[ Cloud Migration 

](https://www.cloudnation.nl/en/services/cloud-migration) [ Cloud Readiness Assessment 

](https://www.cloudnation.nl/en/services/cloud-readiness-assessment) [ Cloud Landing Zone Accelerator 

](https://www.cloudnation.nl/en/services/cloud-landing-zone-accelerator) [ Advice & Strategy 

](https://www.cloudnation.nl/en/services/strategy) [ Application Modernization 

](https://www.cloudnation.nl/en/services/applicationmodernization)

 Improve & Modernize

[ Cloud Security & Compliance 

](https://www.cloudnation.nl/en/services/cloud-security/old) [ Cloud Competence Center 

](https://www.cloudnation.nl/en/cloud-center-of-excellence-cloudnation) [ FinOps 

](https://www.cloudnation.nl/en/cloudnation-finops)

 Integrate

[ Hybrid cloud 

](https://www.cloudnation.nl/en/services/hybrid-cloud) [ Network & connectivity 

](https://www.cloudnation.nl/en/services/networking-connectivity) [ Managed Cloud Services 

](https://www.cloudnation.nl/en/services/managed-cloud-services) [ Managed Kubernetes Services 

](https://www.cloudnation.nl/en/en/services/managed-kubernetes-services)

 Data & AI

[ AI Solutions 

](https://www.cloudnation.nl/en/artificial-intelligence) [ Data Landing Zone 

](https://www.cloudnation.nl/en/services/data-landing-zone) [ Agent Lake 

](https://www.cloudnation.nl/en/agent-lake) 

[Services overview ](https://www.cloudnation.nl/en/services)

[ Success stories ](https://www.cloudnation.nl/en/success-stories)

 Industries

[ FinTech ](https://www.cloudnation.nl/en/industries/finance-fintech)

[ ISV ](https://www.cloudnation.nl/en/industries/isv)

[ Manufacturing ](https://www.cloudnation.nl/en/industries/production-industry)

[ Digital Natives ](https://www.cloudnation.nl/en/branches/digital-natives) 

[Branches overview ](https://www.cloudnation.nl/en/industries)

[ Inspiration ](https://www.cloudnation.nl/en/inspiration)

 About Us

[ About CloudNation ](https://www.cloudnation.nl/en/about-us)

[ Careers ](https://www.cloudnation.nl/en/careers)

[ Our partners ](https://www.cloudnation.nl/en/partners)

[ CloudNation Academy ](https://www.cloudnation.nl/en/cloud-academy)

[ Contact ](https://www.cloudnation.nl/en/contact) 

[Let's talk ](https://www.cloudnation.nl/en/schedule-a-meeting)

[ nl](https://www.cloudnation.nl/inspiratie/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow)

[Back ](https://www.cloudnation.nl/en/inspiration/blogs)

[AWS](https://www.cloudnation.nl/en/inspiration/blogs/tag/aws) [re:invent 2022](https://www.cloudnation.nl/en/inspiration/blogs/tag/reinvent-2022)

# re:Invent 2022 day 1 - A serverless streaming & event-driven ETL flow

![](https://www.cloudnation.nl/hubfs/CN_sq__luuk.jpg)

**Luuk Rutten** Principal Cloud Engineer

 Publish date: 29 November 2022

![](https://www.cloudnation.nl/hs-fs/hubfs/k5Ng-9m4r.webp?width=1600)

<https://www.linkedin.com/shareArticle?mini=true&url=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow> <https://twitter.com/intent/tweet?text=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow> [mailto:info@example.com?&subject=&body=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow](mailto:info@example.com?&subject=&body=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow) <https://web.whatsapp.com/send?text=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow>

## Three architectural patterns for pay-per-use data lakes and data warehouses

##### Table of contents

- [The architecture of a feature-rich serverless streaming ETL flow](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#architecture)<https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#xray> 
    - [Pattern one - Glue jobs for data streaming](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#patternone)
    - [Pattern two - Redshift serverless and QuickSight for reporting](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#patterntwo) 
    - [Pattern three - automated updates of glue data catalog for Athena](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#patternthree)

- [More to come!](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#moretocome)

To kick-off the serverless re:Invent series, we'll start with one on AWS data engineering. It is based on re:Invent session `PEX-305-R` and touches upon a large set of serverless services. The main goal is to illustrate how these services can work together to form a serverless ETL flow for streaming (or batch processing) applications.

> NOTE: This is definitely not an architecture that you need to start out with as it will probably be over-engineered for small use-cases. However, parts of it you can apply to your own models.

 

# The architecture of a feature-rich serverless streaming ETL flow

Below you can a complete overview of a feature-rich serverless ETL flow in AWS. Various components are highlighted below and some of the architectural patterns are discussed more in-depth.

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669679541685/J27k3MzHA.png?auto=compress,format&format=webp)

An overview of serverless data services used:

1. Amazon Kinesis for streaming. Data can be inserted into the stream using any programming language.
2. S3 for storage of static, raw or processed data.
3. Glue jobs for ETL jobs and streaming ETL jobs, from multiple sources and to multiple destinations.
4. Redshift serverless for the data warehouse.
5. QuickSight for dashboarding and insights on data present in redshift.
6. EventBridge for event-based orchestration.
7. Step functions for workflow-based orchestration to kick off the crawlers.
8. Glue crawlers to keep the data catalog up to date when new partitions are being added.
9. Glue data catalog to be able to connect to datasets with other AWS services like Athena.
10. Athena for ad-hoc SQL queries on large datasets on S3.

 

## Pattern one - Glue jobs for data streaming

Glue jobs are basically spark scripts that can read from one or more data sources, transform and clean data, and then store the result in another data store. With glue data streaming, the glue job is continuously running and reading from a data stream like AWS Kinesis. It will read data in batches of `N` records and process then based on the job definition. An example of a visual job definition is given below:

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669679878521/3oiC8Kw2O.png?height=500&auto=compress,format&format=webp)

In the example above, a custom SQL query is used to query both a streaming data source and a static data source like S3 and combine that data and dump it in two locations. Firstly in S3 in for example a partitioned format based on a date-field. Secondly, it further processes the data and dumps it to Redshift.

A cool thing to note is the `Detect sensitive data` action. You can the glue job to configure to look for common sensitive data fields like email addresses, and choose to omit them or obscure them. In this example, the obscured data is stored in redshift to be safely made available for data analysts.

 

## Pattern two - Redshift serverless and QuickSight for reporting

If you use QuickSight for generating scheduled reports or, maybe even cooler, ask questions in natural language using QuickSight Q, this is something for you.

You can create a data source in QuickSight and connect it to a Redshift serverless cluster. Make sure to use 'Direct Query' mode (see image below) to make use of Redshifts native performance. If you don't do this, QuickSight will actually replicate the data and use its own storage to answer to queries.

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669684898451/F9EJiotEs.png?height=400&auto=compress,format&format=webp)

You can now for example use QuickSight Q to answer questions about your data. In the example below, you can ask to `show me attendees by country` and Q will automatically try to understand which fields need to be displayed and aggregated.

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669685045334/NGTlwmnrC.png?height=400&auto=compress,format&format=webp)

 

## Pattern three - automated updates of glue data catalog for Athena

One cool pattern is to automatically update your data catalogs as new data is coming in. Data catalogs are used to optimize queries by providing the query engine where it can find certain partitions of data so it doesn't need to read all files in S3 before answering to the query.

You can do this using S3, AWS EventBridge and Glue crawlers. As a new object is created, automatically an event is emitted. EventBridge can be configured to automatically start a workflow with AWS StepFunctions when this happens. Step functions can do many things, like invoke a lambda that runs a query to update a materialized view in your database, but also to start a Glue crawler. An example step function is shown below.

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669685845176/d6-YGW1KB.png?height=400&auto=compress,format&format=webp)

Glue crawlers crawl your S3 bucket for new partitions and data, and update the data catalog when it finds new partitions. This makes sure that your catalog is always up to date as new files are being added! This is great if you use AWS Athena to query these datasets, since it makes heavy use of the Glue Data catalog.

An example of such an Athena query is shown below. Note that the email addresses are not obscured here, as was intended.

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1669686103816/8eeS4x0sU.png?height=400&auto=compress,format&format=webp)

 

# More to come!

Hopefully you have an understanding of various architectural patterns in serverless ETL flows. In the next days, I will give some more updates in RDS aurora serverless V2, and AWS EMR serverless and how and when to use them. Stay tuned!

--

This blog also appeared on [cloudengineer.hashnode.dev](https://cloudengineer.hashnode.dev/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow-with-glue-redshift-serverless-and-quicksight)

#### Want to read more from our AWS consultants?

We regularly write about the newest updates, learnings and personal experiences on AWS.

[Read more ](https://www.cloudnation.nl/en/inspiration/blogs)

![cloudnation-sem](https://7979774.fs1.hubspotusercontent-na1.net/hubfs/7979774/cloudnation-sem.jpg)

<https://www.linkedin.com/shareArticle?mini=true&url=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow> <https://twitter.com/intent/tweet?text=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow> [mailto:info@example.com?&subject=&body=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow](mailto:info@example.com?&subject=&body=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow) <https://web.whatsapp.com/send?text=https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow>

![](https://www.cloudnation.nl/hubfs/CN_sq__luuk.jpg)

**Luuk Rutten** Principal Cloud Engineer

 Publish date: 29 November 2022

## More knowledge, howto's and insights

[![](https://7979774.fs1.hubspotusercontent-na1.net/hubfs/7979774/Right%20Banner%20size%20%288%29.png)

News

AWS

## CloudNation achieves AWS Premier Partner status

 Read more 

](https://www.cloudnation.nl/en/inspiration/blogs/cloudnation-achieves-aws-premier-partner-status) [![](https://7979774.fs1.hubspotusercontent-na1.net/hubfs/7979774/CloudNation%20Knows%20How%20Leading%20cloud%20modernization%20in%202025%20and%20beyond%20%288%29.png)

News

Knowledge

AWS

## How we built a real-time heartbeat processing pipeline at scale in AWS

 Read more 

](https://www.cloudnation.nl/en/inspiration/blogs/how-we-built-a-real-time-heartbeat-processing-pipeline-at-scale-in-aws) [![](https://7979774.fs1.hubspotusercontent-na1.net/hubfs/7979774/Right%20Banner%20size%20%285%29.png)

News

Knowledge

AWS

## Don't let Amazon EC2 be your attacker's way in

 Read more 

](https://www.cloudnation.nl/en/inspiration/blogs/dont-let-ec2-be-your-attackers-way-in)

<https://www.cloudnation.nl/>

Follow us: <https://www.linkedin.com/company/cloudnation-nl/> <https://twitter.com/cloudnationnl>

##### Contact

 Kosterijland 10

 3981 AJ Bunnik

[![blue-chat-icon](https://www.cloudnation.nl/hubfs/Development%20assets/blue-chat-icon.svg)   Chat with us

](https://www.cloudnation.nl/en/inspiration/blogs/reinvent-2022-day-1-a-serverless-streaming-event-driven-etl-flow#hs-chat-open) [![blue-phone-call](https://www.cloudnation.nl/hubfs/Development%20assets/blue-phone-call.svg)   +31 (0)6 48 016 896

](tel:0031648016896) [![blue-mail](https://www.cloudnation.nl/hubfs/Development%20assets/blue-mail.svg)   info@cloudnation.nl

](mailto:info@cloudnation.nl) ![iso 27001](https://www.cloudnation.nl/hubfs/iso%2027001.svg)  ![gdpr compliant](https://www.cloudnation.nl/hubfs/gdpr%20compliant.svg)

 

![iso 27001](https://www.cloudnation.nl/hubfs/iso%2027001.svg)  ![gdpr compliant](https://www.cloudnation.nl/hubfs/gdpr%20compliant.svg)

 

##### CloudNation

[ About CloudNation ](https://www.cloudnation.nl/en/about-us) [ Careers ](https://www.cloudnation.nl/werken-bij) [ CloudNation Academy ](https://www.cloudnation.nl/en/cloud-academy) [ Contact ](https://www.cloudnation.nl/contact)

##### Partners

[ All our partners ](https://www.cloudnation.nl/en/partners) [ AWS ](https://www.cloudnation.nl/en/partners/amazon-web-services#aws-partner) [ Azure ](https://www.cloudnation.nl/en/partners#microsoft-azure-partner) [ And lots more ](https://www.cloudnation.nl/en/partners)

##### Knowledge, howto's and insights of CloudNation in your mailbox