The 10th edition of Re:invent, AWS’s yearly learning conference, was the opportunity for Amazon’s Cloud Computing company to unveil dozens of new launches and announcements. DataScientest’s AWS community builder Chouaieb Nemri shares it’s top 5 with us.
With over 20 000 attendees and hundreds of thousands people following the event online, Re:Invent was an event not to be missed. Amazon Web Services enjoyed the occasion to release most announcements and new features that will shape Amazon’s cloud computing services this year.
For Chouaieb Nemri, the introduction of Serverless mode for Redshift, the launch of Serverless option for Elastic MapReduce, on-demand capacity mode for Kinesis Data Streams, Serverless option for Amazon MSK and ACID Transactions enabled for S3 Athena users were the one to remember.
Serverless mode for Redshift now in public preview
AWS introduced Serverless mode for Redshift and is now in public preview. Redshift is a fully managed petabyte-scale Data Warehouse technology. It will help you use SQL to analyze structured and semi-structured data across data warehouses, operational databases, and data lakes.
Amazon Redshift Serverless will be a strong help if you have unpredictable compute needs such as variable workloads, periodic workloads with idle time, and steady-state workloads with spikes! And good news, you only pay for the service when your data warehouse is in use, and you are querying or loading data. AWS won’t charge you when your data warehouse is idle.
Serverless option for Elastic MapReduce
Amazon also announced the launch of Serverless option for EMR. Elastic MapReduce is a cloud based big data platform. It can be used to run large-scale distributed data processing jobs, interactive SQL queries, and machine learning applications using open-source analytics frameworks such as Apache Spark, Apache Hive, and Presto. Serverless Option for EMR is currently in Public Review. It provides a serverless runtime environment that simplifies running analytics applications using the latest open source frameworks such as Apache Spark and Apache Hive.
On-demand capacity mode for Kinesis Data Streams
AWS also added on-demand capacity mode for Kinesis Data Streams! As a reminder, Kinesis Data Streams is a fully managed, serverless streaming data service making it easy to elastically ingest and store logs, events, clickstreams, and other forms of streaming data in real time. Shards are the way to define capacity in Kinesis Data Streams in provisioned capacity mode. Each shard, which defines capacity in Kinesis Data Streams, can ingest 1 MB/s and 1,000 records/second and egress up to 2 MB/s.
This new capacity mode eliminates the need for provisioning and managing the capacity for streaming data. Using Kinesis Data Streams On-demand automatically scales the capacity in response to varying data traffic.
For those who prefer open-source Apache Kafka to Kinesis, AWS added Serverless option for Amazon MSK. MSK makes it easy to ingest and process streaming data in real time with fully managed Apache Kafka. MSK Serverless automatically provisions and scales compute and storage resources, so you can use Apache Kafka on demand and only pay for the data you stream and retain.
S3 Athena users able to permform ACID Transactions
Last but not least, S3 Athena users will now be able to perform ACID Transactions. This will help multiple concurrent users make reliable, row-level modifications to their Amazon S3 data from Athena’s console, API, ODBC and JDBC drivers.
Built on top of Apache Iceberg table format, Athena ACID transactions are compatible with other services and engines such as Amazon EMR and Apache Spark that support the Iceberg table format.
A lot of good news during this re:Invent 10th edition, which is making our lives easier as developers. More AWS announcements are being released every week by AWS, you can find out more on the dedicated What’s New at AWS page.
