Contact Us:
07002007332
CheapDeveloper
CheapDeveloper » Webmaster » Articles » AWS re:Invent 2021 Keynotes - Data storage and IoT

AWS re:Invent 2021 Keynotes - Data storage and IoT

03 December 2021, Friday By Priyanka Boruah
233
0

So AWS re:Invent 2021 is coming to an end - one of the main events of the year in the world of cloud technologies. And in this article we would like to share the final portion of announcements. There are a lot of interesting news, starting with the price reduction for S3 storage, the launch of EC2 based on M1, the expansion of network capabilities, and ending with a lot of new products for developers. You can not only read, but also connect to the stream, where architects from AWS will discuss these new items and answer your questions.

New functionality for VPC:

1. IP Address Manager (IPAM) 

AWS IP Address Manager

With IP Address Manager (IPAM), network administrators have the ability to assign, monitor, structure and audit IP addresses on their networks, reducing the amount of time they need to manage and monitor network spaces.

2. VPC Network Access Analyzer

New functionality in Amazon VPC Network Access Analyzer allows you to identify inadvertent network access to AWS resources. It helps you ensure that your VPC AWS networking environment meets the required security and regulatory requirements. More details on the link.

AWS Transit Gateway (TGW) supports peering with another TGW within the same region

AWS Transit Gateway

Until now, AWS Transit Gateway (TGW) could only be peered to another AWS Transit Gateway (TGW) if it was in a different AWS Region. It is now possible to peer two AWS Transit Gateway (TGW) within the same region, which allows for more flexible network spaces and easier network integration with various third-party vendors or partners who have a network environment in the same region.

AWS Direct Connect SiteLink

AWS Direct Connect SiteLink

SiteLink is a new AWS Direct Connect (DX) feature that allows you to send data from one DX location to another, bypassing the AWS region's network infrastructure. Recall that DirectConnect allows you to connect your local network infrastructure to the AWS cloud bypassing the Internet, which provides guaranteed bandwidth and lower network latencies. Prior to SiteLink's functionality, it was not possible to directly route traffic between DX locations. It is now possible to create global network connectivity between different offices and data centers using the AWS global backbone. This reduces network latency by sending data along the shortest path between DX locations.

AWS Local Zones

AWS Local Zones

AWS Local Zones is a type of AWS infrastructure that contains computing resources, data storage, databases, and several other services in locations close to major population and industrial centers. 

Starting in 2022, AWS will launch over 30 new Local Zones in major cities around the world (over 21 countries), in addition to 16 Local Zones in the United States. This will help meet the challenges of end users around the world with much lower network latency. 

AWS Cloud WAN (preview)

AWS Cloud WAN (preview)

The new WAN (wide area networking) is a service that allows you to build, manage and monitor a unified wide area network that manages traffic between resources in the AWS cloud and on-premises environments. 

With Cloud WAN, you use a central dashboard and network policies to create a WAN that spans across multiple locations and networks. Now there is no need to configure and manage different parts of the network separately, perhaps even using a variety of tools. Cloud WAN automatically creates a WAN across multiple AWS Regions using BGP, making it easy to exchange routes within the network around the world. 

AWS Shield Advanced - Automated Application Layer DDoS Mitigation

All AWS customers benefit from automatic protection against known DDoS attacks at the network layer (L3, L4) free of charge using AWS Shield Standard to improve the availability of their AWS services.

For additional protection against advanced DDoS attacks at different network layers (L3-L7) that attack your applications, you can sign up for a paid subscription to AWS Shield Advanced. AWS Shield Advanced provides more sensitive detection methods and customized defenses against large and complex DDoS attacks, provides additional tools for monitoring such attacks in near real time, and also supports integration with  AWS WAF , a firewall for protecting applications from network attacks on 7m ( L7) level. 

With the new AWS Shield Advanced feature, after preconfiguration, it detects patterns associated with DDoS events at Layer 7, isolates this abnormal traffic by automatically creating AWS WAF rules in your Access Control Lists (Web ACLs). These rules can be run in Count mode to analyze their impact on resource traffic, and then switched to blocking mode. These capabilities enable you to quickly respond to and mitigate DDoS events that threaten the availability of your applications.

The action performed by the auto-generated WAF rules can be changed to monitor or block mode at any time. You can also view DDoS detection and mitigation metrics at the application level (L7) for further investigation or impact assessment of any mitigation tools Shield creates.

New EC2 instances on M1

A year ago, at re:Invent, AWS launched a new type of virtual machines based on macOS, which greatly simplified the life of developers in the Apple ecosystem. But the development of ARM processors does not stand still, and a good example of this is the 3rd generation Graviton with significant performance gains. Now you can run ARM-based macOS on new instance types - EC2 M1 Mac. Compared to x86 chips, this gives a performance increase of up to 60% in the process of developing applications for: iPhone, iPad, Apple Watch and Apple TV. EC2 M1 Mac is powered by two physical components:

  • Mac mini with M1 chip configuration: 8-core processor, 8-core GPU, 16 Gb pooled memory and 16-core Neural Engine;
  • AWS Nitro system providing network bandwidth up to 10 Gbps and 8 Gbps to EBS storage connected via Thunderbolt. 

New EC2 instances on M1

Data storage

Amazon S3 Support on AWS Backup (Preview)

Amazon S3 Support on AWS Backup

AWS Backup allows you to centrally manage backup policies for various AWS services. New integration with Amazon S3 makes it possible to create backups of your buckets both at a specific point in time and on a schedule. These include the objects themselves in buckets, their tags, access rules (ACL), and metadata. The first backup is full, and all subsequent ones are incremental. If necessary, you can restore both the entire bucket and individual objects from it. AWS Backup is integrated with organizations to enable you to centrally set backup policies.

In addition, you can protect your backups from accidental deletion using AWS Backup Vault Lock.

You can register for the Preview Program here.

New Amazon S3 Glacier Instant Retrieval Storage Class and Price Cuts

New Amazon S3 Glacier Instant Retrieval Storage Class

AWS has released a new Amazon S3 Glacier Instant Retrieval storage class. It is suitable for archived data and provides the lowest storage cost for long-lived data that is rarely accessed (typically quarterly) but requires millisecond retrieval.

The new storage class is supported in S3 Intelligent-Tiering, which automatically moves objects between multiple classes based on their access pattern.

In addition, the Amazon S3 Glacier class has been renamed S3 Glacier Flexible Retrieval and now provides free bulk retrieval for 5-12 hours, and storage costs have been reduced by 10% in all regions since December 1, 2021.

Finally, AWS announced up to 31% reduction in storage prices for S3 Standard-IA and S3 One Zone-IA in the US West (N. California), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Osaka), Asia Pacific regions (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo) and South America (São Paulo).

Support for VMware and VMware Cloud on AWS on AWS Backup

Support for VMware and VMware Cloud on AWS on AWS Backup

AWS has announced the release of VMware support in AWS Backup, with which you can automate data protection in your VMware virtual machines that are running both in data centers outside the cloud and in VMware Cloud on AWS. AWS Backup allows you to use centralized backup policies for VMware along with other supported services.

Backups are stored in storage in 3 availability zones and provide data safety at 99.999999999% (11 nines) and availability at 99.99%.

Amazon FSx for OpenZFS

Amazon FSx for OpenZFS

Amazon FSx for OpenZFS allows you to run fully managed file systems on AWS that replaces ZFS or other Linux-based on-premises file servers, helping to increase flexibility and reduce costs. FSx for OpenZFS is a new addition to Amazon FSx Services, which offers full-featured, high-performance file storage based on widely used file systems, including NetApp ONTAP, Windows File Server, and Luster.

FSx for OpenZFS is built on the OpenZFS open source file system, which is widely used locally for storing and managing exabytes of data for various workloads, such as machine learning, automation of electronic device design, application build environments, multimedia processing and financial analytics: scale, performance and economic efficiency are of primary importance in them. Amazon Fix for OpenZFS is powered by AMD Graviton processors and the latest AWS disk and network technologies and provides up to 1 million I/O operations per second with a delay of hundreds of microseconds.

Next Generation Amazon FSx for Luster and Better Amazon S3 Integration

Next Generation Amazon FSx for Luster Powered by AWS Graviton Processors, provides several performance and price improvements. First, the new file systems provide up to 5x higher throughput per terabyte (up to 1 GB/s per terabyte size) than previous generation file systems. Second, with support for multi-NI instances, you can now provide up to 400 Gbps of network bandwidth on Amazon EC2 instances such as P4d and DL1. Third, the new generation of FSx for Luster file systems reduces your bandwidth costs by up to 60% compared to previous generation file systems. To help you optimize your storage costs even better, FSx for Luster now allows you to enable data compression so you can reduce storage size without negatively impacting performance.

In addition, AWS announced two additional Amazon FSx features for Luster. The first is full bi-directional file system synchronization with Amazon Simple Storage Service (Amazon S3), including deleted files and objects. The second is the ability to sync file systems with multiple buckets or S3 prefixes.

AWS re:Post - community service for questions and answers from experts

AWS re:Post is a community-driven Q&A service that provides expert guidance and best practices for working with AWS from AWS customers, partners, and employees. The service automatically distributes questions among experts based on their area of ​​expertise, which accordingly improves the quality of answers. With a Premium Support subscription under your account, you can receive prioritized responses from AWS Support Engineers. The service also uses machine learning for high-quality and relevant search results.

AWS re:Post

AWS Amplify Studio - A New Visual Environment for Rapid Application Development

The next item was the release of the AWS Amplify Studio product. AWS Amplify Studio is a visual development environment that allows developers and not only to easily create and release web and mobile applications in a very short time. With the help of Amplify Studio, you can quickly create the backend of an application, create components of a fully functional user interface (UI) and connect the user interface to the backend with minimal effort. Amplify Studio allows you to export all the elements of the user interface and infrastructure in the form of code, while giving you the opportunity to fully control the design and behavior of the application without experience with clouds and AWS in particular.

AWS Amplify Studio - A New Visual Environment for Rapid Application Development

Microservice Extractor for .NET

Interesting news was the release of the Microservice Extractor service for .NET, the purpose of which is to automate and simplify the process of refactoring and dividing monolithic applications developed on the .NET platform into more compact and independent services. The service supports the ASP.NET framework and allows you to plan, visualize and modernize applications written in it.

AWS Microservice Extractor service

AWS Cloud Development Kit (AWS CDK) version 2 and Construct Hub

AWS has released the second version of its AWS Cloud Development Kit, an open source framework for describing the infrastructure and resources of cloud applications using familiar programming languages ​​as part of the Infrastructure as Code approach. For customers who prefer Terraform, there is a Cloud Development Kit for Terraform that provides CDK constructs for defining Terraform HCL state files using TypeScript and Python. For Kubernetes users, the CDK8sproject project allows you to use CDK constructs to define Kubernetes configuration through TypeScript, Python, Java, and Go. 

Construct Hub is an open-source community where everyone can find and share CDK libraries. At the time of launch, Construct Hub contains more than 700 CDK libraries, including key AWS CDK modules, which will allow developers to describe the infrastructure faster using their favorite programming language. You can get acquainted with the list of already created libraries here.

You can learn more about the changes and use of Construct Hub in the AWS blog.

AWS SDK Release for Swift, Kotlin and Rust

Also a pleasant announcement was the release of the AWS SDK for Swift, Kotlin and Rust. This will allow developers to simplify interactions with AWS services and their public APIs on even more platforms.

IoT

Open repository of components for IoT Greengrass

AWS IoT Greengrass is a collection of software modules installed on IoT devices and the IoT Greengrass cloud service. The components allow you to transfer control, processing and data exchange with the cloud to the device.

The IoT Greengrass Component Repository is a collection of ready-made modules available on Github that you can use to accelerate the development of your IoT applications. Currently, such modules are available as uploading video to Amazon Kinesis Video Streams, a Modbus TCP implementation, a local database for InfluxDB time series, and a Grafana data visualizer.

For example, a video upload module can be used to transmit an RTSP stream from video cameras to a local Greengrass device. InfluxDB and Grafana database can be used to visualize sensor data.

Managing AWS IoT Greengrass Devices with AWS Systems Manager

AWS Systems Manager makes it easy to manage EC2 instances, AWS Outpost, and even servers in your datacenter. Now Systems Manager has learned to manage IoT Greengras devices as well. With it, you can make operating system upgrades, perform various maintenance and gain access to devices. With Systems Manager, you can manage all of your devices from one point.

For example, a medical device manufacturer could use AWS IoT Greengrass to update software versions and test updates, but now all this can be done automatically on a schedule through Systems Manager. 

Through the integration, administrators can use AWS Systems Manager to get a single, consolidated view of their entire IT infrastructure. For example, in production, AWS Systems Manager will allow you to manage both industrial computers and shop floor servers and EC2 instances at the same time.

AWS IoT RoboRunner Preview

As elsewhere in IT, the more varied the fleet of machines you have to deal with, the more difficult it is to manage. For many classes of devices, this task has already been solved, and you can manage the infrastructure built on equipment from different vendors from one console. But not for robots. AWS IoT RoboRunner addresses this challenge.

When you have robots of different types and from different vendors, you need a tool that provides unified access to the data you need to build applications that work with your entire fleet of robots. Every time a new robot arrives in the park, it requires a complex, time-consuming integration with the job management system.

AWS IoT RoboRunner

AWS IoT RoboRunner makes it easy to integrate new types of robots, build and deploy applications that help fleets of robots work seamlessly together. For example, you can define the location where your robots are used in the AWS Management Console. AWS IoT RoboRunner will automatically create a centralized repository for all data about robots and their tasks. All robots in this location will form a single robot park, but each robot will be "visible" separately.

You then use the Fleet Gateway Library to design integration code with your robots and work management system to retrieve data from each fleet of robots. You can use the Task Manager Library to control robots, and then run robot control applications in AWS Lambda or directly on devices using IoT Greengrass.

AWS IoT RoboRunner provides an interface and API to provide a consistent way to manage your entire fleet of robots. You can also use sample applications that you can use as a basis to speed up development.

This new service is based on the same technology used in Amazon fulfillment centers. It is now available to all developers to create cutting-edge robotics applications.

AWS IoT TwinMaker Digital Twin Service Preview

AWS IoT TwinMaker allows developers to create digital twins of real systems to monitor and optimize processes. Digital twins are virtual representations of physical objects such as buildings, factories, production lines, and equipment. Their state is constantly updated with real data to mimic the behavior of the systems they represent. The fields of application of digital twins are diverse. One of the most popular is to use the available data to gain a deeper understanding of how these systems work.

AWS IoT TwinMaker Digital Twin Service Preview

With AWS IoT TwinMaker, you can quickly create digital twins of equipment, processes, and objects by integrating data from multiple sources - sensors, videos, and applications - without the need for a centralized repository. You can use the connectors already available for different AWS services: AWS IoT SiteWise for hardware and sensor data, Amazon Kinesis Video Streams for video, Amazon Simple Storage Service (S3) for storing files such as CAD drawings and application data. You can create your own connectors to third-party sources like Snowflake or Siemens MindSphere. AWS IoT TwinMaker understands the relationships and connections between virtual copies of your physical systems and data sources and creates an accurate model of them.

AWS IoT TwinMaker lets you work with 3D models. For example, you can create a virtual copy of a manufacturing site with all machines and other manufacturing equipment and use the data obtained from sensors and equipment to simulate its operation. Or use machine learning to simulate behavior in different situations. TwinMaker has a built-in plugin for Amazon Managed Grafana for visualizing data on dashboards.

AWS IoT FleetWise Preview

A modern car is a computer. AWS IoT FleetWise is a new service that makes it easier and cheaper to collect vehicle data. With FleetWise, vehicle manufacturers can collect data for remote diagnostics, assess fleet health to reduce warranty returns and recalls, and collect data to train machine learning models used in autonomous driving systems or ADAS assistants.

FleetWise provides a set of libraries that are installed on board the vehicle and implement a single data exchange interface, regardless of the formats used by the onboard systems. Manufacturers can choose which events are sent to the cloud. The transmitted data can be used for automatic decision making and proactive actions, allowing the manufacturer to detect potential problems even before the owner contacts the auto service.

Conclusion

AWS has an ambitious goal of training 29 million cloud professionals by 2025. And we believe that re: Invent 2021 has shown a broad development perspective for almost every IT professional. Looking at the number of announcements and the variety of services, you can clearly see that an increasing number of specialists are involved in working with cloud technologies: from developers, DevOps engineers and Data Scientists to analysts and designers whose job is not to write code. 

You can read all important announcements of the past re: Invent 2021:

*Today's stream will be added later in this article.

Discuss

Read also:

AWS re:Invent 2021: Keynotes
02 December 2021, Thursday
AWS re:Invent 2021: Keynotes
AWS re:Invent 2021 Keynotes - AI/ML
03 December 2021, Friday
AWS re:Invent 2021 Keynotes - AI/ML
What is a dashboard
25 November 2021, Thursday
What is a dashboard
What is DHCP protocol
07 December 2021, Tuesday
What is DHCP protocol
What is Jira Software and How To Work With It
24 November 2021, Wednesday
What is Jira Software and How To Work With It
Add a comment
Comments (0)
Comment
Partners