The commoditization of technology has reached its pinnacle with the advent of the recent paradigm of Cloud Computing. Infosys Cloud Computing blog is a platform to exchange thoughts, ideas and opinions with Infosys experts on Cloud Computing

« April 2021 | Main

June 17, 2021

Package AWS Lambda functions as container images

With AWS Lambda, clients upload their code and let it run without having to worry about servers. Many enterprises like how this works, but if they have invested in container tools for their development workflows, it's not easy to apply the same approach to designing applications using Lambda.

To make this easier, you can now bundle and deploy Lambda functions as container images up to 10 GB in size.  This change unlocks many new workload possibilities, especially for data- intensive or dependency- intensive applications.

You may also quickly develop and deploy bigger workloads that rely on a large number of dependencies, such as machine learning or data-intensive workloads, using this method as well. Functions deployed as container images have many of the same characteristics as functions packaged as ZIP files, including operational simplicity, automated scaling, high availability, and native integrations with a wide variety of services.

This new method also provides more portability across the various AWS computing alternatives, like AWS Fargate or Amazon EC2.

This feature blurs the boundary between Lambda and containers and may create misunderstandings. Therefore, I believe it is best to begin by defining what this feature is not.

This new method also provides more portability across the various AWS computing alternatives, like AWS Fargate or Amazon EC2.

This feature blurs the boundary between Lambda and containers and may create misunderstandings. Therefore, I believe it is best to begin by defining what this feature is not.

This is not a substitute for ECS or EKS.

You can't run long-running services with Lambda since your code is still limited by Lambda's invocation mechanism, which means it only runs when the function is called and for a maximum of 15 minutes.

How it works

We can use one of the numerous AWS base images for all of the supported Lambda runtimes (Python, Node.js, Java, .NET, Go, Ruby), which contain the OS and everything else required to support the Lambda programming model. Additional libraries, such as the AWS SDK, are available, much like the managed Lambda runtimes.

Additionally, we can deploy our own arbitrary base images to Lambda, such as those based on Alpine or Debian Linux. These images must implement the Lambda Runtime API in order to function with Lambda.

To make it simpler to create your own base images, AWS provides Lambda Runtime Interface clients for all supported runtimes. These implementations are accessible through native package managers, making them easy to include in your images, and are distributed under an open source license.

Lambda requirements for container images:

Take note of the following prerequisites while deploying a container image to Lambda:

·         Lambda currently supports only container images that are based on Linux.

·         Lambda Runtime API must be implemented by the container image. The API is implemented using the AWS open-source runtime interface clients. To make your chosen base image compatible with Lambda, you may add a runtime interface client.

·         The container image must be able to operate on a file system that is read-only. Your function code may write to the writable /tmp directory, which has a capacity of 512 MB.

·         The default Lambda user must have the ability to read all files necessary to execute your function code. Lambda adheres to security best practices by creating a default Linux user with the fewest possible privileges. Ascertain that your application code does not depend on files that are not executable by other Linux users.

Detailed Implementation:

When you build a Python container image using an AWS base image, you just need to transfer the Python application to the container and install any dependencies.

We can deploy a Lambda application as a container by following the below steps.

·         Develop an application.

·         Create Dockerfile.

·         Run 'docker build'.

·         Test locally.

·         Push created container image to Amazon ECR.

·         Deploy Lambda function.


Container0.jpg

Prerequisites:

·         The AWS Command Line Interface (AWS CLI)

·         Docker Desktop

·         Your function code

Steps: 

1.       Step 1:     Develop an application

Create a project directory for your new function. Create a file called application.py in your project directory that contains your function code. A basic Python handler is shown in the following example.

import time

import sys

def lambda_handler(event, context):  

    print("Lambda function ARN:", context.invoked_function_arn)

    print("CloudWatch log stream name:", context.log_stream_name)

    print("CloudWatch log group name:",  context.log_group_name)

    print("Lambda Request ID:", context.aws_request_id)

    print("Lambda function memory limits in MB:", context.memory_limit_in_mb)

    # We have added a 1 second delay so you can see the time remaining in get_remaining_time_in_millis.

    time.sleep(1)

    print("Lambda time remaining in MS:", context.get_remaining_time_in_millis())

    return 'Hello from AWS Lambda containerization application using Python' + sys.version + '!'   


2. Create a Dockerfile

Create a Dockerfile in your project directory using a text editor.

The Dockerfile for the handler that you built in the previous step is shown in the example below.


# Use the python base image from Amazon ECR public

FROM public.ecr.aws/lambda/python:3.8

# Alternatively, you can pull the base image from Docker Hub: amazon/aws-lambda-python:3.8

 

# Copy handler function (from the local app directory)

COPY application.py .

 

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)

CMD ["application.handler"]   


If your Lambda function imports Python libraries, modify the above procedure as follows:

In your project directory, add a file named requirements.txt. List each required library as a separate line in this file.

Modify your Dockerfile to add the required libraries to the container image. The following example copies the requirements file and installs the required libraries into the app directory.

 

FROM public.ecr.aws/lambda/python:3.8

 

# Create function directory

WORKDIR /app

 

# Install the function's dependencies

# Copy file requirements.txt from your project folder and install

# the requirements in the app directory.

 

COPY requirements.txt  .

RUN  pip3 install -r requirements.txt

 

# Copy handler function (from the local app directory)

COPY  application.py  .

 

# Overwrite the command by providing a different command directly in the template.

CMD ["/app/application.handler"]  


Step 3: Run 'docker build'

Build your Docker image with the docker build command. Enter a name for the image. The following example names the image hello-world.

docker build -t docker-lambda .


Step 4 Test locally.

To check if this is working, start the container image locally using the Lambda Runtime Interface Emulator

docker run -d -p 8080:8080 docker-lambda

     

Step 5:  Push created container image to Amazon ECR.

To upload the container image, I create a new ECR repository in my account and tag the local image to push it to ECR.

 aws ecr create-repository --repository-name docker-lambda

 

Push images to ECR

 aws_region=us-east-1

aws_account_id=<<xxxxxxxxxxxx>>

 

aws ecr get-login-password \

    --region $aws_region \

| docker login \

    --username AWS \

    --password-stdin $aws_account_id.dkr.ecr.$aws_region.amazonaws.com

 

Note: Replace <<xxxxxxxxxxxx>> with the original AWS account id

We need to tag/rename our previously created image to an ECR format. 

docker tag docker-lambda $aws_account_id.dkr.ecr.$aws_region.amazonaws.com/docker-lambda

 

push the image to ECR Registry.

docker push =<<xxxxxxxxxxxx>>.dkr.ecr.us-east1.amazonaws.com/docker-lambda

Step 6: Deploy Lambda function

We're going to deploy a Lambda function using the AWS Management Console. As an alternative, we may utilize the AWS Serverless Application Model (SAM), which was recently upgraded to include container image support.

On the Lambda console, click to create a function, select the container image, name the function, and then browse images to find the appropriate image in the ECR repository.

Container11.jpg

Select the latest image

Container2.jpg

 Let's add the API Gateway as trigger.


Container33.jpg

Configure the API Gateway trigger

Container44.jpg

It will provide us API Gateway url which we can use to trigger the function. 

Container5.jpg

Conclusion

The AWS Lambda Container Support expands the usage of AWS Lambda and Serverless significantly. It resolves a number of current issues and expands our options for deploying serverless apps.

 

Our Differentiator - Infosys Modernization Suite

Infosys Modernization Suite, part of Infosys Cobalt, is a flagship solution that simplifies and accelerates the modernization journey of an enterprise through hyper-automation across the application lifecycle.

Infosys Cloud Modernization Platform (part of Infosys Modernization Suite) helps in accelerating the serverless journey through its ready to use accelerators & best practices that have been leveraged for faster development. Accelerators include startup implementation for common serverless use cases and custom pipeline for CI/CD deployment. 

June 4, 2021

AWS ECS Anywhere: Run Container Workloads in Hybrid Environments

In an endeavor to deliver the best client experience, enterprises must build and modernize applications faster and more efficiently than ever before. Going cloud-native - i.e. loosely coupled and highly scalable applications, built and deployed in containers, on highly automated platforms - is a prevalent trend for application development. Cloud-native architecture enables enterprises to both build and deploy modern applications across a wide range of public cloud and on-premises data center platforms.

Amazon ECS Anywhere, a new capability for Amazon ECS that enables enterprises to run and manage native Amazon ECS tasks in their own environment. Using ECS Anywhere, enterprises can deploy applications in hybrid environments by taking advantage of container elasticity, security, and reliability in their on-premises environment with the ease of cluster management from a single pane of glass.

Amazon ECS is a fully managed container orchestration service that allows enterprises to easily run and manage Docker-enabled applications on EC2/Fargate. Amazon ECS supports workloads that take advantage of Local Zones, Wavelength Zones and AWS Outposts when low latency or local data processing requirements are needed.

Amazon ECS Anywhere (ECS-A) provides a simple installation process for workloads that need to run on physical servers or virtual machines in a non-AWS environment:

·         Install the AWS SSM agent in physical or virtual machine to make it a managed instance.

·         Install the AWS ECS agent on the managed instance we created in the previous step. This allows the end user to then register the managed instance with an existing ECS cluster.

·         To run tasks on on-premises, a new launch type External will be available within the ECS control plane.

Key architectural tenets:

  • The ECS control plane: By using the same control plane for both on-premises and cloud-native applications, we can better manage our hybrid footprint. No customer data is sent to the ECS control plane, only the information needed to manage tasks like container health & state will be sent from the on-premises environment to the ECS control plane.
  • Infrastructure agnostic: ECS Anywhere (ECS-A) is compatible with the following operating systems

- Ubuntu (16/18/20)

- RHEL/CentOS (7/8)

- Fedora (32/33)

- Debian and openSUSE

This also includes operating systems that run on bare metal, self-managed virtual machines (VM's) and even a Raspberry Pi. In the future, Windows OS could also be included.

  • Tolerates disconnections: ECS Anywhere tolerates disconnections and unreliable networks. During disconnection, tasks will continue to run in on-premises infrastructure. Upon reconnection control plane will re-evaluate instance state and pending tasks will be placed and started.

Amazon ECS Anywhere Benefits

Using ECS Anywhere, we can manage containers in on-premises with the same ease as we manage containers in the cloud. Below are the few additional benefits for the same.        

  •  Accelerates application migrations:  ECS Anywhere significantly accelerates workload migration by allowing developers to spend more time on business logic rather than deployment or configuration activities.
  • Scalability: ECS Anywhere can help customers to burst their on-premise workloads to AWS cloud for added capacity and performance.
  • Secure outbound connectivity: We can connect AWS API service endpoints from on-prem environments using ` VPC Endpoint via AWS Direct Connect/VPN for private API communication. All data exchanged between on-premises and AWS environment will be protected using industry-standard TLS encryption.
  • Automated Application Deployment for Hybrid environment: AWS CodePipeline is a fully managed continuous delivery service that helps us to automate release pipelines for fast and reliable application deployment. CodePipeline can be used to deploy applications running in hybrid environments.
  • Logging: Logging and monitoring of the ECS solution with ECS-Anywhere becomes very important to maintain reliability, availability, and performance. Using ECS Anywhere, we can push aggregate containers logs and metrics to Amazon CloudWatch. We can also create a notification solution using Amazon SNS Amazon services SES services on top of CloudWatch logs and metrics to support operational teams/processes.

Use Cases

We can use Amazon ECS Anywhere in multiple enterprise use cases.

  • Run applications on-premises for compliance reasons: The best fit use cases are where enterprises can't move their whole workload to the cloud due to compliance, regulatory, or data residency standards. Amazon ECS Anywhere is an ideal solution here because it allows enterprises to retain sensitive data in an on-premises environment while simultaneously leveraging all the benefits of using Amazon ECS.
  • Modernization before moving to cloud: ECS Anywhere (ECS-A) now provides a management layer on-premises for container orchestration. Enterprises can now containerize their workloads on-premises first, make them portable, resolve on-premises dependencies and get AWS-ready, followed by just updating the ECS services configuration from on-premises hardware.
  • Edge computing: ECS-Anywhere (ECS-A) provides the ability to support edge computing with, enterprises can significantly reduce latency, increase bandwidth, improve reliability and reduce costs. Enterprises can use ECS Anywhere to orchestrate containers at multiple edge locations for workloads like gathering raw data from machines, or raw images from drones and transform them before sending to cloud.
  • Burst to the cloud for added capacity and performance: Application demand can be unpredictable. An application might run efficiently in its on-premise environment today, but may require additional compute power tomorrow. Amazon ECS Anywhere is an ideal solution here as it allows the service to scale up and down based on demand.
Illustrative Solution for e-commerce built using Amazon ECS Anywhere
We designed an illustrative hybrid cloud solution using Amazon ECS Anywhere for an e-commerce application. In this solution, we are running existing compliance workloads such as payment and order API which also include a database in on-premises environment while other non-compliance workload like the web applications and search application running in the AWS Cloud.

ECS-A-1.jpg

Similar solution can be built for other industries such as Healthcare, Financial services, Manufacturing and many more where enterprises can run existing compliance workloads including databases in on-premises environment while other non-compliance workloads in AWS Cloud.

 

ECS Anywhere based hybrid solution can also be used where enterprise has multiple data centers.


ECS-A-2.jpg

INFOSYS DIFFERENTIATORS FOR APPLICATION MODERNIZATION

Infosys Modernization Suite, part of Infosys Cobalt, is a flagship solution that simplifies and accelerates the modernization journey of an enterprise through hyper-automation across the application lifecycle. It supports over 15 application modernization patterns through highly automated workflows. The integrated platform modernizes key areas such as the re-architecture of monolith to microservices, migration, containerization of workloads using Amazon ECS, EKS, and serverless adoption.

Infosys Cloud Modernization Platform (part of Infosys Modernization Suite) helps in accelerating the cloud adoption journey, it assists in every life cycle phase of modernization.

In discovery phase it supports data discovery of application & infrastructure inventory data. In assessment phase it helps in determining the application migration strategy to cloud and firm up cloud disposition to CSP of choice. Application disposition is based on the 7Rs migration strategies. Its discrepancy finder scans and reports instances for cloud readiness and goes a step further to auto-remediate certain cases resulting in significant effort and time savings before migration phase. It creates all AWS Compute services like Amazon EC2, AWS Elastic Beanstalk, Amazon ECS, Amazon EKS & AWS Lambda including other managed services like Amazon RDS, Amazon DynamoDB, Amazon S3, Amazon SQS, Amazon ElastiCache and many more using Infrastructure-as-Code for reusability and extensibility. Its automated DevSecOps pipelines provide simplified deployment of applications to various compute services.


ECS-A-3.jpg


Conclusion:

Amazon ECS Anywhere allows enterprises to run their workloads seamlessly in hybrid environments, providing options for use cases like modernizing workloads using AWS services on-premise before migrating to cloud or creating new solutions for edge. AWS' software-based approach for managing hybrid environments along with the Infosys Modernization Suite can help enterprises migrate, modernize and manage their workloads on hybrid cloud.

Reference:

https://aws.amazon.com/blogs/containers/introducing-amazon-ecs-anywhere/

Joint blog by Nirmal Singh Tomar, Principal Consultant Infosys & Vikrama Adethyaa, Sr. Partner Solutions Architect AWS