AWS Lambda Cold Starts

Everything you need to know about cold starts

When a Lambda function is invoked for the first time, or after it has been idle for a while, AWS creates a new execution environment.

This initialization process is what we call a “cold start.”

It involves:
↳ Loading the function’s code and dependencies.
↳ Setting up the runtime environment.
↳ Establishing network connections.

Many engineers have written about Lambda cold starts, presenting their advice on reducing it.

It’s easily one of the most talked-about and yet misunderstood topics.

Before the initialization, there are 2 more things that the lambda service does:-

Finding a space in its EC2 fleet to allocate the worker

Initializing the worker

It takes around 80-100ms and you have no control over this duration. It’s an area where AWS has optimized aggressively.

When optimizing cold start durations, the module initialization process is where you should focus on.

---

When to optimize cold starts:

Before going into optimizations, You should ask yourself a question,

Does it matter to you?

In a production environment with a relatively stable and constant workload, about 1% of all Lambda executions are cold starts (source).

If your functions are used for asynchronous or batch processing,
↳ 1-second delay is generally not a problem.

Consider the following use case:

↳ A user adds an item to the cart
↳ Lambda function experiences a cold start,
↳ Within a second or 2 Cart is updated.

Does it matter if the cart update is delayed by a second or two?

Probably not.

In this case, Cold starts do matter:-

↳ Direct User Interaction &
↳ Time-Sensitive Processing.

You need to optimize, When the execution time is directly experienced by the user.

An example can be a Lambda Function generating homepage content:

↳ The user visits your site,
↳ The Lambda Function starts executing,
↳ Only when the Lambda Function is done does the user receive a response.

If this takes 100 ms for warm starts but 1 second when no execution environment is available, 1% of visitors are going to have a bad experience.

In this scenario optimizing cold starts will be worth the effort.

You can proceed further If your function impacts user experience or processes a time-sensitive event.

If not, ask yourself if optimizing cold start is worth the development time.

---

Optimization Tips

Provisioned Concurrency:

Enabling Provisioned concurrency is the easiest way to reduce cold starts.

It allows you to have function instances that are always ready to respond immediately to invocations.

But it comes with a cost, you’ll be billed for every second you leave it enabled.

For effective use, concurrency levels must be estimated based on the application’s demand and usage patterns.

↳ Find out peak times
↳ Estimate the No. of instances needed for your workload.

Language Choice:

If cold starts and performance matter, choosing the most performant language is crucial.

In Node.js and Python, The code is converted to machine code on the fly.

So cold starts are faster than those on Java and C# which runs on Virtual Machine.

While Go and Rust are compiled languages they’re compiled for a specific CPU architecture (e.g. x86–64 or arm).

It needs no runtime translation. Nor does it need a VM or interpreter to start ↳ the code executes as is, as soon as it is called.

That’s why the cold start duration for compiled languages is negligible, measured in single-digit milliseconds.

So choose according to your requirements and business needs.

Use AOT-Compilation:

The ahead-of-time (AOT) compilation technique compiles function code into native binaries before deployment.

This eliminates the need for runtime compilation, allowing functions to start and execute more quickly.

AOT compilation is beneficial for languages such as Java or Python which are interpreted or compiled at runtime.

Implementing AOT compilation involves tools and frameworks supporting compiling code into optimized binaries.

For Instance,
↳ GraalVM for Java function

Keep The Functions Warm:

Cold start can be prevented by keeping Lambda functions warm invoking them periodically.

It ensures that there is always a ready instance to handle requests,
↳ reducing latency.

You can implement it using scheduled events,
↳ CloudWatch Events or AWS Step Functions.

Note:- Keeping the instance warm comes with additional costs as you’re invoking functions.

---

Things to Note

Adrian Tanasaz has tested the lambda function, “Covered In” and found out:-

Increasing memory does not reduce cold start duration.

Using a VPC setup does not affect cold start duration.

Changing the runtime version impacts cold start duration.

Using ARM architecture reduces cold Start duration for large-sized functions.

Enabling provisioned concurrency results in zero synchronous cold starts.

Using InlineCode does not reduce cold start duration.

Changing the AWS region can reduce cold start duration.

Using AWS Lambda layers reduces cold start duration for medium to large-sized functions.