A Total Newbie Explores Serverless SaaS

A version of this post originally appeared on the Stackery blog. I wrote it while attending and covering AWS re:Invent LIVE in 2019 — that feels like a mighty long time ago!

For background, I was tasked with attending a handful of serverless-related talks and writing about what I learned from a newbie's perspective. This recap is all about serverless SaaS and a talk given by Todd Golding — Principal Cloud Architect, Global SaaS Tech Lead at Amazon Web Services (AWS). I hope it communicates even half of how much I learned about serverless and SaaS from his talk!

---

Have you ever have that dream where you’re in a class on Classical Tibetan Algebra? And you haven’t done any homework all semester? AND it’s the final? That was me in this session: I was WAY in over my head with this one.

Tod Golding’s material was so high-level that I got a nose bleed. Seriously. My nose started gushing 10 minutes in. This Portland dewdrop is NOT used to the dry desert and casino AC air. For anyone who attended ARC410, I’m sorry to have temporarily made the front row look like a crime scene. Something to remember for next year: 400-level sessions are no joke!

From the jump, Tod explained that we weren’t about to play pattycake here. No coverage of the basics today. But after two days of discussing serverless development and AWS tooling with the many awesome folks who have visited at the booth I was working, I was actually feeling pretty limber for the marathon that was “Serverless SaaS Deep Dive: Building Serverless on AWS”. I might have been wrong and pulled a muscle, but I’m better for it.

Tod is a Principal Cloud Architect and Global SaaS Tech Lead at AWS, so it’s no shocker that he wanted to get stratospheric here. He modestly described himself as “being involved in building and delivering solutions based on AWS” but I knew the truth: Tod is legit and my fingers were about to cramp up from typing too fast. But since re:Invent is really just a massive endurance experiment (in many senses,) I was game.

Serverless for SaaS

The main reason I chose this talk (and ignored the session level) was because of its title. Many folks talk about the value of developing AWS serverless quickly and with cost-efficiency, I was intrigued by a talk that focused on why serverless is compelling for SaaS in the first place.

Tod began by explaining the short version of why serverless is ideal for SaaS: agility, cost-optimization, operational efficiency, blast radius, and the ability to focus on IP. Serverless and the managed nature of it allows architects to write and build code above all else. Simple. And microservice-composition is much more interesting when you don’t have to worry about scaling.

In SaaS, Tod explained, you’re always trying to figure out how to be more agile. Once you introduce microservices, the kind of problems you’re chasing are simpler. This way, if something breaks, it’s much less catastrophic for your application overall.

“Ok, I think I’m good so far!”

Taking it up a notch

Just as soon as my confidence started to swell, things got real. Tod introduced a high-level sample architecture for SaaS with serverless: web application (in this case, React hosting in an S3 Bucket) → Amazon API Gateway as our entry point into the actual services of our app → and the actual application services; in this case, a Lambda Function.

Again, the benefit of using microservices is that they own the service they manage.

Onboarding and Identity

Next, Tod introduced a term I was unfamiliar with: tenants. In this case, a tenant is not some dude who time and again refuses to mop your precious hardwood floors with the right cleaning solution. Here, a tenant is a group of users that share a common access with specific privileges to a particular software instance.

He explained that tenant organization is critical: you don’t want to rewrite your entire app because you added other supporters as tenants in a disorganized fashioned. He suggested using user pools in AWS Cognito, within which a group of users will sit. So, in short: you can have many users connected to one tenant. This is a concept called “tenant isolation configuration”.

Applying tenant strategies with API Gateway

Next, Tod moved onto another sample architecture and one that is configured to avoid what he called the “noisy neighbor problem” – extending the risks-of-renting-out-your-condo metaphor… I like it.

Beyond some security concerns, the noisy neighbor issue is best exemplified when you separate your most valuable accounts onto a very high-performance instance and one of them starts using your service so much that it impacts performance for all your high-value accounts.

The architecture went something like this:

Tenant 1 and 2 → (JWT token) API Gateway → (tenant context, role, etc) Custom authorizer Lambda

This architecture allows you to put barriers in place to make it: A) A more richly secure system and B) Less problematic for your top-level usage tier users.

What I mean by this is that you never want a lower-level tier’s usage affecting that of your premium tier folks i.e. the aforementioned “noisy neighbor” problem. This is where usage plans come into place. You’ll need to add different API keys to improve the experience of your environment based on the status tier. I sort of prickle at calling groups of people "lower-level users" and placing them into "status tiers" but that's a me problem.

Provisioning tenants and isolation

This exact concept is one of the first things you’ll want to think about when deciding what’s in your microservice layout. There are really three categories to pick from and they all depend on the unique needs of your tenants/users:

Silo model: All resources are siloed for the tenant that wants an isolated experience. To achieve this, you can deploy, configure, and send by attaching an IAM execution role to that function. This is core to the isolation strategy that totally avoids the noisy neighbor problem.
Pool model: Resources are shared by all tenants. In this model, you are unable to deploy with a change at runtime; you have no choice but to deploy with a wider execution role.

The isolation story isn’t so awesome here when compared to the silo method and tenants will have some understanding of wanting a separate footprint from other tiers; that will be the hesitation you’ll get from those tenants. But the pool model is easier to manage and deploy, which makes it more agile. Architects should always try to push for this model for that reason.

One thought I had here is that, while a pool model will offer better performance for all users, it can’t offer guaranteed performance for your most important customers.

A combination of the two: You can also choose a set of services that are isolatable on a per-tier model. This is the absolute best approach for SaaS agility, says Tod.

Using concurrency as a tiering strategy

Once you’ve determined your ideal tenant strategy, you need to consider how exactly you’ll want to delineate various pricing tiers. Obviously, you really want to avoid having the basic tier folks hitting the function so hard that they run up against its limits and degrade the experience of premium users.

It’s pretty straightforward, but essentially you make this clear to your users and offer the next tier to circumvent the non-optimal experience.

Breaking up your system into microservices

Next, Tod addressed a question that I’ve admittedly looked up myself many times but it’s just one of those concepts I can’t seem to hammer in because I’m afraid to actually ask someone to explain it clearly for fear of looking like a total n00b. This sort of harkens back to that aforementioned “Algebra Final” nightmare.

The question is: “Isn’t a Lambda function a microservice itself? Is there any difference?”

There’s nothing like hearing that you aren’t the only one with a question that you think is obvious and would make you look vapid to ask. I was thrilled when Tod addressed it, unprompted.

His answer is that an AWS Lambda could be a microservice, but if you actually look at the scope of most microservices, there are considered as a collective whole (which is a little bit ironic based on the name). The reality is: microservices exist as a concept for collective work and a Lambda just happens to emit 5-8 functions in an application. Developers generally treat Lambdas as individual parts because they represent individual data sets.

Fundamentally, a Lambda on its own can’t have any persistence: it only has the logic you give it and can’t store data or any accumulated information. So while we could have a microservice that just does a math problem or gives one of three standard responses, the chances are we want our microservice to do a bit more. Ok, but how does this answer about Lambda functions vs microservices relate to multi-tenancy? Stay with me…

If you’re building a serverless SaaS application, you’ve created an application with many entry points, which again can saturate the service in some way for your premium tier and the only unit of scale is by default the entire service.

So, to avoid making your life miserable, it probably makes sense to let your Lambdas measure the scaling of your application for you. The beauty of serverless best practices reveals itself again!

Spare the developers

One serverless truism I'm certain of: you really want to hide a lot of the architectural details we’re talking about and take concerns away from your developers. Entirely.

Your developers shouldn’t be worrying about multi-tenant functions; they should be building, managing, and delivering awesome applications. Anything else in a SaaS model would NOT be a manageable or productive experience for anyone. That’s why we’ve selected serverless in the first place.

But how do you take all the concepts of multi-tenant functions and configure it so that they are moved to the edge (read: out of the view of your developers)?

To avoid writing In Search of Lost Time over here, I’m not going to explain everything that Tod proposed (dynamically generated policies, IAM policy templates, etc). Instead, I’ll leave you with a neat little answer called…

Lambda Layers

To quote my friends over at serverless monitoring platform, Epsagon, “A Lambda Layer is a ZIP archive that contains libraries. […] The main benefit of this is that you utilize your dependencies more efficiently, resulting in smaller deployment packages. And with smaller packages, you get faster deployments.”

In this case, we’re using Layers to help with logging and metrics for your developers, as we need a multi-tenant context in our logs. Again, developers shouldn’t need to think about anything besides logging so you should, in essence, hide the details of multi-tenancy.

This way you can put a shared construct there, reference it, and all the functions that reference the construct will get that updated version.

Then you can take the JWT token (this stands for JSON Web Token… token) that came in as a service and your token manager will deal with the details of the metric that came in. From there, it’s logged with the correct tenant context. Lambda Layers are then deployed and versioned separately.

When you introduce Layers into your environment, you use language directory conventions to import the layer dependency. This will look like any other dependency, but Layers requires you to use a specific convention for each dependency – in short: it knows what to do so you don’t have to.

The payoff is a simplified developer experience. The word tenant appears nowhere in this function and the code doesn’t look like it knows anything about tenant management!

Technical takeaways

• Serverless enables SaaS agility, resilience, and innovation
• Isolation strategies directly shape deployment footprint
• Consider supporting multiple isolation models
• Factor account limits into your isolation model
• Hide details of multi-tenancy from your service developers
• Consider using Lambda Layers as the home for shared multi-tenant concepts

What did I actually retain from this talk?

... Something that I’ve had to learn and re-learn throughout my entire career: “stupid questions” shouldn’t exist as a concept, especially in serverless, because many other developers (and non-developers like me) will often have the same questions as you do.

Moreover, serverless is the right choice for building SaaS applications for many reasons, not least of all that it spares your hardworking development team from building science projects around “multi-tenancy” instead of applications that work.

Finally, if you attend re:Invent live in-person in Las Vegas (whenever large events are permitted again,) bring Kleenex and Vaseline for those nosebleeds induced by dry Vegas air and 400-level sessions or suffer the social consequences.