Project 1: A network by hand
VPC, subnets, NAT, endpoints and least-privilege roles.
| What you build | A VPC across two zones, public, app and data subnets, internet and NAT gateways, route tables, VPC endpoints, a security group chain, and least-privilege IAM roles. |
| What it proves | You understand every networking piece, because you placed each one yourself. |
QuickCart is about to leave its single server behind. Before any new server, load balancer or database can move in, it needs a network: one where customers can reach the front door, servers can reach the internet and AWS, and nothing outside can ever reach the database.
In this project you build that network by hand, one piece at a time, in the AWS console. No wizard fills anything in for you. At the end you’ll prove each piece works from inside the network, with real commands and real answers.
What you’ll have at the end
The finished network, drawn from what you’ll build. Choose a kind of traffic to see which route-table line it uses.
| Piece | How many | What it’s for |
|---|---|---|
VPC quickcart-vpc, 10.0.0.0/16 | 1 | The private network everything lives in |
| Subnets | 6: public, app and data, in each of two zones | Keeping each tier separate, in two buildings |
| Internet gateway | 1 | The VPC’s only door to the internet |
| NAT gateway, with an Elastic IP | 1, in zone a | Lets private servers reach out, with nothing getting in |
| Route tables | 4: public, app a, app b, data | Deciding which subnets are public, private or isolated |
| Security groups | 4: load balancer, app, database, endpoints | A chain in which each group trusts only the one before it |
| VPC endpoints | 1 gateway (S3) and 4 interface (Secrets Manager, SSM, SSM messages, EC2 messages) | Reaching AWS services without going through the internet |
| IAM roles | 2: one for app servers, one for flow logs | Each can do exactly its job and nothing else |
| Flow logs | 1 | A record of every connection accepted or rejected in the VPC |
Project 2 builds a self-healing app on top of exactly this network.
Before you start
Read these first. The project assumes you know what each piece is; here you put them together.
| Lesson | What you need from it |
|---|---|
| What is a VPC? | VPCs, subnets and the one route that makes a subnet public |
| Subnets and IP ranges | Reading /16 and /24, and planning ranges |
| Routes, gateways and NAT | Internet gateways, NAT gateways and route tables |
| Security groups | Inbound and outbound rules, and naming another group as the source |
| Who can do what: IAM | Roles, policies and ARNs |
| Regions and zones | Why everything comes in twos |
You need:
- An AWS account, signed in to the console as a user who can create VPC, EC2, IAM, S3, Secrets Manager and CloudWatch resources. Set up your computer covers a safe login.
- A budget alert, so a forgotten NAT gateway can’t surprise you. Billing and Cost Management → Budgets → Create budget, a monthly cost budget of $20, with an email alert at 80%.
- About 3 to 4 hours. You can stop between stages, but the NAT gateway and endpoints cost money while they exist.
Pick the region first. At the top right of the console, choose US East (N. Virginia) us-east-1. Every resource in this project goes in that region. Check it at the start of each stage: a form that “can’t see” your VPC is almost always in the wrong region.
Your account ID appears in this project as 111122223333. Find yours in the account menu at the top right, and use it wherever you see that number.
Cost
Costs money — about $3.50 a day while the NAT gateway and endpoints exist. Delete them when you’re done.
| Resource | Rough price in us-east-1 | For this project |
|---|---|---|
| NAT gateway | about $0.045 an hour, plus $0.045 per GB it processes | about $1.10 a day |
| Public IPv4 address (the NAT’s Elastic IP) | about $0.005 an hour | about $0.12 a day |
| Interface endpoints | about $0.01 an hour per endpoint per zone: 4 endpoints × 2 zones | about $1.92 a day |
| S3 gateway endpoint | free | $0 |
Test server, t3.micro, with an 8 GB disk | about $0.0104 an hour, plus a few cents a day for the disk | about $0.30 a day, while it runs |
| Secrets Manager secret | about $0.40 a month, charged by the day | about 1 cent a day |
| S3 bucket with one small file | fractions of a cent | pennies |
| Flow logs in CloudWatch Logs | about $0.50 per GB stored; a quiet VPC writes very little | pennies |
| VPC, subnets, route tables, internet gateway, security groups, IAM roles | free | $0 |
Prices change and differ by region. Check the AWS pricing pages for current figures.
Stage 1: The VPC
The VPC is the range of addresses everything else will live in. It has one important setting, the range, and one setting that’s off by default and quietly breaks three things later.
Create the VPC
In the AWS console: VPCYour VPCsCreate VPC
Field Enter Resources to create VPC only. The VPC and more option builds everything at once, which is exactly what this project avoids Name tag quickcart-vpcIPv4 CIDR block IPv4 CIDR manual input IPv4 CIDR 10.0.0.0/16IPv6 CIDR block No IPv6 CIDR block Tenancy Default. Dedicated costs much more and is for special compliance needs Choose Create VPC.
A
/16gives you 65,536 addresses. You’ll use a few dozen. The spare room is free, and it means you never have to renumber.Turn on DNS hostnames
In the AWS console: VPCYour VPCsquickcart-vpcActionsEdit VPC settings
Field Enter Enable DNS resolution Ticked. It already is; leave it Enable DNS hostnames Ticked. It’s off by default for a VPC you create yourself Choose Save.
Without DNS hostnames, the interface endpoints in stage 7 can’t use private DNS. Your servers would keep sending AWS traffic out through the NAT gateway, and the endpoints would bill you for doing nothing. The database’s address in Project 2 wouldn’t resolve either.
Checkpoint
On Your VPCs, quickcart-vpc with State: Available, IPv4 CIDR: 10.0.0.0/16, and on its Details tab both DNS resolution and DNS hostnames say Enabled.
Stage 2: Six subnets
A subnet is a slice of the VPC that lives in exactly one availability zone. Surviving the loss of a zone means having a copy of each tier in a second zone, so three tiers become six subnets.
The numbering lets you read a subnet’s job from its address: 1 and 2 are public, 11 and 12 are app, 21 and 22 are data. The gaps leave room to grow.
Create all six on one form
In the AWS console: VPCSubnetsCreate subnet
For VPC ID, choose
quickcart-vpc. Fill in the first subnet, then choose Add new subnet for each of the others:Subnet name Availability Zone IPv4 subnet CIDR block quickcart-public-aUS East (N. Virginia) / us-east-1a 10.0.1.0/24quickcart-public-bUS East (N. Virginia) / us-east-1b 10.0.2.0/24quickcart-app-aUS East (N. Virginia) / us-east-1a 10.0.11.0/24quickcart-app-bUS East (N. Virginia) / us-east-1b 10.0.12.0/24quickcart-data-aUS East (N. Virginia) / us-east-1a 10.0.21.0/24quickcart-data-bUS East (N. Virginia) / us-east-1b 10.0.22.0/24Leave IPv4 VPC CIDR block as
10.0.0.0/16. Choose Create subnet.Nothing on this form makes a subnet public. The names are only labels for people. Stage 5’s route tables decide what each subnet really is.
Check each subnet is in the zone you meant
In the AWS console: VPCSubnets
Look at the Availability Zone column. Two subnets that should be in different zones but ended up in the same one is a silent mistake. Nothing complains until the zone failure test in Project 2.
Checkpoint
Six subnets in Subnets, filtered to quickcart-vpc. Three are in us-east-1a and three are in us-east-1b. Each /24 shows 251 available IPv4 addresses, because AWS keeps 5 addresses in every subnet for itself.
Stage 3: The internet gateway
The internet gateway is the VPC’s door to the internet. You create it and then attach it: two steps that people often only do half of.
Create it
In the AWS console: VPCInternet gatewaysCreate internet gateway
Field Enter Name tag quickcart-igwChoose Create internet gateway. Its state says Detached. It does nothing yet.
Attach it to the VPC
In the AWS console: VPCInternet gatewaysquickcart-igwActionsAttach to VPC
Field Enter Available VPCs quickcart-vpcChoose Attach internet gateway.
An internet gateway costs nothing, and AWS runs it across every zone, so it can’t be a single point of failure.
Checkpoint
quickcart-igw with State: Attached and VPC ID showing quickcart-vpc.
Stage 4: The NAT gateway
Servers in the app subnets must never be reachable from the internet, but they still need to reach out: to install packages when they boot, for example. A NAT gateway is that one-way door.
It goes in a public subnet, which feels backwards. It serves the private subnets, but it needs its own route to the internet gateway, and only a public subnet has one.
Create the NAT gateway, with an Elastic IP
In the AWS console: VPCNAT gatewaysCreate NAT gateway
Field Enter Name quickcart-nat-aSubnet quickcart-public-aConnectivity type Public Elastic IP allocation ID Choose Allocate Elastic IP. A new address is filled in If your console also asks for an Availability mode, choose Zonal. This project deliberately builds the classic NAT gateway, which lives in one zone, so you can see the zone dependency it creates.
Choose Create NAT gateway. It takes two or three minutes to become available.
Understand the trade-off you just made
One NAT gateway, in zone a, serves both zones. That halves its cost, but zone b’s servers now depend on zone a for their way out to the internet. If zone a fails, they lose it.
Stage 7 softens this: the endpoints give both zones a private path to the AWS services they need, without the NAT. The fully resilient design has one NAT gateway per zone, each app subnet using the one in its own zone, for about $1.10 more a day. Write this trade-off down now. You’ll see it for yourself in Project 2’s zone failure test.
Checkpoint
quickcart-nat-a with State: Available, Connectivity type: Public, and a Primary public IPv4 address. Write that address down. You’ll see it again in Prove it.
Stage 5: Route tables
This is the stage that decides which subnets are public. Before it, every subnet uses the VPC’s main route table, which has only the local route, so every subnet is isolated.
You’ll build four tables. Every table also gets 10.0.0.0/16 → local, which AWS adds and you can’t remove.
| Route table | Its one extra route | Subnets | So the subnets are |
|---|---|---|---|
quickcart-rt-public | 0.0.0.0/0 → quickcart-igw | public-a, public-b | Public |
quickcart-rt-app-a | 0.0.0.0/0 → quickcart-nat-a | app-a | Private: out only |
quickcart-rt-app-b | 0.0.0.0/0 → quickcart-nat-a | app-b | Private: out only, through zone a |
quickcart-rt-data | none | data-a, data-b | Isolated: no way in or out |
The two app tables say the same thing today. Keeping them separate makes the cross-zone dependency visible, and adding a second NAT gateway later becomes a one-line change.
The public route table
In the AWS console: VPCRoute tablesCreate route table
Field Enter Name quickcart-rt-publicVPC quickcart-vpcChoose Create route table. On its Routes tab, choose Edit routes, then Add route:
Destination Target 0.0.0.0/0Internet Gateway, then quickcart-igwChoose Save changes. On the Subnet associations tab, choose Edit subnet associations, tick
quickcart-public-aandquickcart-public-b, and choose Save associations.The two app route tables
Create two more route tables in the same way, in
quickcart-vpc:Name Route to add Associate with quickcart-rt-app-a0.0.0.0/0→ NAT Gateway →quickcart-nat-aquickcart-app-aquickcart-rt-app-b0.0.0.0/0→ NAT Gateway →quickcart-nat-aquickcart-app-bThe data route table, with no routes added
Field Enter Name quickcart-rt-dataVPC quickcart-vpcAdd no routes. Associate it with
quickcart-data-aandquickcart-data-b.An empty table is a security control. The database has no reason to reach the internet. With no route out, even a compromised database couldn’t send data anywhere or download anything.
Check the main route table is harmless
In the AWS console: VPCRoute tables
Find the table in
quickcart-vpcthat shows Main: Yes. It should have only thelocalroute and no explicit subnet associations.Any subnet you forget to associate falls back to the main table. Keeping it local-only means a forgotten subnet has too little access, which you notice in minutes, rather than too much, which you might never notice.
Checkpoint
On Subnets, open each subnet’s Route table tab. public-a and public-b show quickcart-rt-public; app-a shows quickcart-rt-app-a; app-b shows quickcart-rt-app-b; data-a and data-b show quickcart-rt-data. None shows the main table.
Stage 6: Security groups
A route table decides whether traffic can arrive. A security group decides whether it’s allowed when it does. You’ll build four, as a chain: each one names the group before it as its source, never a range of addresses.
| Group | Inbound | From | Why |
|---|---|---|---|
quickcart-alb-sg | HTTPS 443, HTTP 80 | 0.0.0.0/0, anywhere | The load balancer is the only thing customers talk to. Port 80 is only there to redirect to HTTPS |
quickcart-app-sg | HTTP 80 | quickcart-alb-sg | Only the load balancer can reach the app servers |
quickcart-db-sg | MySQL 3306 | quickcart-app-sg | Only app servers can reach the database |
quickcart-endpoints-sg | HTTPS 443 | quickcart-app-sg | Only app servers can use the interface endpoints |
Naming a group means “anything wearing that group”. When a server is replaced and gets a new address, nothing needs updating.
The load balancer's group
In the AWS console: VPCSecurity groupsCreate security group
Field Enter Security group name quickcart-alb-sgDescription Load balancer: HTTPS and HTTP from anywhereVPC quickcart-vpcUnder Inbound rules, choose Add rule twice:
Type Source HTTPS Anywhere-IPv4 ( 0.0.0.0/0)HTTP Anywhere-IPv4 ( 0.0.0.0/0)Leave Outbound rules as they are: All traffic to
0.0.0.0/0. Choose Create security group.The app servers' group
Field Enter Security group name quickcart-app-sgDescription App servers: HTTP from the load balancer onlyVPC quickcart-vpcOne inbound rule:
Type Source HTTP Custom, then start typing quickcart-alb-sgand choose the group, not an address rangeLeave the outbound rule in place: All traffic to
0.0.0.0/0. Security groups are stateful, so replies to incoming requests go back out on their own. But connections the server starts need an outbound rule, and a booting server does little else: it registers with Systems Manager, fetches its secret and downloads packages. Choose Create security group.There’s no SSH rule, on purpose. You’ll reach servers through Session Manager, which needs no open inbound port at all.
The database's group
Field Enter Security group name quickcart-db-sgDescription Database: MySQL from app servers onlyVPC quickcart-vpcType Source MYSQL/Aurora (port 3306) Custom → quickcart-app-sgChoose Create security group.
The endpoints' group
Field Enter Security group name quickcart-endpoints-sgDescription Interface endpoints: HTTPS from app servers onlyVPC quickcart-vpcType Source HTTPS Custom → quickcart-app-sgChoose Create security group.
Checkpoint
Open quickcart-db-sg → Inbound rules. The source shows a group ID followed by / quickcart-app-sg, not an address like 10.0.11.0/24. The same goes for quickcart-app-sg (source quickcart-alb-sg) and quickcart-endpoints-sg (source quickcart-app-sg). No group except quickcart-alb-sg allows anything from 0.0.0.0/0.
Stage 7: VPC endpoints
App servers need S3, Secrets Manager and Systems Manager. Without endpoints, those requests go out through the NAT gateway to AWS’s public addresses. Endpoints keep them inside AWS’s network, and keep them working if the NAT gateway or its zone fails.
| Kind | How it works | Cost |
|---|---|---|
| Gateway endpoint (S3) | Adds a route to your route tables. S3’s addresses go to the endpoint instead of the NAT gateway | Free |
| Interface endpoint (everything else) | Puts a network interface with a private address in your subnets, and makes the service’s normal name resolve to it | About $0.01 an hour per zone, each |
The S3 gateway endpoint
In the AWS console: VPCEndpointsCreate endpoint
Field Enter Name tag quickcart-ep-s3Type AWS services Services Search for s3, and choosecom.amazonaws.us-east-1.s3with Type: Gateway. There’s also an Interface one with the same name; not that oneVPC quickcart-vpcRoute tables Tick quickcart-rt-app-aandquickcart-rt-app-bPolicy Full access Choose Create endpoint.
Four interface endpoints
Create four more endpoints on the same form. Everything is the same except the name and the service:
Name tag Service (Type: Interface) What it’s for quickcart-ep-secretsmanagercom.amazonaws.us-east-1.secretsmanagerReading the database password quickcart-ep-ssmcom.amazonaws.us-east-1.ssmSystems Manager itself. Without it, Session Manager can’t start quickcart-ep-ssmmessagescom.amazonaws.us-east-1.ssmmessagesThe channel a Session Manager shell runs over. Without it, a session connects and then hangs quickcart-ep-ec2messagescom.amazonaws.us-east-1.ec2messagesThe agent’s command channel. Without it, the server may never show up as managed For each one:
Field Enter Type AWS services VPC quickcart-vpcAdditional settings → Enable DNS name Ticked. This is the whole point: see below DNS record IP type IPv4 Subnets us-east-1a → quickcart-app-a; us-east-1b →quickcart-app-bIP address type IPv4 Security groups quickcart-endpoints-sgonly. Untick the VPC’s default groupPolicy Full access Why the DNS setting matters. With it ticked, the normal name
secretsmanager.us-east-1.amazonaws.comresolves to the endpoint’s private address inside your subnet. Your code doesn’t change at all. With it unticked, the name still resolves to a public address, traffic still goes through the NAT gateway, and the endpoint sits there billing you.
Checkpoint
Endpoints lists five, all with Status: Available: one of Type: Gateway and four of Type: Interface. Open quickcart-rt-app-a → Routes: there’s a new route whose destination starts pl- (a prefix list, AWS’s managed list of S3’s addresses) with a target starting vpce-. Open any interface endpoint → Subnets tab: one private address in 10.0.11.x and one in 10.0.12.x.
Stage 8: A bucket and a secret to test with
To prove the endpoints and the roles work, you need something to read. Project 2 uses the same bucket for its app files.
Create the bucket
In the AWS console: S3General purpose bucketsCreate bucket
Field Enter Bucket type General purpose Bucket name quickcart-assets-followed by your account ID, likequickcart-assets-111122223333. Names are shared across every AWS account, so the number keeps yours uniqueObject Ownership ACLs disabled (the default) Block all public access On (the default). Leave all four boxes ticked Bucket Versioning Enable Default encryption SSE-S3 (the default) Choose Create bucket.
Upload a small test file
On your computer, create a text file called
hello.txtcontaining one line:hello from quickcart. Then:In the AWS console: S3quickcart-assets-111122223333Upload
Choose Add files, pick
hello.txt, and choose Upload.Store a test secret
In the AWS console: Secrets ManagerSecretsStore a new secret
Field Enter Secret type Other type of secret Key/value pairs Key hello, valueworldEncryption key aws/secretsmanager (the default) Choose Next.
Field Enter Secret name quickcart/testDescription Test secret for the network projectChoose Next. Leave Automatic rotation off, choose Next, then Store.
Checkpoint
The bucket holds hello.txt, and its Permissions tab says Block all public access: On. Secrets Manager lists quickcart/test. Open it and copy its Secret ARN: it ends in quickcart/test- followed by six random characters. Those six characters matter in the next stage.
Stage 9: Least-privilege IAM roles
A server that calls AWS needs permission. Access keys on the server are how credentials leak. Instead, the server wears an IAM role: AWS hands it short-lived credentials that change on their own, and nothing is stored on disk.
Least privilege means each role can do exactly its job. The app role can read one secret and one bucket, not every secret and every bucket.
Create the app servers' role
In the AWS console: IAMRolesCreate role
Field Enter Trusted entity type AWS service Service or use case EC2 Use case EC2 Choose Next. Search for and tick these two AWS managed policies:
Policy What it allows AmazonSSMManagedInstanceCoreRegistering with Systems Manager, so you can use Session Manager with no SSH CloudWatchAgentServerPolicySending logs and metrics to CloudWatch. Project 2 uses it Choose Next.
Field Enter Role name quickcart-app-roleDescription QuickCart app servers: SSM, CloudWatch, one secret, one bucketChoose Create role.
Add a policy for exactly one secret
In the AWS console: IAMRolesquickcart-app-roleAdd permissionsCreate inline policy
Choose JSON, delete what’s there, and paste this. Put your own account ID in place of the highlighted number:
quickcart-read-secret{ "Version": "2012-10-17", "Statement": [ { "Sid": "ReadTheTestSecret", "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:quickcart/test-*" } ] }Part What it means "Version": "2012-10-17"The policy language version. Always this date; it isn’t when you wrote it "Sid"A label for this statement, for people reading it "Effect": "Allow"This statement grants permission. Anything not allowed is denied "Action": "secretsmanager:GetSecretValue"Only reading a secret’s value. Not listing secrets, changing them or deleting them "Resource"Only this secret. The -*at the end matches the six random characters AWS adds to every secret’s ARN. Without it, the policy matches nothingChoose Next, name it
quickcart-read-secret, and choose Create policy.Add a policy for exactly one bucket
Create a second inline policy in the same way:
quickcart-read-assets{ "Version": "2012-10-17", "Statement": [ { "Sid": "ReadObjects", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::quickcart-assets-111122223333/*" }, { "Sid": "ListTheBucket", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::quickcart-assets-111122223333" } ] }Part What it means s3:GetObjecton…/*Read any object (file) in the bucket. Objects are addressed as bucket name, a slash, then the object’s key, hence /*s3:ListBucketon the bucket’s ARNList what’s in the bucket. This action acts on the bucket itself, so its ARN has no /*Name it
quickcart-read-assetsand choose Create policy. Mixing up the two ARNs is one of the most common IAM mistakes there is: see Troubleshooting.Create the flow logs' log group
Flow logs need somewhere to write. Create the log group first, so the role can be limited to exactly it.
In the AWS console: CloudWatchLog groupsCreate log group
Field Enter Log group name /quickcart/vpc-flow-logsRetention setting 1 week. Logs kept forever cost money forever Log class Standard Choose Create.
Create the flow logs' role
This role is used by the VPC Flow Logs service, not by a server, so it needs a custom trust policy.
In the AWS console: IAMRolesCreate role
For Trusted entity type, choose Custom trust policy, and paste:
Trust policy{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vpc-flow-logs.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "aws:SourceAccount": "111122223333" } } } ] }Part What it means "Principal": { "Service": "vpc-flow-logs.amazonaws.com" }Only the VPC Flow Logs service may use this role "Action": "sts:AssumeRole"“Use” means taking on the role’s permissions for a while "Condition"→"aws:SourceAccount"And only when it’s acting for your account. This stops another account’s flow logs being pointed at your role Choose Next, add no managed policies, choose Next, name it
quickcart-flow-logs-role, and choose Create role. Then add this inline policy to it, namedquickcart-write-flow-logs:quickcart-write-flow-logs{ "Version": "2012-10-17", "Statement": [ { "Sid": "WriteToOneLogGroup", "Effect": "Allow", "Action": [ "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": "arn:aws:logs:us-east-1:111122223333:log-group:/quickcart/vpc-flow-logs:*" }, { "Sid": "FindTheLogGroup", "Effect": "Allow", "Action": "logs:DescribeLogGroups", "Resource": "*" } ] }Part What it means logs:CreateLogStream,logs:PutLogEventsCreate a stream inside the group, and write records to it logs:DescribeLogStreamsSee which streams already exist Resource ending log-group:/quickcart/vpc-flow-logs:*Only this one log group, and the streams inside it logs:DescribeLogGroupson*List the names of log groups, to find this one. It can’t read or change any of them
Checkpoint
IAM → Roles lists quickcart-app-role with 2 managed and 2 inline policies, and quickcart-flow-logs-role with 1 inline policy. Neither has anything like AdministratorAccess or a * action.
Stage 10: Flow logs
Flow logs record every connection in the VPC: who talked to whom, on which port, and whether it was ACCEPTed or REJECTed. When something times out, they tell you whether a security group dropped it.
Turn on flow logs for the VPC
In the AWS console: VPCYour VPCsquickcart-vpcFlow logsCreate flow log
Field Enter Name quickcart-flow-logsFilter All. In production you might record only Reject, to save money Maximum aggregation interval 1 minute, so records appear quickly Destination Send to CloudWatch Logs Destination log group /quickcart/vpc-flow-logsService access / IAM role Use an existing service role, then quickcart-flow-logs-roleLog record format AWS default format Choose Create flow log.
Checkpoint
On the VPC’s Flow logs tab, quickcart-flow-logs with Status: Active. The log group fills up after the next section starts some traffic.
Prove it
Every piece is in place. Now prove it from inside the network, with a throwaway server in an app subnet: no public address, no SSH key, and only the app security group and app role.
Each check below tests one or more pieces. If one fails, Troubleshooting says which piece to look at.
Launch a test server in app subnet a
In the AWS console: EC2InstancesLaunch instances
Field Enter Name quickcart-testApplication and OS Images Amazon Linux 2023 AMI, 64-bit (x86) Instance type t3.microKey pair Proceed without a key pair Under Network settings, choose Edit:
Field Enter VPC quickcart-vpcSubnet quickcart-app-aAuto-assign public IP Disable Firewall (security groups) Select existing security group → quickcart-app-sgUnder Advanced details:
Field Enter IAM instance profile quickcart-app-roleMetadata version V2 only (token required) Choose Launch instance. Wait until Status check shows 3/3 checks passed, a few minutes.
Check 1: Session Manager connects
In the AWS console: EC2Instancesquickcart-testConnectSession ManagerConnect
You should seeA black terminal in your browser, with a prompt like
sh-5.2$. That alone proves several things: the app role reached the server, thessm,ssmmessagesandec2messagesendpoints work, the endpoints’ security group lets the app group in, and the app group’s outbound rule is there. No port was opened and no key was used.If the Connect button says the agent isn’t online, wait two more minutes. If it still isn’t, see Troubleshooting.
Check 2: the NAT gateway works
In the Session Manager terminal:
Terminalcurl -s checkip.amazonaws.comPart What it means curlfetch a web address and print what comes back -ssilent: no progress bar checkip.amazonaws.coman AWS page that replies with the public address your request came from You should see203.0.113.25The address should match the NAT gateway’s public address from stage 4, not anything belonging to the server, which has no public address. That proves the app route table sends traffic to the NAT, the NAT works, the public route table sends it to the internet gateway, and the gateway is attached.
Check 3: Secrets Manager's name resolves to a private address
Terminalgetent hosts secretsmanager.us-east-1.amazonaws.comPart What it means getent hostslook up the address a name resolves to, the same way programs on this server do secretsmanager.us-east-1.amazonaws.comthe normal, public name of Secrets Manager in this region You should see10.0.11.143 secretsmanager.us-east-1.amazonaws.comYour address differs, but it starts
10.0.11.: the endpoint’s interface in this subnet. That’s private DNS working.Check 4: read the secret through the endpoint
Terminalaws secretsmanager get-secret-value --secret-id quickcart/test --query SecretString --output text --region us-east-1Part What it means aws secretsmanager get-secret-valueask Secrets Manager for a secret’s value --secret-id quickcart/testwhich secret, by name --query SecretStringprint only the secret itself, not the other details --output textas plain text instead of JSON --region us-east-1the region the secret is in You should see{"hello":"world"}The server used the role’s short-lived credentials, reached Secrets Manager over the endpoint, and was allowed by the one-secret policy.
Check 5: read from S3 through the gateway endpoint
Terminalaws s3 cp s3://quickcart-assets-111122223333/hello.txt - --region us-east-1Part What it means aws s3 cpcopy a file to or from S3 s3://quickcart-assets-111122223333/hello.txtthe file you uploaded. Use your own bucket name -copy it to the screen instead of to a file --region us-east-1the bucket’s region You should seehello from quickcartCheck 6: least privilege says no
Now ask for something the role was never given: a list of every bucket in the account.
Terminalaws s3 ls --region us-east-1Part What it means aws s3 lswith no bucket named, list every bucket in the account --region us-east-1the region to ask You should seeAn error occurred (AccessDenied) when calling the ListBuckets operation: User: arn:aws:sts::111122223333:assumed-role/quickcart-app-role/i-0a1b2c3d is not authorized to perform: s3:ListAllMyBuckets because no identity-based policy allows the s3:ListAllMyBuckets actionThis error is a pass. The role can read one bucket, and can’t even see that others exist. Your instance ID in the message differs.
Check 7: flow logs are recording
In the AWS console: CloudWatchLog groups/quickcart/vpc-flow-logs
You should seeOne or more log streams, named after network interfaces (
eni-…). Open one: each line is one flow, ending inACCEPT OKorREJECT OK. Traffic to port 443 on the endpoints’ private addresses shows up asACCEPT.
Record your results. Everything should pass:
| Check | Proves | Result |
|---|---|---|
| 1. Session Manager connects | App role, three SSM endpoints, endpoints group, app group’s outbound rule | |
2. curl shows the NAT’s address | App route tables, NAT gateway, public route table, internet gateway | |
3. The name resolves to 10.0.11.x | DNS hostnames, private DNS on the interface endpoints | |
| 4. Secret read | Secrets Manager endpoint, secret policy | |
| 5. File read | S3 gateway endpoint, bucket policy | |
| 6. Listing all buckets refused | The role is least-privilege | |
| 7. Flow log records | Flow logs role and log group |
Nothing here tested the data subnets directly: nothing lives there yet. Their route table, with only local, and the database group, which only trusts quickcart-app-sg, are checked by reading them in stages 5 and 6. Project 2 puts a database there and proves the chain end to end.
Troubleshooting
These are real mistakes from building exactly this network.
Clean up
Going straight on to Project 2? Terminate the test server (step 1) and keep everything else: Project 2 builds on this network. If you’ll start it another day, do the whole clean-up now and rebuild later. The network costs about $3.50 a day even when idle.
Delete things in this order. Some can’t be deleted while something else still uses them.
Terminate the test server
In the AWS console: EC2Instancesquickcart-testInstance stateTerminate (delete) instance
Confirm with Terminate (delete). Its state goes to Terminated.
Delete the five endpoints
In the AWS console: VPCEndpoints
Tick all five
quickcart-ep-…endpoints, choose Actions → Delete VPC endpoints, typedelete, and choose Delete.Delete the NAT gateway, then release its address
In the AWS console: VPCNAT gatewaysquickcart-nat-aActionsDelete NAT gateway
Type
deleteand confirm. Wait until its state is Deleted, a minute or two. Then:In the AWS console: VPCElastic IPsthe addressActionsRelease Elastic IP addresses
Choose Release.
Delete the flow log, then the VPC
In the AWS console: VPCYour VPCsquickcart-vpcFlow logsquickcart-flow-logsActionsDelete flow logs
Then delete the VPC itself:
In the AWS console: VPCYour VPCsquickcart-vpcActionsDelete VPC
The console lists what goes with it: the six subnets, the route tables, the internet gateway and the security groups. Type
deleteand choose Delete.Empty and delete the bucket
In the AWS console: S3quickcart-assets-111122223333Empty
Type
permanently deleteand choose Empty. With versioning on, this also removes every old version. Then choose Delete on the bucket and type its name to confirm.Schedule the secret for deletion
In the AWS console: Secrets Managerquickcart/testActionsDelete secret
Set the waiting period to 7 days, the shortest the console allows, and choose Schedule deletion. You’re not charged for a secret that’s waiting to be deleted.
Delete the log group and the two roles
In the AWS console: CloudWatchLog groups/quickcart/vpc-flow-logsActionsDelete log group(s)
In the AWS console: IAMRoles
Tick
quickcart-app-roleandquickcart-flow-logs-role, choose Delete, typedeleteand confirm. Their inline policies go with them.Clean-up step: Confirm nothing is left billing
Check each of these:
Where Should show VPC → NAT gateways Nothing, or only quickcart-nat-aas DeletedVPC → Elastic IPs No addresses VPC → Endpoints No endpoints VPC → Your VPCs Only the account’s default VPC EC2 → Instances quickcart-testas Terminated, or goneResource Groups & Tag Editor → Tag Editor, region us-east-1, resource types All supported resource types, search No resources with a quickcartnameYou should seeAll six checks come back empty. Tomorrow, check Billing and Cost Management → Bills: the charges for this project stop on the day you cleaned up.