AWS | Databricks Platform | EKS | Terraform | Team Lead
Wipro
Led a seven-engineer team building the AWS platform behind Alight Solutions: a Databricks lakehouse, an event-driven analytics pipeline, and an EKS delivery platform, all provisioned in Terraform and secured end to end.
What I owned
- The AWS Databricks platform: customer-managed VPC, private-subnet clusters with Secure Cluster Connectivity, S3 gateway and STS interface endpoints, cross-account control-plane IAM, Unity Catalog external locations, and JSON cluster policies for auto-termination, instance limits, autoscaling caps, and cost tagging
- An event-driven analytics pipeline from AppFlow, a Glue crawler, and Step Functions through a Python Lambda ELT into Redshift, validated with Athena and cached in Redis for Tableau
What changed
- Traced a climbing AWS bill to a Lambda re-reading the same S3 object in a loop on every run, fixed it, then right-sized instances and tuned autoscaling for about $70,000 a year in savings
- Gave the data team a governed lakehouse and gave analysts trusted Redshift and Tableau datasets to build on