CS 220 Learning Infrastructure

Course infrastructure has a sharp reliability requirement: one failed deployment, overloaded service, or broken grader can block hundreds of students at the same deadline. As Head Teaching Assistant and engineering lead, I owned the production path behind CS 220's AI tutor, autograder, analytics, and assignment releases—not just individual features.

• Supported more than 900 students across student-facing and staff-facing systems.
• Owned development, testing, deployment, monitoring, and incident response.
• Designed for deadline-driven bursts rather than misleading average traffic.
• Balanced student access, instructor controls, reliability, security, and API cost.

The original single-threaded Flask service could not absorb synchronized assignment traffic. I rebuilt it as an asynchronous FastAPI service, containerized it, and deployed it on Azure Kubernetes Service behind Traefik so requests could be handled concurrently and replicas could scale with demand.

• Scaled the tutoring service beyond 20,000 requests on peak days.
• Moved blocking request handling to an asynchronous FastAPI architecture.
• Used Docker, AKS, and Traefik for repeatable deployment, routing, and horizontal scaling.
• Kept the service responsive during assignment and exam-driven traffic spikes.

Scaling requests was only half the problem: every avoidable model call carried a real cost. I added controls at the gateway so legitimate students retained fast access while repeated work and unauthorized traffic stopped reaching the paid provider.

• Cached repeated responses in Redis instead of paying for identical model work.
• Applied per-student rate limits at the service boundary.
• Built enrollment-aware authentication around the paid API.
• Reduced LLM spending by 5× while serving the full course population.

The inherited Gradescope image executed untrusted student notebooks as root in the same environment as hidden tests and credentials authorized to write grades to Canvas. I built a non-destructive proof of concept, traced the full exposure path, and redesigned the grader around a real operating-system privilege boundary.

• Verified that submitted code could reach grader-owned files and use normal error output as an exfiltration channel.
• Separated the trusted grader and publisher from the untrusted notebook with distinct Unix users and processes.
• Protected answer keys and credentials with explicit ownership, permissions, a scrubbed environment, and narrow inter-process communication.
• Closed paths to hidden-test disclosure, score tampering, and replayable grade-write credentials.

Before instrumentation, diagnosing a student-facing failure meant reconstructing behavior from scattered symptoms. I added traces, service metrics, alerts, and course analytics so staff could move from a report to the responsible request or subsystem quickly.

• Instrumented service paths with OpenTelemetry tracing.
• Added Prometheus monitoring and alerting for operational health.
• Deployed Metabase dashboards for student activity, request volume, and usage patterns.
• Reduced average incident response time by 80%.

Course deadlines leave little room for maintenance windows. I moved releases into a GitLab CI/CD pipeline that verifies changes before a blue-green cutover, separating deployment from the moment students begin using the new version.

• Automated build and test gates before production rollout.
• Deployed changes to the inactive environment before switching traffic.
• Replaced manual release steps with a repeatable pipeline.
• Delivered zero-downtime upgrades for student-facing services.