ASAysha Shafiq
AI SECURITY RESEARCH · NSF-FUNDED

AI-Generated Code Reliability

I built a harness for AI coding agents to reimplement 10 Linux utilities, then built the compiler and fuzzing systems needed to measure whether the results were actually reliable. A Go-based tool traces C source into the linked binary, while the legacy fuzzer I rebuilt runs concurrently, resumes from checkpoints, and makes testing faster.

10 utilities · 98% of upstream tests passed
Go-based tool
Go fuzzer · concurrent + resumable
10× faster evaluation across 24+ implementations

The question behind the code

A convincing implementation is not necessarily a reliable one. The research needed a way to separate fluency from correctness, especially for systems programs whose failures hide in edge cases, memory bugs, and process behavior that tests alone don't catch.

I used the compiler as a measurement tool

Not every line of C source makes it into the final executable. I built a Go-based tool that follows code through compilation and linking to measure what actually survives. It streams Clang's AST without loading it all into memory, maps definitions to DWARF source ranges, and uses GNU ld's garbage-collection data to identify discarded sections. This produces a more accurate LOC estimate by counting the source that contributes to the final binary rather than every line in the source tree.

I rebuilt the fuzzer itself

I rebuilt a legacy fuzzer as a concurrent, resumable, checkpointed Go tool for faster testing. Bounded workers run implementations in parallel, hang detection stops stalled cases, and campaign checkpoints let multi-day evaluations continue after interruptions instead of starting over.

I made the development process repeatable

I designed a multi-agent workflow that starts from documentation, records decisions, and turns recurring failure patterns into reusable guidance. The workflow produced 10 C reimplementations that passed 98% of their upstream tests across utilities as different as make, grep, dash, and less.

CS 220 AI Learning InfrastructureAwaaz-e-SehatMeta Production Engineering Fellowship