Coursework heavily focused on systems and algorithms
With Sam Madden and Mike Stonebraker
I'm working with the Treasury team to build out infrastructure that lets us accurately track all fund flows within the Stripe universe.
I was the Chief Architect at PipelineDB. My work included architecting and implementing all of our core technologies. Most of the work I did is open-source and can be found on their GitHub. Some of the things we built at PipelineDB were:
- PipelineDB, a fork of PostgreSQL that runs SQL queries continuously on streaming data
pipeline_cluster, an extension for PipelineDB that adds high availability and horizontal scalability support
pipeline_kafka, an extension for PipelineDB that adds support for ingesting data from and producing data into Kafka
- Stride, a real-time analytics API powered by PipelineDB. The system is implemented in Go, is fully containerized and runs on Mesos
Locu got acquired by GoDaddy after which I started working there. Apart from the things I was already doing at Locu, most of the year was spent on migrating our infrastructure over to GoDaddy's private cloud. I was mostly involved in porting our users & payment management and analytics infrastructure to GD's stack.
Being an early engineer at Locu, I wore a lot of different hats. Some of the things I worked on were:
- Designed and implemented both v1 and v2 of Locu's Developer API. The technologies used included ElasticSearch, RabbitMQ, Celery and Redis
- Accounts and payment management using Stripe and later GoDaddy's checkout cart service
- Caching infrastructure for Locu's embeddable widgets, which are dynamically generated and so caching them was trickier than simply putting static data behind a CDN
- Served as Head of Metrics and was responsible for ensuring we were on top of user and infrastructure metrics. I led the effort to build an open-source analytics stack called Chronology
I worked in the Database Engineering team. During the internship, I made some improvements to HBase, including implementing async RPC support for their internal HBase fork. I hacked on HDFS and implemented file system snapshot support for it. I also made the prototype port of LinkBench from Java to C.
- Databases: PostgreSQL, Redis, Cassandra, ElasticSearch, MongoDB, HBase
- Infrastructure: Kafka, Docker, Mesos, Marathon, HAproxy, AWS