Experience

Work

Applied Scientist, Vijil

2024-present

  • Improve and create automated methods to evaluate the trustworthiness and ethical aspects of large language models. Conducted quality control on red-teaming prompts from academic research; evaluated and improved on methods for detecting undesirable model responses.
  • Contribute to codebase containing large quantities of red-teaming prompt objects and detectors of undesirable model responses.
  • Design evaluation harnesses for both safety and performance aspects of LLMs, using latest academic research as a guide.

Senior Data Scientist, Capital One

2020 - 2024

  • Do research on ethical and explainable machine learning, including internal user studies and peer-reviewed work on methodological issues.
  • Lead evaluation efforts for large language models. Work with business stakeholders to create reliable human labels, help create synthetic data for human evaluation, experiment with hallucination detection methods, and investigate which automatic methods of evaluation adequately capture what stakeholders want.
  • Research and implement technical explainability solutions (SHAP, LIME, GAMs, gradient-based methods) for black-box machine learning models, customizing dashboards and visualizations to user needs.
  • Published multiple papers on ethical values in machine learning at venues like FAccT, winning best paper award at FAccT 2021. Served multiple times on FAccT program committee. Currently serving as area chair for FAccT 2024.
  • Educate internal and external audiences on best practices for responsible AI through both written materials and talks.

Senior Technical Writer, Capital One

2019 - 2020

  • Write tutorials, conceptual guides, and other documentation for a Kubernetes machine learning platform.
  • Write playbook for data scientists providing guidance on best practices for responsible AI.

Senior Technical Writer, Laserfiche

2017 - 2019

  • Create, manage, and edit diverse types of documentation, including user guides, configuration guides for systems administrators, and API tutorials for third-party developers.
  • Wrote text for user interfaces and provided recommendations on improving the user experience for these interfaces.
  • Research software features and user needs by interviewing product managers, support, developers, and sales engineers.
  • Promoted to Senior Technical Writer after 1.5 years for proposing and leading initiatives to improve user experience of online documentation.

Freelance API Technical Writer

Part-time, 2018, then again in 2022

  • (in 2022 for Twilio) Audit and edit existing REST API tutorials (in Python, Ruby, JavaScript, and Java) for accuracy, consistency, clarity, and relevance to customer needs.Test code samples in multiple programming languages and update tutorials to correct mistakes or improve readability.
  • (in 2018 for MessageBird) Write sample Python web applications and tutorials for MessageBird’s REST API. Click here and here for samples.

Editor, Research Square

Part-time, 2017 - 2019

  • Edit academic journal manuscripts for good English. Disciplines covered include physics, mathematics, and machine learning.

Postdoctoral Researcher/Teacher, University of Southern California

2015 - 2017

  • Designed writing courses for college freshmen, creating writing exercises of increasing difficulty and providing constructive feedback.
  • Worked on data science project that predicted restaurant’s ethnic categorization based on their non-food characteristics. Code and analysis publicly available on GitHub.

Researcher and Teacher, University of Pittsburgh

2009 - 2015

  • Independently planned and carried out multi-year research project, culminating in PhD dissertation.
  • Designed and taught courses on statistical reasoning, ethics, and history of philosophy.
  • Published peer-reviewed research on mathematical modeling strategies in quantum field theory.

Research Officer, Bioinformatics Institute (Singapore)

2007 - 2009

  • Used C++ and OpenCV to create a machine-learning algorithm for tracking breast cancer cells in videos.

Research Assistant, University of Chicago

2005-2007

  • Wrote Fortran programs to analyze data from a high energy physics experiment, estimating the probability of a rare particle decay.
  • Won award for physics research.

Education

PhD, History and Philosophy of Science, University of Pittsburgh

2009 - 2015

  • Conceived of, planned, and managed complex research project on mathematical modeling techniques in quantum field theory.

BA, Physics, University of Chicago

2004 - 2007

  • Won prize for research in physics and two popular science writing prizes.

Skills

Machine learning: Tree ensemble models, explainability methods (SHAP, ICE, etc.), fairness definitions

Programming: Python, Kubernetes, Java, C#

Others: Technical communication, user research, project management