Experience

Work

2024-present

Improve and create automated methods to evaluate the trustworthiness and ethical aspects of large language models. Conducted quality control on red-teaming prompts from academic research; evaluated and improved on methods for detecting undesirable model responses.
Contribute to and maintain codebase containing large quantities of red-teaming prompt objects and detectors of undesirable model responses.
Design evaluation harnesses for both safety and performance aspects of LLMs, using latest academic research as a guide.

2020 - 2024

Do research on ethical and explainable machine learning, including internal user studies and peer-reviewed work on methodological issues.
Lead evaluation efforts for large language models. Work with business stakeholders to create reliable human labels, help create synthetic data for human evaluation, experiment with hallucination detection methods, and investigate which automatic methods of evaluation adequately capture what stakeholders want.
Research and implement technical explainability solutions (SHAP, LIME, GAMs, gradient-based methods) for black-box machine learning models, customizing dashboards and visualizations to user needs.
Published multiple papers on ethical values in machine learning at venues like FAccT, winning best paper award at FAccT 2021. Served multiple times on FAccT program committee. Currently serving as area chair for FAccT 2024.
Educate internal and external audiences on best practices for responsible AI through both written materials and talks.

2019 - 2020

Write tutorials, conceptual guides, and other documentation for a Kubernetes machine learning platform.
Write playbook for data scientists providing guidance on best practices for responsible AI.

2017 - 2019

Create, manage, and edit diverse types of documentation, including user guides, configuration guides for systems administrators, and API tutorials for third-party developers.
Wrote text for user interfaces and provided recommendations on improving the user experience for these interfaces.
Research software features and user needs by interviewing product managers, support, developers, and sales engineers.
Promoted to Senior Technical Writer after 1.5 years for proposing and leading initiatives to improve user experience of online documentation.

Part-time, 2018, then again in 2022

(in 2022 for Twilio) Audit and edit existing REST API tutorials (in Python, Ruby, JavaScript, and Java) for accuracy, consistency, clarity, and relevance to customer needs.Test code samples in multiple programming languages and update tutorials to correct mistakes or improve readability.
(in 2018 for MessageBird) Write sample Python web applications and tutorials for MessageBird’s REST API. Click here and here for samples.

Part-time, 2017 - 2019

Edit academic journal manuscripts for good English. Disciplines covered include physics, mathematics, and machine learning.

2015 - 2017

Designed writing courses for college freshmen, creating writing exercises of increasing difficulty and providing constructive feedback.
Worked on data science project that predicted restaurant’s ethnic categorization based on their non-food characteristics. Code and analysis publicly available on GitHub.

2009 - 2015

Independently planned and carried out multi-year research project, culminating in PhD dissertation.
Designed and taught courses on statistical reasoning, ethics, and history of philosophy.
Published peer-reviewed research on mathematical modeling strategies in quantum field theory.

2007 - 2009

Used C++ and OpenCV to create a machine-learning algorithm for tracking breast cancer cells in videos.

2005-2007

Wrote Fortran programs to analyze data from a high energy physics experiment, estimating the probability of a rare particle decay.
Won award for physics research.

2009 - 2015

Conceived of, planned, and managed complex research project on mathematical modeling techniques in quantum field theory.

2004 - 2007

FAccT: Area Chair, Reviewer

NeurIPS: Reviewer, Ethics Reviewer

ICML: Ethics Reviewer