ML Applied Scientist
Qualtrics
Bio
Originally a hardware engineer, I received my bachelor's and first master's degrees in computer engineering from Drexel University in Philadelphia, PA. I spent over 10 years working in industry primarily on wireless power semiconductors for consumer cellular and wearable electronics.
As my career progressed, I chose to transition to the rapidly evolving machine learning and artificial intelligence (AI) domains. I received my second master's degree in computer science from San José State University in 2016. I spent a year at UC Santa Cruz working on problems related to satisfiability and sampling of solutions to #P problems.
I did my Ph.D. at the University of Oregon, where my research focused on provably robust machine learning and understanding the causal relationship between training data and model predictions.
Professional Positions
-
Present 2023
-
2023 2018
Graduate Researcher
University of Oregon
-
2018 2017
Graduate Researcher
University of California, Santa Cruz, Baskin School of Engineering
-
2017 2011
Wireless Power Engineer
Integrated Device Technology Inc.
-
2011 2006
Applications Development Engineer
Teradyne Inc.
-
2006 2003
Research & Teaching Assistant
Drexel University, College of Engineering
Education
-
Ph.D. 2023
Computer Science
University of Oregon
-
M.S.2016
Computer Science
San José State University
-
B.S. & M.S.2006
Computer Engineering (Dual Degree)
Drexel University
Honors, Awards, and Grants
-
2022ICLR - Highlighted Reviewer
Awarded to the top 10% of ICLR reviewers. -
2022University of Oregon - Gurdeep Pall Graduate Student Fellowship
Awarded for outstanding graduate students in the University of Oregon's Department of Computer Science. Candidates are evaluated based on the overall quality of their academic work, their commitment to learning, and their potential for further academic achievement. -
2021University of Oregon - J. Donald Hubbard Family Scholarship
Awarded to a University of Oregon senior undergrad or a graduate student in Computer Science. Preference is given to students who show an interest in human-computer interaction, computer graphics, or multimedia. Secondary criteria are significant contributions of time and energy to the department through volunteer efforts. Academic ranking is also considered.
-
2019IJCAI - Travel Award
Financial award to present a paper at IJCAI 2019. -
2018SAT2018 Conference - Best Student Paper Award
Winner of the best student paper award at SAT2018 (part of FLoC'18) for our work "Fast Sampling of Perfectly Uniform Satisfying Assignments". -
2018FLoC - Travel Award
Travel award to present papers at the 2018 Federated Logic Conference (FLoC). -
2018SAT Association - Travel Award
Travel award to present papers at the 2018 SAT conference. -
2017 - 2018University of California Chancellor's Fellowship
Merit-based fellowship awarded to select first-year graduate students enrolled in doctoral programs at the University of California. -
2005Drexel University - Undergraduate Student Research Award
Awarded annually to one Drexel University undergraduate engineering student that is engaged in high-quality, self-directed research with significant promise for future post-graduate work. -
2005Drexel University - Arnold H. Kaplan Scholarship Award
Established by former Drexel University Prof. Arnold H. Kaplan, and his wife Deanne, in recognition of students that have exhibited outstanding scholastic achievement and possess excellent academic credentials.
-
2004Drexel University - Alvin W. Wene Engineering Scholarship
Awarded in memory of Alvin W. Wene to a Drexel University engineering student in recognition of scholarship and character.
-
2004, 2005Drexel University - Teaching Assistant Excellence Award
Drexel University award "presented to graduate students who serve as teaching assistants and who exhibit exemplary commitment to student learning, leadership and a commitment to professional growth as a teacher."
Evaluation criteria:- Contributions to Student Learning
- Reflective Teaching Practices
- Leadership/Potential Growth
- Outstanding Contributions and Innovation
Winner 2004. Honorable mention 2005.
Selected Publications
The following is a list of selected publications. Clicking on the publication will reveal the associated abstract. Clicking the "" button below any publication opens a link to download that specific document.
Filter by type:
Provable Robustness Against a Union of ℓ0 Attacks
Conference Paper
38th AAAI Conference on Artificial Intelligence (AAAI-24)
Vancouver, Canada, February 20-27, 2024.
(To appear)
[Code]
Abstract
Sparse or ℓ0 adversarial attacks arbitrarily perturb an unknown subset of the features. ℓ0 robustness analysis is particularly well-suited for heterogeneous (tabular) data where features have different types or scales. State-of-the-art ℓ0 certified defenses are based on randomized smoothing and apply to evasion attacks only. This paper proposes feature partition aggregation (FPA) -- a certified defense against the union of ℓ0 evasion, backdoor, and poisoning attacks. FPA generates its stronger robustness guarantees via an ensemble whose submodels are trained on disjoint feature sets. Compared to state-of-the-art ℓ0 defenses, FPA is up to 3,000× faster and provides larger median robustness guarantees (e.g., median certificates of 13 pixels over 10 for CIFAR10, 12 pixels over 10 for MNIST, 4 features over 1 for Weather, and 3 features over 1 for Ames), meaning FPA provides the additional dimensions of robustness essentially for free.
Training Data Influence Analysis and Estimation: A Survey
Abstract
Good models require good training data. For overparameterized deep models, the causal relationship between training data and model predictions is increasingly opaque and poorly understood. Influence analysis partially demystifies training's underlying interactions by quantifying the amount each training instance alters the final model. Measuring the training data's influence exactly can be provably hard in the worst case; this has led to the development and use of influence estimators, which only approximate the true influence. This paper provides the first comprehensive survey of training data influence analysis and estimation. We begin by formalizing the various, and in places orthogonal, definitions of training data influence. We then organize state-of-the-art influence analysis methods into a taxonomy; we describe each of these methods in detail and compare their underlying assumptions, asymptotic complexities, and overall strengths and weaknesses. Finally, we propose future research directions to make influence analysis more useful in practice as well as more theoretically and empirically sound. A curated, up-to-date list of resources related to influence analysis is available at https://github.com/ZaydH/influence_analysis_papers.
Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers
Conference Paper
Findings of the Association for Computational Linguistics (EMNLP 2023)
Singapore. Dec. 2023.
Abstract
Backdoor attacks manipulate model predictions by inserting innocuous triggers into training and test data. We focus on more realistic and more challenging clean-label attacks where the adversarial training examples are correctly labeled. Our attack, LLMBkd, leverages language models to automatically insert diverse style-based triggers into texts. We also propose a poison selection technique to improve the effectiveness of both LLMBkd as well as existing textual backdoor attacks. Lastly, we describe REACT, a baseline defense to mitigate backdoor attacks via antidote training examples. Our evaluations demonstrate LLMBkd's effectiveness and efficiency, where we consistently achieve high attack success rates across a wide range of styles with little effort and no model training.
Reducing Certified Regression to Certified Classification for General Poisoning Attacks
Abstract
Adversarial training instances can severely distort a model's behavior. This work investigates certified regression defenses, which provide guaranteed limits on how much a regressor's prediction may change under a poisoning attack and training outliers. Our key insight is that certified regression reduces to voting-based certified classification when using median as a model's primary decision function. Coupling our reduction with existing certified classifiers, we propose six new regressors provably-robust to poisoning attacks. To the extent of our knowledge, this is the first work that certifies the robustness of individual regression predictions without any assumptions about the data distribution and model architecture. We also show that the assumptions made by existing state-of-the-art certified classifiers are often overly pessimistic. We introduce a tighter analysis of model robustness, which in many cases results in significantly improved certified guarantees. Lastly, we empirically demonstrate our approaches' effectiveness on both regression and classification data, where the accuracy of up to 50% of test predictions can be guaranteed under 1% training set corruption and up to 30% of predictions under 4% corruption. Our source code is available at https://github.com/ZaydH/certified-regression.
Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees
Journal
Journal of Machine Learning Research, vol. 24, 2023.
[Code]
Abstract
Influence estimation analyzes how changes to the training data can lead to different model predictions; this analysis can help us better understand these predictions, the models making those predictions, and the data sets they're trained on. However, most influence-estimation techniques are designed for deep learning models with continuous parameters. Gradient-boosted decision trees (GBDTs) are a powerful and widely-used class of models; however, these models are black boxes with opaque decision-making processes. In the pursuit of better understanding GBDT predictions and generally improving these models, we adapt recent and popular influence-estimation methods designed for deep learning models to GBDTs. Specifically, we adapt representer-point methods and TracIn, denoting our new methods TREX and BoostIn, respectively; source code is available at https://github.com/jjbrophy47/tree_influence. We compare these methods to LeafInfluence and other baselines using 5 different evaluation measures on 22 real-world data sets with 4 popular GBDT implementations. These experiments give us a comprehensive overview of how different approaches to influence estimation work in GBDT models. We find BoostIn is an efficient influence-estimation method for GBDTs that performs equally well or better than existing work while being four orders of magnitude faster. Our evaluation also suggests the gold-standard approach of leave-one-out~(LOO) retraining consistently identifies the single-most influential training example but performs poorly at finding the most influential set of training examples for a given target prediction.
Feature Partition Aggregation: A Fast Certified Defense Against a Union of ℓ0 Attacks
Abstract
Sparse or ℓ0 adversarial attacks arbitrarily perturb an unknown subset of the features. ℓ0 robustness analysis is particularly well-suited for heterogeneous (tabular) data where features have different types or scales. State-of-the-art ℓ0 certified defenses are based on randomized smoothing and apply to evasion attacks only. This paper proposes feature partition aggregation (FPA) -- a certified defense against the union of ℓ0 evasion, backdoor, and poisoning attacks. FPA generates its stronger robustness guarantees via an ensemble whose submodels are trained on disjoint feature sets. Compared to state-of-the-art ℓ0 defenses, FPA is up to 3,000× faster and provides larger median robustness guarantees (e.g., median certificates of 13 pixels over 10 for CIFAR10, 12 pixels over 10 for MNIST, 4 features over 1 for Weather, and 3 features over 1 for Ames), meaning FPA provides the additional dimensions of robustness essentially for free.
Identifying a Training Set Attack's Target Using Renormalized Influence Estimation
Abstract
Targeted training-set attacks inject malicious instances into the training set to cause a trained model to mislabel one or more specific test instances. This work proposes the task of target identification, which determines whether a specific test instance is the target of a training-set attack. This can then be combined with adversarial-instance identification to find (and remove) the attack instances, mitigating the attack with minimal impact on other predictions. Rather than focusing on a single attack method or data modality, we build on influence estimation, which quantifies each training instance's contribution to a model's prediction. We show that existing influence estimators' poor practical performance often derives from their over-reliance on instances and iterations with large losses. Our renormalized influence estimators fix this weakness; they far outperform the original ones at identifying influential groups of training examples in both adversarial and non-adversarial settings, even finding up to 100% of adversarial training instances with no clean-data false positives. Target identification then simplifies to detecting test instances with anomalous influence values. We demonstrate our method's generality on backdoor and poisoning attacks across various data domains, including text, vision, and speech. Our source code is available at https://github.com/ZaydH/target_identification.
Simple, Attack-Agnostic Defense Against Targeted Training Set Attacks Using Cosine Similarity
Abstract
Targeted training set attacks inject adversarially perturbed instances into the training set to cause the trained model to behave aberrantly on specific test instances. As a defense, we propose to identify the most influential training instances (likely to be attacks) and the most influenced test instances (likely to be targets). Among prior influence estimation methods, TracIn shows the most promise but still performs poorly. We therefore propose a cosine similarity influence estimator, CosIn, which improves upon TracIn by focusing on gradient direction over magnitude. In experiments on vision, NLP, and speech domains, CosIn identifies up to 100% of adversarial instances in poisoning and backdoor training attacks. Our source code is available at https://github.com/ZaydH/cosin.
What Models Know About Their Attackers: Deriving Attacker Information From Latent Representations
Workshop Paper
Punta Cana, Dominican Republic, November 11, 2021.
[Video]
Abstract
Adversarial attacks curated against NLP models are increasingly becoming practical threats. Although various methods have been developed to detect adversarial attacks, securing learning-based NLP systems in practice would require more than identifying and evading perturbed instances. To address these issues, we propose a new set of adversary identification tasks, Attacker Attribute Classification via Textual Analysis (AACTA), that attempts to obtain more detailed information about the attackers from adversarial texts. Specifically, given a piece of adversarial text, we hope to accomplish tasks such as localizing perturbed tokens, identifying the attacker’s access level to the target model, determining the evasion mechanism imposed, and specifying the perturbation type employed by the attacking algorithm. Our contributions are as follows: we formalize the task of classifying attacker attributes, and create a benchmark on various target models from sentiment classification and abuse detection domains. We show that signals from BERT models and target models can be used to train classifiers that reveal the properties of the attacking algorithms. We demonstrate that adversarial attacks leave interpretable traces in both feature spaces of pre-trained language models and target models, making AACTA a promising direction towards more trustworthy NLP systems.
Learning from Positive & Unlabeled Data with Arbitrary Positive Shift
Abstract
Positive-unlabeled (PU) learning trains a binary classifier using only positive and unlabeled data. A common simplifying assumption is that the positive data is representative of the target positive class. This assumption rarely holds in practice due to temporal drift, domain shift, and/or adversarial manipulation. This paper shows that PU learning is possible even with arbitrarily non-representative positive data given unlabeled data from the source and target distributions. Our key insight is that only the negative class's distribution need be fixed. We integrate this into two statistically consistent methods to address arbitrary positive bias - one approach combines negative-unlabeled learning with unlabeled-unlabeled learning while the other uses a novel, recursive risk estimator. Experimental results demonstrate our methods' effectiveness across numerous real-world datasets and forms of positive bias, including disjoint positive class-conditional supports. Additionally, we propose a general, simplified approach to address PU risk estimation overfitting.
On the Practicality of Learning Models for Network Telemetry
Conference Paper
12th Conference on Network Traffic Measurement and Analysis (TMA'20),
Online, June 10-11, 2020.
[Video]
Abstract
Today’s data plane network telemetry systems enable network operators to capture
fine-grained data streams of many different network traffic features (e.g., loss or flow
arrival rate) at line rate. This capability facilitates data-driven approaches to network
management and motivates leveraging either statistical or machine learning models (e.g.,
for forecasting network data streams) for automating various network management tasks.
However, current studies on network automation-related problems are in general not concerned
with issues that arise when deploying these models in practice (e.g., (re)training overhead).
In this paper, we examine various training-related aspects that affect the accuracy and
overhead (and thus feasibility) of both LSTM and SARIMA, two popular types of models used
for forecasting real-world network data streams in telemetry systems. In particular, we
study the impact of the size, choice, and recency of the training data on accuracy and
overhead and explore using separate models for different segments of a data stream (e.g.,
per-hour models). Using two real-world data streams, we show that (i) per-hour LSTM models
exhibit high accuracy after training with only 24 hours of data, (ii) the accuracy of LSTM
models does not depend on the recency of the training data (i.e., no frequent (re)training
is required), (iii) SARIMA models can have comparable or lower accuracy than LSTM models,
and (iv) certain segments of the data streams are inherently more challenging to forecast
than others. While the specific findings reported in this paper depend on the considered data
streams and specified models, we argue that irrespective of the data streams at hand, a
similar examination of training-related aspects is needed before deploying any statistical
or machine learning model in practice.
Positive-Unlabeled Learning with Arbitrarily Non-Representative Labeled Data
Workshop Paper
ICML Workshop on Uncertainty in Deep Learning (UDL'20),
Online, July 17, 2020.
Abstract
Positive-unlabeled (PU) learning trains a binary classifier using only labeled-positive and unla- beled data. A common simplifying assumption is that the labeled data is representative of the target positive class, but this assumption rarely holds in practice. This papers show that PU learning is possible even with arbitrarily non-representative labeled-positive data. Our key insight is that only the negative class’s distribution need be fixed. We integrate this idea into two statistically consistent methods to address arbitrary positive bias – one approach combines negative-unlabeled learning with unlabeled-unlabeled learning while the other uses a novel, recursive risk estimator. Addition- ally, we propose a general, simplified approach to address PU risk estimation overfitting.
Fast Sampling of Perfectly Uniform Satisfying Assignments
Abstract
We present an algorithm for perfectly uniform sampling of satisfying assignments, based on the exact model counter sharpSAT and reservoir sampling. In experiments across several hundred formulas, our sampler is faster than the state of the art by 10 to over 100,000 times.
Clustering-Based, Fully Automated Mixed-Bag Jigsaw Puzzle Solving
Conference Paper
17th International Conference on the Computer Analysis of Images and Patterns, Ystad, Sweden, August 22-24, 2017,
(2):205‑217. [Slides]
Abstract
The jig swap puzzle is a variant of the traditional jigsaw puzzle, wherein all pieces are equal-sized squares that must be placed adjacent to one another to reconstruct an original, unknown image. This paper proposes an agglomerative hierarchical clustering-based solver that can simultaneously reconstruct multiple, mixed jig swap puzzles. Our solver requires no additional information beyond an unordered input bag of puzzle pieces, and it significantly outperforms the current state of the art in terms of both the reconstructed output quality as well the number of input puzzles it supports. In addition, we define the first quality metrics specifically tailored for multi-puzzle solvers, the Enhanced Direct Accuracy Score (EDAS), the Shiftable Enhanced Direct Accuracy Score (SEDAS), and the Enhanced Neighbor Accuracy Score (ENAS).
A Fully-Automated Solver for Multiple Square Jigsaw Puzzles Using Hierarchical Clustering
Master's Thesis San José State University - December 2016. Advisor: Chris Pollett.
Abstract
The square jigsaw puzzle is a variant of traditional jigsaw puzzles, wherein all pieces are equal-sized squares; these pieces must be placed adjacent to one another to reconstruct an original image. This thesis proposes an agglomerative hierarchical clustering based solver that can simultaneously reconstruct multiple square jigsaw puzzles. This solver requires no additional information beyond an input bag of puzzle pieces and significantly outperforms the current state of the art in terms of both the quality of the reconstructed outputs as well the number of input puzzles it supports. In addition, this thesis defines Enhanced Direct Accuracy Score (EDAS), Shiftable Enhanced Direct Accuracy Score (SEDAS), and Enhanced Neighbor Accuracy Score (ENAS), which are the first quality metrics specifically tailored for multi-puzzle solvers. This thesis also outlines the first standards for visualizing best buddies and the quality of solver solutions.
ForPowER: A Novel Architecture for Energy Efficient Implementation of Fork-Join Parallelism Using System on a Chip
Master's Thesis Drexel University - June 2006. Advisors: Moshe Kam & Nagarajan Kandasamy
Abstract
We describe ForPowER, a power-efficient architecture for handling fork-join parallelism using system on a chip.
Our design consists of 16 processor cores, capable of dynamically scaling their clock frequencies and supply
voltages under different workloads. The processors are divided into four sets of four, with each set sharing a
multiported two-level cache. This arrangement reduces the energy wasted on powering redundant data.
ForPowER also uses a central scheduler, which assigns tasks to the processors, taking advantage of the
shared memory and of the processors ability to scale their clock frequencies under varied workload.
We also describe power models for all components of the SoC design, namely the caches, processors, and the network.
We show that in simulation, ForPowER outperforms the most widely used fork-join architecture on the
SPEC-95 Hydro2D benchmark, saving over 65% more energy.
Professional Service
Reviewing
- Artificial Intelligence Journal (AIJ)
- NeurIPS: 2020, 2022, 2023
- ICLR: 2022, 2023, 2024
- ICML: 2023, 2024
- IJCAI: 2024
Teaching
Below is a selection of previous courses I have taught at the University of Oregon and Drexel University.
-->
Teaching History
-
22S, 21S
CIS315 - Introduction to Algorithms
University of Oregon
Dynamic Programming, Greedy, Divide & Conquer, Graphs
-
21W
CIS473 & CIS573 - Probabilistic Methods in Artificial Intelligence
University of Oregon
Probabilistic graphical models, Markov chain Monte Carlo, Belief Propagation
-
18F
CIS212 - Computer Science III
University of Oregon
C/C++, Unix, Bash
-
2003 to 2006
TDEC221 & TDEC222 - Teaching Assistant
Drexel University
Differential Equations, Transforms, and Fundamentals of Systems
-
2003 to 2006
TDEC231 & TDEC232 - Teaching Assistant
Drexel University
Evaluation and Presentation of Experimental Data and Ethics
Contact
I am always looking for opportunities to collaborate with other researchers; I am also happy to answer any questions regarding my current or previous work. The best way to reach me is through email.
Generally, I try to open-source all of my coursework notes and development material, which I publish through Github. If you observe bugs or errors in any of my repositories, I would sincerely appreciate if you contacted me directly or even better initiated a Github pull request.
- zayd.hammoudeh at gmail dot com
- Google Scholar
- ZaydH