Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning-Reference-Cited by-同舟云学术

Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning

Published:2023-04-19 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
language:
Short-container-title:

Author:

Cabrera Ángel Alexander¹^ORCID,Fu Erica²^ORCID,Bertucci Donald³^ORCID,Holstein Kenneth¹^ORCID,Talwalkar Ameet⁴^ORCID,Hong Jason I.¹^ORCID,Perer Adam¹^ORCID

Affiliation:

1. Human-Computer Interaction Institute, Carnegie Mellon University, United States

2. Carnegie Mellon University, United States

3. School of Electrical Engineering and Computer Science, Oregon State University, United States

4. Machine Learning Department, Carnegie Mellon University, United States

Funder

National Science Foundation

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3544548.3581268

Reference61 articles.

1. FactSheets: Increasing trust in AI services through supplier's declarations of conformity

2. Josh Attenberg Panagiotis G. Ipeirotis and Foster Provost. 2011. Beat the Machine: Challenging Workers to Find the Unknown Unknowns. Josh Attenberg Panagiotis G. Ipeirotis and Foster Provost. 2011. Beat the Machine: Challenging Workers to Find the Unknown Unknowns.

3. Big Data’s Disparate Impact;Barocas Solon;SSRN Electronic Journal,2018

4. Donald Bertucci , Md Montaser Hamid , Yashwanthi Anand , Anita Ruangrotsakun , Delyar Tabatabai , Melissa Perez , and Minsuk Kahng . 2022. DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps . IEEE Transactions on Visualization and Computer Graphics (TVCG) ( 2022 ). https://div-lab.github.io/dendromap/ Publisher : IEEE. Donald Bertucci, Md Montaser Hamid, Yashwanthi Anand, Anita Ruangrotsakun, Delyar Tabatabai, Melissa Perez, and Minsuk Kahng. 2022. DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps. IEEE Transactions on Visualization and Computer Graphics (TVCG) (2022). https://div-lab.github.io/dendromap/ Publisher: IEEE.

5. Rishi Bommasani Drew A. Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill Erik Brynjolfsson Shyamal Buch Dallas Card Rodrigo Castellon Niladri Chatterji Annie Chen Kathleen Creel Jared Quincy Davis Dora Demszky Chris Donahue Moussa Doumbouya Esin Durmus Stefano Ermon John Etchemendy Kawin Ethayarajh Li Fei-Fei Chelsea Finn Trevor Gale Lauren Gillespie Karan Goel Noah Goodman Shelby Grossman Neel Guha Tatsunori Hashimoto Peter Henderson John Hewitt Daniel E. Ho Jenny Hong Kyle Hsu Jing Huang Thomas Icard Saahil Jain Dan Jurafsky Pratyusha Kalluri Siddharth Karamcheti Geoff Keeling Fereshte Khani Omar Khattab Pang Wei Koh Mark Krass Ranjay Krishna Rohith Kuditipudi Ananya Kumar Faisal Ladhak Mina Lee Tony Lee Jure Leskovec Isabelle Levent Xiang Lisa Li Xuechen Li Tengyu Ma Ali Malik Christopher D. Manning Suvir Mirchandani Eric Mitchell Zanele Munyikwa Suraj Nair Avanika Narayan Deepak Narayanan Ben Newman Allen Nie Juan Carlos Niebles Hamed Nilforoshan Julian Nyarko Giray Ogut Laurel Orr Isabel Papadimitriou Joon Sung Park Chris Piech Eva Portelance Christopher Potts Aditi Raghunathan Rob Reich Hongyu Ren Frieda Rong Yusuf Roohani Camilo Ruiz Jack Ryan Christopher Ré Dorsa Sadigh Shiori Sagawa Keshav Santhanam Andy Shih Krishnan Srinivasan Alex Tamkin Rohan Taori Armin W. Thomas Florian Tramèr Rose E. Wang William Wang Bohan Wu Jiajun Wu Yuhuai Wu Sang Michael Xie Michihiro Yasunaga Jiaxuan You Matei Zaharia Michael Zhang Tianyi Zhang Xikun Zhang Yuhui Zhang Lucia Zheng Kaitlyn Zhou and Percy Liang. 2022. On the Opportunities and Risks of Foundation Models. http://arxiv.org/abs/2108.07258 arXiv:2108.07258 [cs]. Rishi Bommasani Drew A. Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill Erik Brynjolfsson Shyamal Buch Dallas Card Rodrigo Castellon Niladri Chatterji Annie Chen Kathleen Creel Jared Quincy Davis Dora Demszky Chris Donahue Moussa Doumbouya Esin Durmus Stefano Ermon John Etchemendy Kawin Ethayarajh Li Fei-Fei Chelsea Finn Trevor Gale Lauren Gillespie Karan Goel Noah Goodman Shelby Grossman Neel Guha Tatsunori Hashimoto Peter Henderson John Hewitt Daniel E. Ho Jenny Hong Kyle Hsu Jing Huang Thomas Icard Saahil Jain Dan Jurafsky Pratyusha Kalluri Siddharth Karamcheti Geoff Keeling Fereshte Khani Omar Khattab Pang Wei Koh Mark Krass Ranjay Krishna Rohith Kuditipudi Ananya Kumar Faisal Ladhak Mina Lee Tony Lee Jure Leskovec Isabelle Levent Xiang Lisa Li Xuechen Li Tengyu Ma Ali Malik Christopher D. Manning Suvir Mirchandani Eric Mitchell Zanele Munyikwa Suraj Nair Avanika Narayan Deepak Narayanan Ben Newman Allen Nie Juan Carlos Niebles Hamed Nilforoshan Julian Nyarko Giray Ogut Laurel Orr Isabel Papadimitriou Joon Sung Park Chris Piech Eva Portelance Christopher Potts Aditi Raghunathan Rob Reich Hongyu Ren Frieda Rong Yusuf Roohani Camilo Ruiz Jack Ryan Christopher Ré Dorsa Sadigh Shiori Sagawa Keshav Santhanam Andy Shih Krishnan Srinivasan Alex Tamkin Rohan Taori Armin W. Thomas Florian Tramèr Rose E. Wang William Wang Bohan Wu Jiajun Wu Yuhuai Wu Sang Michael Xie Michihiro Yasunaga Jiaxuan You Matei Zaharia Michael Zhang Tianyi Zhang Xikun Zhang Yuhui Zhang Lucia Zheng Kaitlyn Zhou and Percy Liang. 2022. On the Opportunities and Risks of Foundation Models. http://arxiv.org/abs/2108.07258 arXiv:2108.07258 [cs].

Cited by 13 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Transparency in the Wild: Navigating Transparency in a Deployed AI System to Broaden Need-Finding Approaches;The 2024 ACM Conference on Fairness, Accountability, and Transparency;2024-06-03

2. Human-Centered Evaluation and Auditing of Language Models;Extended Abstracts of the CHI Conference on Human Factors in Computing Systems;2024-05-11

3. JupyterLab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data Scientists;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11

4. Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11

5. EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11