Learning from Emergence: A Study on Proactively Inhibiting the Monosemantic Neurons of Artificial Neural Networks-Reference-Cited by-同舟云学术

Learning from Emergence: A Study on Proactively Inhibiting the Monosemantic Neurons of Artificial Neural Networks

Published:2024-08-24 Issue: Volume:5 Page:3092-3103
ISSN:
Container-title:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
language:
Short-container-title:

Author:

Wang Jiachuan¹^ORCID,Di Shimin¹^ORCID,Chen Lei²^ORCID,Ng Charles Wang Wai³^ORCID

Affiliation:

1. The Hong Kong University of Science and Technology, Hong Kong SAR, China

2. The Hong Kong University of Science and Technology, (Guangzhou), Guangzhou, China

3. The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Funder

National Science Foundation of China

Zhujiang scholar program

Microsoft Research Asia Collaborative Research Grant

National Key Research and Development Program of China

Hong Kong RGC

Hong Kong ITC

HKUST-Webank joint research lab grants

Guangdong Province Science and Technology Plan Project

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3637528.3671776

Reference41 articles.

1. 2024. Technical report of our paper. https://github.com/dominatorX/MEmeLcode/ blob/main/EmeL_tech_report.pdf

2. Paszke Adam Gross Sam Chintala Soumith Chanan Gregory Yang Edward DeVito Zachary Lin Zeming Desmaison Alban Antiga Luca and Lerer Adam. 2017. Automatic differentiation in PyTorch. (2017).

3. Understanding the role of individual units in a deep neural network

4. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning. PMLR, 2397--2430.

5. Zoom in: An introduction to circuits;Chris Olah;Distill,2020