arXiv cs.LG· Abdul Kadir (University of Oldenburg, Oldenburg, Germany, German Research Center for Artificial Intelligence), Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbrucken, Germany), Daniel Sonntag (University of Oldenburg, Oldenburg, Germany, German Research Center for Artificial Intelligence)·· 9 小时前AI 评分27
语言模型网络信息检索的注意力头与神经元归因研究
Finding the Heads and the Neurons Responsible for Network Information Retrieval in Language Models
AI 导读
研究因果验证了语言模型识别网络基础设施信息（主机名与IP地址配对）的注意力头：在三个架构家族的五个模型中，每个模型仅需1至9个注意力头（候选128至1152个）即可支撑检测器达到99.5–100%的留出准确率。
来源：arXiv cs.LG · arxiv.org