Neuron-based explanations of neural networks sacrifice completeness and interpretability.

Nolan Dey Nolan S. Dey Nolan Simran Dey

Nolan Simran Dey J. Eric Taylor Alexander Wong Bryan P. Tripp Graham W. Taylor Neuron-based explanations of neural networks sacrifice completeness and interpretability. 2025 2025 Trans. Mach. Learn. Res. https://openreview.net/forum?id=UWNa9Pv6qA db/journals/tmlr/tmlr2025.html#DeyTWTT25

Shane Bergsma Nolan Simran Dey Gurpreet Gosal Gavia Gray Daria Soboleva Joel Hestness Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. 2025 ICLR https://openreview.net/forum?id=hrOlBgHsMI conf/iclr/2025 db/conf/iclr/iclr2025.html#BergsmaDGGSH25

Shane Bergsma Nolan Dey Gurpreet Gosal Gavia Gray Daria Soboleva Joel Hestness Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. 2025 February abs/2502.15938 CoRR https://doi.org/10.48550/arXiv.2502.15938 db/journals/corr/corr2502.html#abs-2502-15938 streams/journals/corr

Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Bill Li Blake Bordelon Shane Bergsma Cengiz Pehlevan Boris Hanin Joel Hestness Don't be lazy: CompleteP enables compute-efficient deep transformers. 2025 May abs/2505.01618 CoRR https://doi.org/10.48550/arXiv.2505.01618 db/journals/corr/corr2505.html#abs-2505-01618 streams/journals/corr

Shane Bergsma Nolan Dey Gurpreet Gosal Gavia Gray Daria Soboleva Joel Hestness Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training. 2025 May abs/2505.13738 CoRR https://doi.org/10.48550/arXiv.2505.13738 db/journals/corr/corr2505.html#abs-2505-13738 streams/journals/corr

Shane Bergsma Bin Claire Zhang Nolan Dey Shaheer Muhammad Gurpreet Gosal Joel Hestness Scaling with Collapse: Efficient and Predictable Training of LLM Families. 2025 September abs/2509.25087 CoRR https://doi.org/10.48550/arXiv.2509.25087 db/journals/corr/corr2509.html#abs-2509-25087 streams/journals/corr

Shane Bergsma Nolan Dey Joel Hestness Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs. 2025 September abs/2509.25380 CoRR https://doi.org/10.48550/arXiv.2509.25380 db/journals/corr/corr2509.html#abs-2509-25380 streams/journals/corr

Nolan Dey Shane Bergsma Joel Hestness Sparse maximal update parameterization: A holistic approach to sparse training dynamics. 2024 NeurIPS http://papers.nips.cc/paper_files/paper/2024/hash/3b6aaffec941f98930753fa6d6de7263-Abstract-Conference.html conf/nips/2024 db/conf/nips/neurips2024.html#DeyBH24

Nolan Dey Shane Bergsma Joel Hestness Sparse maximal update parameterization: A holistic approach to sparse training dynamics. 2024 abs/2405.15743 CoRR https://doi.org/10.48550/arXiv.2405.15743 db/journals/corr/corr2405.html#abs-2405-15743

Nolan Dey Gurpreet Gosal Zhiming Chen Hemant Khachane William Marshall Ribhu Pathria Marvin Tom Joel Hestness Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster. 2023 abs/2304.03208 CoRR https://doi.org/10.48550/arXiv.2304.03208 db/journals/corr/corr2304.html#abs-2304-03208

Nolan Dey Daria Soboleva Faisal Al-Khateeb Bowen Yang Ribhu Pathria Hemant Khachane Shaheer Muhammad Zhiming Chen Robert Myers Jacob Robert Steeves Natalia Vassilieva Marvin Tom Joel Hestness BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model. 2023 abs/2309.11568 CoRR https://doi.org/10.48550/arXiv.2309.11568 db/journals/corr/corr2309.html#abs-2309-11568

Faisal Al-Khateeb Nolan Dey Daria Soboleva Joel Hestness Position Interpolation Improves ALiBi Extrapolation. 2023 abs/2310.13017 CoRR https://doi.org/10.48550/arXiv.2310.13017 db/journals/corr/corr2310.html#abs-2310-13017

Victor Reyes Osorio Rajan Iyengar Xueyang Yao Presish Bhattachan Adrian Ragobar Nolan Dey Bryan P. Tripp 37, 000 Human-Planned Robotic Grasps With Six Degrees of Freedom. 3346-3351 2020 5 IEEE Robotics Autom. Lett. 2 https://doi.org/10.1109/LRA.2020.2976295 db/journals/ral/ral5.html#OsorioIYBRDT20

Nolan S. Dey J. Eric Taylor Bryan P. Tripp Alexander Wong Graham W. Taylor Identifying and interpreting tuning dimensions in deep networks. 2020 abs/2011.03043 CoRR https://arxiv.org/abs/2011.03043 db/journals/corr/corr2011.html#abs-2011-03043

Faisal Al-Khateeb Shane Bergsma Presish Bhattachan Blake Bordelon Zhiming Chen Gurpreet Gosal Gavia Gray Boris Hanin Joel Hestness Rajan Iyengar Hemant Khachane Mufan Bill Li William Marshall Shaheer Muhammad Robert Myers Lorenzo Noci Victor Reyes Osorio Ribhu Pathria Cengiz Pehlevan Adrian Ragobar Daria Soboleva Jacob Robert Steeves Graham W. Taylor J. Eric Taylor Marvin Tom Bryan P. Tripp Natalia Vassilieva Alexander Wong Bowen Yang Xueyang Yao Bin Claire Zhang