On Stochastic Roundoff Errors in Gradient Descent with Low-Precision Computation
-
Published:2023-12-20
Issue:2
Volume:200
Page:634-668
-
ISSN:0022-3239
-
Container-title:Journal of Optimization Theory and Applications
-
language:en
-
Short-container-title:J Optim Theory Appl
Author:
Xia LuORCID, Massei Stefano, Hochstenbach Michiel E., Koren Barry
Funder
H2020 LEIT Information and Communication Technologies
Publisher
Springer Science and Business Media LLC
Subject
Applied Mathematics,Management Science and Operations Research,Control and Optimization
Reference36 articles.
1. Bertsekas, D.P., Tsitsiklis, J.N.: Gradient convergence in gradient methods with errors. SIAM J. Optim. 10(3), 627–642 (2000) 2. Böhning, D.: Multinomial logistic regression algorithm. Ann. Inst. Stat. Math. 44(1), 197–200 (1992) 3. Chung, E., Fowers, J., Ovtcharov, K., Papamichael, M., Caulfield, A., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., Abeydeera, M., Adams, L., Angepat, H., Boehn, C., Chiou, D., Firestein, O., Forin, A., Gatlin, K., Ghandi, M., Heil, S., Holohan, K., Husseini, A., Juhász, T., Kagi, K., Kovvuri, R., Lanka, S., Megen, F.V., Mukhortov, D., Patel, P., Perez, B., Rapsang, A., Reinhardt, S., Rouhani, B., Sapek, A., Seera, R., Shekar, S., Sridharan, B., Weisz, G., Woods, L., Xiao, P.Y., Zhang, D., Zhao, R., Burger, D.: Serving DNNs in real time at datacenter scale with project brainwave. IEEE Micro 38(2), 8–20 (2018) 4. Connolly, M.P., Higham, N.J., Mary, T.: Stochastic rounding and its probabilistic backward error analysis. SIAM J. Sci. Comput. 43(1), A566–A585 (2021) 5. Croci, M., Fasi, M., Higham, N.J., Mary, T., Mikaitis, M.: Stochastic rounding: implementation, error analysis and applications. R. Soc. Open Sci. 9(3), 211631 (2022)
|
|