Article
| Title: | Exponential expressivity of ${\rm ReLU}^k$ neural networks on Gevrey classes with point singularities (English) |
| Author: | Opschoor, Joost A. A. |
| Author: | Schwab, Christoph |
| Language: | English |
| Journal: | Applications of Mathematics |
| ISSN: | 0862-7940 (print) |
| ISSN: | 1572-9109 (online) |
| Volume: | 69 |
| Issue: | 5 |
| Year: | 2024 |
| Pages: | 695-724 |
| Summary lang: | English |
| . | |
| Category: | math |
| . | |
| Summary: | We analyze deep Neural Network emulation rates of smooth functions with point singularities in bounded, polytopal domains ${\rm D} \subset \mathbb R^d$, $d=2,3$. We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons and in terms of the number of nonzero coefficients for Gevrey-regular solution classes defined in terms of weighted Sobolev scales in ${\rm D}$, comprising the countably-normed spaces of I. M. Babuška and B. Q. Guo. \endgraf As intermediate result, we prove that continuous, piecewise polynomial high order (``$p$-version'') finite elements with elementwise polynomial degree $p\in \mathbb{N} $ on arbitrary, regular, simplicial partitions of polyhedral domains ${\rm D} \subset \mathbb R^d$, $d\geq 2$, can be \emph {exactly emulated} by neural networks combining ReLU and ReLU$^2$ activations. \endgraf On shape-regular, simplicial partitions of polytopal domains ${\rm D}$, both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the $hp$ finite element space of I. M. Babuška and B. Q. Guo. (English) |
| Keyword: | neural network |
| Keyword: | $hp$-finite element method |
| Keyword: | singularities |
| Keyword: | Gevrey regularity |
| Keyword: | exponential convergence |
| MSC: | 41A25 |
| MSC: | 65N30 |
| DOI: | 10.21136/AM.2024.0052-24 |
| . | |
| Date available: | 2024-11-05T12:05:37Z |
| Last updated: | 2024-11-05 |
| Stable URL: | http://hdl.handle.net/10338.dmlcz/152637 |
| . | |
| Reference: | [1] Arora, R., Basu, A., Mianjy, P., Mukherjee, A.: Understanding deep neural networks with rectified linear units.Available at https://arxiv.org/abs/1611.01491 (2018), 17 pages. 10.48550/arXiv.1611.01491 |
| Reference: | [2] Babuška, I., Guo, B. Q.: Regularity of the solution of elliptic problems with piecewise analytic data. I. Boundary value problems for linear elliptic equations of second order.SIAM J. Math. Anal. 19 (1988), 172-203. Zbl 0647.35021, MR 0924554, 10.1137/0519014 |
| Reference: | [3] Babuška, I., Guo, B. Q.: The $h$-$p$ version of the finite element method for domains with curved boundaries.SIAM J. Numer. Anal. 25 (1988), 837-861. Zbl 0655.65124, MR 0954788, 10.1137/0725048 |
| Reference: | [4] Babuška, I., Guo, B. Q.: Regularity of the solution of elliptic problems with piecewise analytic data. II. The trace spaces and application to the boundary value problems with nonhomogeneous boundary conditions.SIAM J. Math. Anal. 20 (1989), 763-781. Zbl 0706.35028, MR 1000721, 10.1137/0520054 |
| Reference: | [5] Babuška, I., Guo, B. Q.: Approximation properties of the $h$-$p$ version of the finite element method.Comput. Methods Appl. Mech. Eng. 133 (1996), 319-346. Zbl 0882.65096, MR 1399640, 10.1016/0045-7825(95)00946-9 |
| Reference: | [6] Banjai, L., Melenk, J. M., Schwab, C.: Exponential convergence of $hp$ FEM for spectral fractional diffusion in polygons.Numer. Math. 153 (2023), 1-47. Zbl 1511.65117, MR 4530200, 10.1007/s00211-022-01329-5 |
| Reference: | [7] Banjai, L., Melenk, J. M., Schwab, C.: $hp$-FEM for reaction-diffusion equations. II. Robust exponential convergence for multiple length scales in corner domains.IMA J. Numer. Anal. 43 (2023), 3282-3325. Zbl 07800835, MR 4673336, 10.1093/imanum/drac070 |
| Reference: | [8] Bao, G., Hu, G., Liu, D.: An $h$-adaptive finite element solver for the calculations of the electronic structures.J. Comput. Phys. 231 (2012), 4967-4979. Zbl 1245.65125, 10.1016/j.jcp.2012.04.002 |
| Reference: | [9] Bolley, P., Dauge, M., Camus, J.: Régularité Gevrey pour le problème de Dirichlet dans des domaines à singularités coniques.Commun. Partial Differ. Equations 10 (1985), 391-431 French. Zbl 0573.35024, MR 0784683, 10.1080/03605308508820383 |
| Reference: | [10] Cancès, E., Chakir, R., Maday, Y.: Numerical analysis of nonlinear eigenvalue problems.J. Sci. Comput. 45 (2010), 90-117. Zbl 1203.65237, MR 2679792, 10.1007/s10915-010-9358-1 |
| Reference: | [11] Chemin, J.-Y.: Perfect Incompressible Fluids.Oxford Lecture Series in Mathematics and Its Applications 14. Clarendon Press, New York (1998). Zbl 0927.76002, MR 1688875, 10.1093/oso/9780198503972.001.0001 |
| Reference: | [12] Chen, Q., Babuška, I.: The optimal symmetrical points for polynomial interpolation of real functions in the tetrahedron.Comput. Methods Appl. Mech. Eng. 137 (1996), 89-94. Zbl 0877.65004, MR 1420062, 10.1016/0045-7825(96)01051-1 |
| Reference: | [13] Costabel, M., Dauge, M., Nicaise, S.: Singularities of Maxwell interface problems.M2AN, Math. Model. Numer. Anal. 33 (1999), 627-649. Zbl 0937.78003, MR 1713241, 10.1051/m2an:1999155 |
| Reference: | [14] Costabel, M., Dauge, M., Nicaise, S.: Analytic regularity for linear elliptic systems in polygons and polyhedra.Math. Models Methods Appl. Sci. 22 (2012), Article ID 1250015, 63 pages. Zbl 1257.35056, MR 2928103, 10.1142/S0218202512500157 |
| Reference: | [15] Dubiner, M.: Spectral methods on triangles and other domains.J. Sci. Comput. 6 (1991), 345-390. Zbl 0742.76059, MR 1154903, 10.1007/BF01060030 |
| Reference: | [16] Elbrächter, D., Grohs, P., Jentzen, A., Schwab, C.: DNN expression rate analysis of high-dimensional PDEs: Application to option pricing.Constr. Approx. 55 (2022), 3-71. Zbl 1500.35009, MR 4376559, 10.1007/s00365-021-09541-6 |
| Reference: | [17] Ern, A., Guermond, J.-L.: Finite Elements. I. Approximation and Interpolation.Texts in Applied Mathematics 72. Springer, Cham (2021). Zbl 1476.65003, MR 4242224, 10.1007/978-3-030-56341-7 |
| Reference: | [18] Faustmann, M., Marcati, C., Melenk, J. M., Schwab, C.: Exponential convergence of $hp$-FEM for the integral fractional Laplacian in polygons.SIAM J. Numer. Anal. 61 (2023), 2601-2622. Zbl 1533.65224, MR 4667264, 10.1137/22M152493X |
| Reference: | [19] Feischl, M., Schwab, C.: Exponential convergence in $H^1$ of $hp$-FEM for Gevrey regularity with isotropic singularities.Numer. Math. 144 (2020), 323-346. Zbl 1433.65290, MR 4057425, 10.1007/s00211-019-01085-z |
| Reference: | [20] Fournais, S., Hoffmann-Ostenhof, M., Hoffmann-Ostenhof, T., Sørensen, T. Ø.: Analytic structure of solutions to multiconfiguration equations.J. Phys. A. Math. Theor. 42 (2009), Article ID 315208, 11 pages. Zbl 1181.35282, MR 2521310, 10.1088/1751-8113/42/31/315208 |
| Reference: | [21] Guo, B.: The $h$-$p$ Version of Finite Element Method in Two Dimensions. Mathematical Theory and Computational Experience: Ph.D. Thesis.University of Maryland, College Park (1985). MR 2634330 |
| Reference: | [22] Guo, B. Q., Babuška, I.: On the regularity of elasticity problems with piecewise analytic data.Adv. Appl. Math. 14 (1993), 307-347. Zbl 0790.35028, MR 1228743, 10.1006/aama.1993.1016 |
| Reference: | [23] Guo, B., Babuška, I.: Regularity of the solutions for elliptic problems on nonsmooth domains in $\Bbb R^3$. I. Countably normed spaces on polyhedral domains.Proc. R. Soc. Edinb., Sect. A 127 (1997), 77-126. Zbl 0874.35019, MR 1433086, 10.1017/S0308210500023520 |
| Reference: | [24] Guo, B., Babuška, I.: Regularity of the solutions for elliptic problems on nonsmooth domains in $\Bbb R^3$. II. Regularity in neighbourhoods of edges.Proc. R. Soc. Edinb., Sect. A 127 (1997), 517-545. Zbl 0884.35022, MR 1453280, 10.1017/S0308210500029899 |
| Reference: | [25] Guo, B., Schwab, C.: Analytic regularity of Stokes flow on polygonal domains in countably weighted Sobolev spaces.J. Comput. Appl. Math. 190 (2006), 487-519. Zbl 1121.35098, MR 2209521, 10.1016/j.cam.2005.02.018 |
| Reference: | [26] He, J., Li, L., Xu, J., Zheng, C.: ReLU deep neural networks and linear finite elements.J. Comput. Math. 38 (2020), 502-527. Zbl 1463.68072, MR 4087799, 10.4208/jcm.1901-m2018-0160 |
| Reference: | [27] He, J., Xu, J.: Deep neural networks and finite elements of any order on arbitrary dimensions.Available at https://arxiv.org/abs/2312.14276 (2024), 29 pages. 10.48550/arXiv.2312.14276 |
| Reference: | [28] He, Y., Marcati, C., Schwab, C.: Analytic regularity of solutions to the Navier-Stokes equations with mixed boundary conditions in polygons.SIAM J. Math. Anal. 56 (2024), 2488-2520. Zbl 07849793, MR 4719953, 10.1137/22M1527428 |
| Reference: | [29] Herrmann, L., Opschoor, J. A. A., Schwab, C.: Constructive deep ReLU neural network approximation.J. Sci. Comput. 90 (2022), Article ID 75, 37 pages. Zbl 1500.41002, MR 4362469, 10.1007/s10915-021-01718-2 |
| Reference: | [30] Hesthaven, J. S.: From electrostatics to almost optimal nodal sets for polynomial interpolation in a simplex.SIAM J. Numer. Anal. 35 (1998), 655-676. Zbl 0933.41004, MR 1618874, 10.1137/S003614299630587X |
| Reference: | [31] Holm, B., Wihler, T. P.: Continuous and discontinuous Galerkin time stepping methods for nonlinear initial value problems with application to finite time blow-up.Numer. Math. 138 (2018), 767-799. Zbl 1453.65118, MR 3767700, 10.1007/s00211-017-0918-2 |
| Reference: | [32] Li, B., Tang, S., Yu, H.: Better approximations of high dimensional smooth functions by deep neural networks with rectified power units.Commun. Comput. Phys. 27 (2020), 379-411. Zbl 1474.65031, MR 4040947, 10.4208/cicp.OA-2019-0168 |
| Reference: | [33] Longo, M., Opschoor, J. A. A., Disch, N., Schwab, C., Zech, J.: De Rham compatible deep neural network FEM.Neural Netw. 165 (2023), 721-739. Zbl 1532.65083, 10.1016/j.neunet.2023.06.008 |
| Reference: | [34] Maday, Y., Marcati, C.: Regularity and $hp$ discontinuous Galerkin finite element approximation of linear elliptic eigenvalue problems with singular potentials.Math. Models Methods Appl. Sci. 29 (2019), 1585-1617. Zbl 1431.65209, MR 3986800, 10.1142/S0218202519500295 |
| Reference: | [35] Marcati, C., Opschoor, J. A. A., Petersen, P. C., Schwab, C.: Exponential ReLU neural network approximation rates for point and edge singularities.Found. Comput. Math. 23 (2023), 1043-1127. Zbl 1519.35253, MR 4603228, 10.1007/s10208-022-09565-9 |
| Reference: | [36] Marcati, C., Schwab, C.: Analytic regularity for the incompressible Navier-Stokes equations in polygons.SIAM J. Math. Anal. 52 (2020), 2945-2968. Zbl 1447.35252, MR 4113068, 10.1137/19M1247334 |
| Reference: | [37] Melenk, J. M.: $hp$-Finite Element Methods for Singular Perturbations.Lecture Notes in Mathematics 1796. Springer, Berlin (2002). Zbl 1021.65055, MR 1939620, 10.1007/b84212 |
| Reference: | [38] Melenk, J. M., Schwab, C.: $hp$ FEM for reaction-diffusion equations. I. Robust exponential convergence.SIAM J. Numer. Anal. 35 (1998), 1520-1557. Zbl 0972.65093, MR 1626030, 10.1137/S0036142997317602 |
| Reference: | [39] Opschoor, J. A. A.: Constructive Deep Neural Network Approximations of Weighted Analytic Solutions to Partial Differential Equations in Polygons: Ph.D. Thesis.ETH, Zürich (2023). 10.3929/ethz-b-000614671 |
| Reference: | [40] Opschoor, J. A. A., Petersen, P. C., Schwab, C.: Deep ReLU networks and high-order finite element methods.Anal. Appl., Singap. 18 (2020), 715-770. Zbl 1452.65354, MR 4131037, 10.1142/S0219530519410136 |
| Reference: | [41] Opschoor, J. A. A., Schwab, C.: Deep ReLU networks and high-order finite element methods. II. Chebyšev emulation.Comput. Math. Appl. 169 (2024), 142-162. Zbl 7894795, MR 4768195, 10.1016/j.camwa.2024.06.008 |
| Reference: | [42] Opschoor, J. A. A., Schwab, C., Xenophontos, C.: Neural networks for singular perturbations.Available at https://arxiv.org/abs/2401.06656 (2024), 39 pages. 10.48550/arXiv.2401.06656 |
| Reference: | [43] Opschoor, J. A. A., Schwab, C., Zech, J.: Exponential ReLU DNN expression of holomorphic maps in high dimension.Constr. Approx. 55 (2022), 537-582. Zbl 1500.41008, MR 4376568, 10.1007/s00365-021-09542-5 |
| Reference: | [44] Petersen, P., Voigtlaender, F.: Optimal approximation of piecewise smooth functions using deep ReLU neural networks.Neural Netw. 108 (2018), 296-330. Zbl 1434.68516, 10.1016/j.neunet.2018.08.019 |
| Reference: | [45] Samarskii, A. A., Galaktionov, V. A., Kurdyumov, S. P., Mikhailov, A. P.: Blow-Up in Quasilinear Parabolic Equations.de Gruyter Expositions in Mathematics 19. Walter De Gruyter, Berlin (1995). Zbl 1020.35001, MR 1330922, 10.1515/9783110889864.535 |
| Reference: | [46] Schwab, C.: $p$- and $hp$-Finite Element Methods: Theory and Applications in Solid and Fluid Mechanics.Numerical Mathematics and Scientific Computation. Clarendon Press, Oxford (1998). Zbl 0910.73003, MR 1695813 |
| Reference: | [47] Szabó, B., Babuška, I.: Finite Element Analysis.John Wiley & Sons, New York (1991). Zbl 0792.73003, MR 1164869 |
| Reference: | [48] Yang, Y., Wu, Y., Yang, H., Xiang, Y.: Nearly optimal approximation rates for deep super ReLU networks on Sobolev spaces.Available at https://arxiv.org/abs/2310.10766 (2023), 32 pages. 10.48550/arXiv.2310.10766 |
| Reference: | [49] Yarotsky, D.: Error bounds for approximations with deep ReLU networks.Neural Netw. 94 (2017), 103-114. Zbl 1429.68260, 10.1016/j.neunet.2017.07.002 |
| . |
Fulltext not available (moving wall 24 months)
Search
Partner of
