Approximation of functions of several variables by deep operators activated by sigmoidal functions and rectified power units

In the present paper the constructive theory of multivariate deep operators activated by sigmoidal functions is introduced and studied. The main result here establishes a pointwise and uniform convergence theorem in the space of continuous functions on the compact [-1,1]d\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$[-1,1]<^>d$$\end{document}, d >= 2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d\ge 2$$\end{document}. Moreover, we also prove quantitative estimates for the order of approximation and the corresponding qualitative rate of convergence. In order to achieve the above theorems, some auxiliary results have been preliminary proved. The above introduced operators belong to the field of positive linear operators, and they also represent a sort of deep (multi-layer) counterpart of the well-known (shallow) operators activated by sigmoidal functions studied in recent years. The above family of deep operators can be considered activated also by the ReLU and powers of ReLU, also known as rectified linear unit and rectified powers units, respectively.