English
Related papers

Related papers: A new algorithm for estimating the effective dimen…

200 papers

Optimizing machine learning algorithms that are used to solve the objective function has been of great interest. Several approaches to optimize common algorithms, such as gradient descent and stochastic gradient descent, were explored. One…

Machine Learning · Computer Science 2022-10-06 Hilal AlQuabeh , Farha AlBreiki , Dilshod Azizov

Dimensionality reduction (DR) is a popular method for preparing and analyzing high-dimensional data. Reduced data representations are less computationally intensive and easier to manage and visualize, while retaining a significant…

Machine Learning · Computer Science 2022-05-02 Avraam Bardos , Ioannis Mollas , Nick Bassiliades , Grigorios Tsoumakas

Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional…

Machine Learning · Computer Science 2014-04-29 Voot Tangkaratt , Ning Xie , Masashi Sugiyama

Unsupervised machine learning lacks ground truth by definition. This poses a major difficulty when designing metrics to evaluate the performance of such algorithms. In sharp contrast with supervised learning, for which plenty of quality…

Machine Learning · Computer Science 2023-03-20 Raúl Lara-Cabrera , Ángel González-Prieto , Diego Pérez-López , Diego Trujillo , Fernando Ortega

Estimating the effective dimension reduction (EDR) space, related to the semiparametric regression model introduced by Li \cite{sir}, is based on the estimation of the covariance matrix $\Lambda$ of the conditional expectation of the vector…

Statistics Theory · Mathematics 2018-11-08 Emmanuel De Dieu Nkou , Guy Martial Nkiet

Sufficient dimension reduction is used pervasively as a supervised dimension reduction approach. Most existing sufficient dimension reduction methods are developed for data with a continuous response and may have an unsatisfactory…

Machine Learning · Computer Science 2021-02-03 Cheng Meng , Jun Yu , Jingyi Zhang , Ping Ma , Wenxuan Zhong

It is a standard assumption that datasets in high dimension have an internal structure which means that they in fact lie on, or near, subsets of a lower dimension. In many instances it is important to understand the real dimension of the…

Machine Learning · Statistics 2025-07-21 James A. D. Binnie , Paweł Dłotko , John Harvey , Jakub Malinowski , Ka Man Yim

Dimension reduction algorithms are a crucial part of many data science pipelines, including data exploration, feature creation and selection, and denoising. Despite their wide utilization, many non-linear dimension reduction algorithms are…

Machine Learning · Statistics 2024-08-06 Ryan Murray , Adam Pickarski

Searching for the $k$-nearest neighbors (KNN) in multimodal data retrieval is computationally expensive, particularly due to the inherent difficulty in comparing similarity measures across different modalities. Recent advances in multimodal…

Machine Learning · Computer Science 2025-08-25 Chengyu Gong , Gefei Shen , Luanzheng Guo , Nathan Tallent , Dongfang Zhao

Nonlinear regression problem is one of the most popular and important statistical tasks. The first methods like least squares estimation go back to Gauss and Legendre. Recent models and developments in statistics and machine learning like…

Statistics Theory · Mathematics 2025-02-20 Vladimir Spokoiny

We introduce a computationally efficient method for the automation of inverse design in science and engineering. Based on simple least-square regression, the underlying dynamic mode decomposition algorithm can be used to construct a…

Machine Learning · Computer Science 2025-02-14 Yunpeng Zhu , Liangliang Cheng , Anping Jing , Hanyu Huo , Ziqiang Lang , Bo Zhang , J. Nathan Kutz

Sufficient dimension reduction (SDR) is a popular class of regression methods which aim to find a small number of linear combinations of covariates that capture all the information of the responses i.e., a central subspace. The majority of…

Methodology · Statistics 2024-10-15 Linh H. Nghiem , F. K. C. Hui

Data-driven decision-making is performed by solving a parameterized optimization problem, and the optimal decision is given by an optimal solution for unknown true parameters. We often need a solution that satisfies true constraints even…

Optimization and Control · Mathematics 2020-03-03 Akihiro Yabe , Takanori Maehara

We consider the problem of sufficient dimensionality reduction (SDR), where the high-dimensional observation is transformed to a low-dimensional sub-space in which the information of the observations regarding the label variable is…

Machine Learning · Computer Science 2018-12-20 Ershad Banijamali , Amir-Hossein Karimi , Ali Ghodsi

Clustering methods with dimension reduction have been receiving considerable wide interest in statistics lately and a lot of methods to simultaneously perform clustering and dimension reduction have been proposed. This work presents a novel…

Methodology · Statistics 2014-06-17 Michio Yamamoto , Kenichi Hayashi

A method is introduced to perform simultaneous sparse dimension reduction on two blocks of variables. Beyond dimension reduction, it also yields an estimator for multivariate regression with the capability to intrinsically deselect…

Methodology · Statistics 2024-11-28 Sven Serneels

We develop a linear-algebraic framework for dimensional analysis in systems with constraints, particularly when variables are numerous or related by implicit relations so that direct elimination is impractical. By expressing both…

Mathematical Physics · Physics 2026-03-31 Umpei Miyamoto

Latent variable models represent a useful tool for the analysis of complex data when the constructs of interest are not observable. A problem related to these models is that the integrals involved in the likelihood function cannot be solved…

Methodology · Statistics 2015-03-05 Silvia Bianconcini , Silvia Cagnone , Dimitris Rizopoulos

Sliced inverse regression (SIR) is the most widely-used sufficient dimension reduction method due to its simplicity, generality and computational efficiency. However, when the distribution of the covariates deviates from the multivariate…

Methodology · Statistics 2018-01-09 Jia Zhang , Xin Chen , Wang Zhou

An analysis of high-dimensional data can offer a detailed description of a system but is often challenged by the curse of dimensionality. General dimensionality reduction techniques can alleviate such difficulty by extracting a few…

Methodology · Statistics 2021-09-28 Di Bo , Hoon Hwangbo , Vinit Sharma , Corey Arndt , Stephanie C. TerMaath