Age K. Smilde is a Professor of Biosystems Data Analysis at the Swammerdam Institute for Life Sciences at the University of Amsterdam. He also holds a part-time position at the Department of Machine Intelligence of Simula Metropolitan Center for Digital Engineering in Oslo, Norway. His research interest is multiblock data analysis and its implementation in different fields of life sciences. He is currently the Editor-in-Chief of the Journal of Chemometrics.

Tormod Næs is a Senior Scientist at Nofima, a food research institute in Norway. He is also currently employed as adjoint professor at the Department of Food Science, University of Copenhagen, Denmark and as extraordinary professor at University of Stellenbosch, South Africa. His main research interest is multivariate analysis with special emphasis on applications in sensory science and spectroscopy.

Kristian Hovde Liland is an Associate Professor with a top scientist scholarship in Data Science at the Norwegian University of Life Sciences and works in the areas of chemometrics, data analysis and machine learning. His main research is in linear prediction modelling, spectroscopy, and the transition between chemometrics and machine learning.

Foreword xiii

Preface xv

List of Figures xvii

List of Tables xxxi

Part I Introductory Concepts and Theory 1

1 Introduction 3

1.1 Scope of the Book 3

1.2 Potential Audience 4

1.3 Types of Data and Analyses 5

1.3.1 Supervised and Unsupervised Analyses 5

1.3.2 High-, Mid- and Low-level Fusion 5

1.3.3 Dimension Reduction 7

1.3.4 Indirect Versus Direct Data 8

1.3.5 Heterogeneous Fusion 8

1.4 Examples 8

1.4.1 Metabolomics 8

1.4.2 Genomics 11

1.4.3 Systems Biology 13

1.4.4 Chemistry 13

1.4.5 Sensory Science 15

1.5 Goals of Analyses 16

1.6 Some History 17

1.7 Fundamental Choices 17

1.8 Common and Distinct Components 19

1.9 Overview and Links 20

1.10 Notation and Terminology 21

1.11 Abbreviations 22

2 Basic Theory and Concepts 25

2.i General Introduction 25

2.1 Component Models 25

2.1.1 General Idea of Component Models 25

2.1.2 Principal Component Analysis 26

2.1.3 Sparse PCA 30

2.1.4 Principal Component Regression 31

2.1.5 Partial Least Squares 32

2.1.6 Sparse PLS 36

2.1.7 Principal Covariates Regression 37

2.1.8 Redundancy Analysis 38

2.1.9 Comparing PLS, PCovR and RDA 38

2.1.10 Generalised Canonical Correlation Analysis 38

2.1.11 Simultaneous Component Analysis 39

2.2 Properties of Data 39

2.2.1 Data Theory 39

2.2.2 Scale-types 42

2.3 Estimation Methods 44

2.3.1 Least-squares Estimation 44

2.3.2 Maximum-likelihood Estimation 45

2.3.3 Eigenvalue Decomposition-based Methods 47

2.3.4 Covariance or Correlation-based Estimation Methods 47

2.3.5 Sequential Versus Simultaneous Methods 48

2.3.6 Homogeneous Versus Heterogeneous Fusion 50

2.4 Within- and Between-block Variation 52

2.4.1 Definition and Example 52

2.4.2 MAXBET Solution 54

2.4.3 MAXNEAR Solution 54

2.4.4 PLS2 Solution 55

2.4.5 CCA Solution 55

2.4.6 Comparing the Solutions 56

2.4.7 PLS, RDA and CCA Revisited 56

2.5 Framework for Common and Distinct Components 60

2.6 Preprocessing 63

2.7 Validation 64

2.7.1 Outliers 64

2.7.1.1 Residuals 64

2.7.1.2 Leverage 66

2.7.2 Model Fit 67

2.7.3 Bias-variance Trade-off 69

2.7.4 Test Set Validation 70

2.7.5 Cross-validation 72

2.7.6 Permutation Testing 75

2.7.7 Jackknife and Bootstrap 76

2.7.8 Hyper-parameters and Penalties 77

2.8 Appendix 78

3 Structure of Multiblock Data 87

3.i General Introduction 87

3.1 Taxonomy 87

3.2 Skeleton of a Multiblock Data Set 87

3.2.1 Shared Sample Mode 88

3.2.2 Shared Variable Mode 88

3.2.3 Shared Variable or Sample Mode 88

3.2.4 Shared Variable and Sample Mode 89

3.3 Topology of a Multiblock Data Set 90

3.3.1 Unsupervised Analysis 90

3.3.2 Supervised Analysis 93

3.4 Linking Structures 95

3.4.1 Linking Structure for Unsupervised Analysis 95

3.4.2 Linking Structures for Supervised Analysis 96

3.5 Summary 98

4 Matrix Correlations 99

4.i General Introduction 99

4.1 Definition 99

4.2 Most Used Matrix Correlations 101

4.2.1 Inner Product Correlation 101

4.2.2 GCD coefficient 101

4.2.3 RV-coefficient 102

4.2.4 SMI-coefficient 102

4.3 Generic Framework of Matrix Correlations 104

4.4 Generalised Matrix Correlations 105

4.4.1 Generalised RV-coefficient 105

4.4.2 Generalised Association Coefficient 106

4.5 Partial Matrix Correlations 108

4.6 Conclusions and Recommendations 110

4.7 Open Issues 111

Part II Selected Methods for Unsupervised and Supervised Topologies 113

5 Unsupervised Methods 115

5.i General Introduction 115

[...] Relations to the General Framework 115

5.1 Shared Variable Mode 117

5.1.1 Only Common Variation 117

5.1.1.1 Simultaneous Component Analysis 117

5.1.1.2 Clustering and SCA 123

5.1.1.3 Multigroup Data Analysis 125

5.1.2 Common, Local, and Distinct Variation 126

5.1.2.1 Distinct and Common Components 127

5.1.2.2 Multivariate Curve Resolution 130

5.2 Shared Sample Mode 133

5.2.1 Only Common Variation 133

5.2.1.1 SUM-PCA 133

5.2.1.2 Multiple Factor Analysis and STATIS 135

5.2.1.3 Generalised Canonical Analysis 136

5.2.1.4 Regularised Generalised Canonical Correlation Analysis 139

5.2.1.5 Exponential Family SCA 140

5.2.1.6 Optimal-scaling 143

5.2.2 Common, Local, and Distinct Variation 146

5.2.2.1 Joint and Individual Variation Explained 146

5.2.2.2 Distinct and Common Components 147

5.2.2.3 PCA-GCA 148

5.2.2.4 Advanced Coupled Matrix and Tensor Factorisation 153

5.2.2.5 Penalised-ESCA 156

5.2.2.6 Multivariate Curve Resolution 158

5.3 Generic Framework 159

5.3.1 Framework for Simultaneous Unsupervised Methods 159

5.3.1.1 Description of the Framework 159

5.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods 161

5.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods 161

5.4 Conclusions and Recommendations 162

5.5 Open Issues 164

6 ASCA and Extensions 167

6.i General Introduction 167

[...] Relations to the General Framework 167

6.1 ANOVA-Simultaneous Component Analysis 168

6.1.1 The ASCA Method 168

6.1.2 Validation of ASCA 176

6.1.2.1 Permutation Testing 176

6.1.2.2 Back-projection 178

6.1.2.3 Confidence Ellipsoids 178

6.1.3 The ASCA+ and LiMM-PCA Methods 181

6.2 Multilevel-SCA 182

6.3 Penalised-ASCA 183

6.4 Conclusions and Recommendations 185

6.5 Open Issues 186

7 Supervised Methods 187

7.i General Introduction 187

[...] Relations to the General Framework 187

7.1 Multiblock Regression: General Perspectives 188

7.1.1 Model and Assumptions 188

7.1.2 Different Challenges and Aims 188

7.2 Multiblock PLS Regression 190

7.2.1 Standard Multiblock PLS Regression 190

7.2.2 MB-PLS Used for Classification 194

7.2.3 Sparse Multiblock PLS Regression (sMB-PLS) 196

7.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression) 199

7.3.1 The SO-PLS Method 199

7.3.2 Order of Blocks 202

7.3.3 Interpretation Tools 202

7.3.4 Restricted PLS Components and their Application in SO-PLS 203

7.3.5 Validation and Component Selection 204

7.3.6 Relations to ANOVA 205

7.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks 212

7.3.8 Further Applications of SO-PLS 215

7.3.9 Relations Between SO-PLS and ASCA 215

7.4 Parallel and Orthogonalised PLS (PO-PLS) Regression 217

7.5 Response Oriented Sequential Alternation 222

7.5.1 The ROSA Method 222

7.5.2 Validation 225

7.5.3 Interpretation 225

7.6 Conclusions and Recommendations 228

7.7 Open Issues 229

Part III Methods for Complex Multiblock Structures 231

8 Complex Block Structures; with Focus on L-Shape Relations 233

8.i General Introduction 233

[...] Relations to the General Framework 234

8.1 Analysis of L-shape Data: General Perspectives 235

8.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA 236

8.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First 236

8.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First 238

8.2.3 Analysis of Segments/Clusters of X1 Data 240

8.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data 246

8.3.1 The Original L-PLS Method, Endo-L-PLS 247

8.3.2 Exo- Versus Endo-L-PLS 250

8.4 Modifications of the Original L-PLS Idea 252

8.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter ¿252

8.4.2 Three-blocks Bifocal PLS 253

8.5 Alternative L-shape Data Analysis Methods 254

8.5.1 Principal Component Analysis with External Information 254

8.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration 255

8.5.3 Multivariate Curve Resolution for Incomplete Data 256

8.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1 257

8.6 Domino PLS and More Complex Data Structures 258

8.7 Conclusions and Recommendations 258

8.8 Open Issues 260

Part IV Alternative Methods for Unsupervised and Supervised Topologies 261

9 Alternative Unsupervised Methods 263

9.i General Introduction 263

[...] Relationship to the General Framework 263

9.1 Shared Variable Mode 263

9.2 Shared Sample Mode 265

9.2.1 Only Common Variation 265

9.2.1.1 DIABLO 265

9.2.1.2 Generalised Coupled Tensor Factorisation 266

9.2.1.3 Representation Matrices 267

9.2.1.4 Extended PCA 272

9.2.2 Common, Local, and Distinct Variation 273

9.2.2.1 Generalised SVD 273

9.2.2.2 Structural Learning and Integrative Decomposition 273

9.2.2.3 Bayesian Inter-battery Factor Analysis 275

9.2.2.4 Group Factor Analysis 276

9.2.2.5 OnPLS 277

9.2.2.6 Generalised Association Study 278

9.2.2.7 Multi-Omics Factor Analysis 278

9.3 Two Shared Modes and Only Common Variation 281

9.3.1 Generalised Procrustes Analysis 282

9.3.2 Three-way Methods 282

9.4 Conclusions...