Data mining systems contain a large (and quickly growing) number of machine learning methods based on neural, fuzzy, pattern recognition and statistical ideas. Despite significant progress in various theoretical and applied areas many problems remain unsolved, comprehensive theory presenting a unified perspective on various learning methods is missing, large component-based data mining packages contain now hundreds of learning methods, input transformations, pre- and post-processing components that may be combined in more than 10 million ways. Although there is "no free lunch" (no single method is the best for all test problems) several methods that are close to optimal may be found through meta-learning based on heuristic search in the space of all possible learning models. Various model spaces are considered as the basis for meta-learning: 1) similarity-based algorithms that identify prototypes and optimize similarity measures; 2) heterogeneous systems, that include neural, fuzzy, prototype-based and hierarchical partitioning algorithms; 3) general transformation-based systems.
Most general implementation of meta-learning is possible within transformation-based learning paradigm that unifies most of computational intelligence research and shows how to solve the "crises of the richness" selecting optimal transformations to minimize complexity and maximize quality of the resulting data models. Meta-learning systems learn simplest data models that many sophisticated methods miss, generate multi-resolution models whenever needed, and solve difficult, highly non-separable problems that are beyond capabilities of current state-of-the-art algorithms, including neural networks and support vector machines. In contrast to backpropagation that tries to achieve linear separability in one shot additional criteria are defined after each transformation to create appropriate internal representations. Visualization of learning dynamics in transformation-based systems shows how to set simpler goals for learning, for example k-separability instead of linear separability.

This tutorial will include:
1) Review of various approaches to meta-learning.
2) Meta-learning as search in the model space - general idea.
3) Model space based on similarity-based learning.
4) Model space based on composition of transformations creating internal representations.
5) Complexity control of the search process.
6) Some implementation details.
7) Examples and some lessons learned from the use of meta-learning.


References:

1. Jankowski N, Grabczewski K, Increasing efficiency of metalearning machines with complexity control. Journal of Machine Learning Research (submitted).
2. Maszczyk T, Grochowski M, Duch W, Discovering Data Structures using Meta-learning, Visualization and Constructive Neural Networks. Book chapter, in print, Springer 2009
3. Jankowski N, Grąbczewski K, Building meta-learning algorithms basing on search controlled by machine’s complexity and machines generators. IEEE World Congress on Computational Intelligence, IEEE Press, pp. 3600–3607, 2008.
4. Duch W, Towards comprehensive foundations of computational intelligence. In: W. Duch and J. Mandziuk, Challenges for Computational Intelligence. Springer Studies in Computational Intelligence, Vol. 63, 261-316, 2007
5. Duch W, Setiono R, Zurada J.M, Computational intelligence methods for understanding of data. Proc. of the IEEE 92(5) (2004) 771- 805
6. Duch W, Grudziński K, Meta-learning via search combined with parameter optimization. Inteligent Information Systems, Advances in Soft Computing, Physica Verlag (Springer) 2002, pp. 13-22
Project page: http://www.is.umk.pl/projects/meta.html

Włodzisław Duch web page)

Krzysztof Grąbczewski received his PhD degree in 2003 from Systems Research Institute, Polish Academy of Sciences. He is working as assistant professor at Department of Informatics, Nicolaus Copernicus University, Toru , Poland. His scientific interests include broad spectrum of computational intelligence algortithms and applications, especially all the aspects of advanced meta-learning. He has published over 50 reviewed papers including book chapters, journal articles and peer-reviewed conference papers. His data mining skills have been confirmed by the 3rd place in NIPS 2003 Feature Extraction Challenge, and the 1st place in ICAISC 2006 Handwritten Digit Recognition Contest at The Eighth International Conference on Artificial Intelligence and Soft Computing.

Tutorial introduces a novel approach in machine learning i.e., data mining based on a local design of machine learning models (tools). In particular it introduces the Adaptive Local Hyperplane and Locally Linear Support Vector Machines (LLSVMs) which have shown powerful capabilities for both pattern recognition (classification) and high dimensional function approximation (regression). The experimental results on about two dozens of benchmarking data sets demonstrate that the proposed algorithm outperforms, on average, all the other various classifiers and regressors. The novel algorithm is a result of an attempt to create maximal local margin in the original, weighted, input space and not in the artificial, so-called, feature space. The tutorial will be supported by computer simulations. Participants are welcomed in bringing their (small for the sake of time available) data sets for testing LLSVM capacities.

Vojislav Kecman is with a CS Department at the Virginia Commonwealth University in Richmond, VA, USA, where he directs the Learning Algorithms and Applications Laboratory (LAAL). He was Fulbright Professor at MIT, Cambridge, MA; DFG Professor at TH Darmstadt; DAAD Konrad Zuse Professor at FH Heilbronn, FHTW Berlin and SWFH Soest; Research Fellow at Drexel University, Philadelphia, PA and at Stuttgart University, as well as the associate professor at The University of Auckland and Zagreb University. He authored several books in the areas of machine learning (data mining) and in the fields of mathematical modeling and simulation of system dynamics (see, www.support-vector.ws and www.learning-from-data.com). Dr. Kecman is a member of IEEE.

Qi Li and Raied Salman are PhD students at the VCU's SoE, Dept of Computer Science.