Machine Learning for Sequences and Structured Data : Tools for Non-Experts
Project Summary This project proposes to dramatically improve the ability of people who are not experts in machine learning to design and automatically train models for analyzing and transforming sequences and other structured data such as text, signals, handwriting, and biological sequences. Working in the context of recent successes with conditional random fields (CRFs) and other conditional models of structured data, the proposed work will achieve its goals through scientific advances in model definition and combination, robust parameter estimation, and data-efficient training procedures, supported by an innovative compositional software architecture. Intellectual merit: Automatic classification and function fitting (regression) are mature techniques used in many scientific, engineering, and national security applications. Free and commercial software packages allow novice users to train with reasonable confidence of attaining useful results, and to use these results in sophisticated predictive and decision-making systems. However, for sequence and structured data problems, while task-specific machine-learning software has become increasingly available (e.g., speech recognition and genomics software tools), there are no modular, easy-to-use packages for domain experts working in science and engineering problems such as predictive data modeling of spatio-temporal patterns of brain activity or extraction of conceptual and citation networks from the scientific literature, or applications in national and homeland security such as the extraction of social networks from the open Web, or dynamically recognizing suspicious temporal patterns in network logs. The proposed work will make available for the first time a user-oriented software toolkit for integrated analysis and transformation of sequential and graph-structured data, which will be a major innovation in the data, models, and communications technical focus area. What makes this possible is the convergence of three scientific innovations in learning from structured data. First, powerful, trainable analyzers and transformers for sequences and other structured data can be built by combining simpler conditional models with general composition and product operations based on the theory of weighted automata. Second, a range of capacity-control techniques (feature induction, margin maximization, Bayesian automatic relevance determination) can be generalized to these complex models to control overfitting without the need for extensive hyperparameter adjustments. Third, the need for fully annotated training data can be reduced by combining partial evidence for multiple sequences into a single graph labeling problem. Together, these ideas provide a framework for flexibly specifying and learning parameters for multistep probabilistic transformations from complex data to their structured representations, and for designing and documenting efficient and usable software for …