Boosting – Foundations and Algorithms

Excellent book. Convers all important aspects of Boosting. Very-well written, clear and detailed explenations. A must!

It's a kind of problem I come across way too often: in trying to determine whether a specific condition exists, a number of symptoms might help the diagnosis. But, some of the symptoms can also appear when something else is going on, instead. On the other hand, not all of the symptoms necessarily appear when the condition in fact is active. The question then becomes, given a number of indicators that have some diagnostic value, and given that all of them are inaccurate some of the time, how do I combine the indicators' answers to get the best diagnosis of that condition? This book proposes "Boosting" as the answer. Start with remarkably few technical requirements on the diagnostic indicators, plus some number of cases in which the condition's presence or absence is already known. Given that, Boosting iteratively determines the weight to assign each indicator. One requirement is that each indicator suggests presence or absence of the condition at least somewhat differently than random guessing - and being wrong most of the time is just as useful as being right more often than not, since the algorithm automatically assigns negative weights to such indicators. After that, the authors present rigorous development in a number of directions. I emphasize "rigor" - this text offers detailed development, analysis, and formal proof of the algorithm and its properties, far beyond the needs of someone who just wants to implement the technique. Implementable detail is there, but you'll spend a fair bit of time teasing it out of the dense notation used here. Then, once basics have been established, the discussion branches out. The authors offer game-theoretic analysis of the algorithm, along with comparisons to related optimization techniques. After tightening some of the technical requirements, they apply it to decision trees with binary outcomes, to N-ary classification problems for N>2, and to more complex kinds of tasks, always with the same mathematical rigor. Frankly, it's a bit much for someone just looking to get the broad picture or someone looking to code something up fast and try it out. But, it's not meant for those readers. To get the most out of it, you should come prepared to take your time, work out the meaning of each equation, follow the development carefully, and maybe even do some of the exercises. If you've already plowed through some "yellow books," you'll know what I mean, even though this doesn't come from the Springer yellow series. For the prepared and patient reader, this has my highest recommendation -- wiredweird

Un buon libro per chi vuole capire e approfondire l'argomento del boosting. Tratta l'argomento da varie angolazioni in maniera abbastanza accessibile, anche se alcuni capitoli sono abbastanza impegnativi. Tratta principalmente la parte teorica con qualche accenno anche agli algoritmi che ne derivano. Non penso sia un libro adatto a chi cerca solo un manuale pratico sull'argomento.

Perfect.

It's a quite comprehensive book, describing lots of different ways to look at the AdaBoost family of algorithms. For all I can tell, the authors have collected all the state-of-the-art knowledge about boosting at the time the book was written, from the publications developed both by them and by the other people. So if you want to learn pretty much everything about boosting (up to the publication date), this is the book to read. But be prepared that it's not a quick and casual introduction, it's a collection of the in-depth mathematical papers. It's a book that takes a very long time and much effort to read thoroughly and understand. You can skim over the proofs but it still takes a long time, after all it's pretty much everything known about boosting. I actually highly recommend not spending too much time on the proofs when you read it for the first time. This will give you a good overall picture, and then if you want to go deeper, read the book for the second time, the mathematics will make more sense on the second reading. You can also skip chapters depending on your interests, if you're not out to learn everything about boosting. Probably the only really annoying thing from the engineering standpoint is that the algorithms in the book are what the mathematicians and the fans of functional programming call "algorithms", not the algorithms in the normal engineering sense. It takes some deciphering to turn them into a straightforward readable and understandable form. Bug again, it's a book about math, not engineering. I've had some of the deciphering I've done posted to a blog but I probably can't post a link to it here.