Book details of 'Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification'

Title	Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification
Author(s)	Jonathan Zdziarski
ISBN	1593270526
Language	English
Published	July 2005
Publisher	No Starch Press

---

Back to shelf Computer security
Amazon.com info for Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification

Score:

Vote for this book

The Virtual Bookcase Reviews of 'Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification':

Reviewer Rob Slade wrote:
The preface states that the book is for those seriously interested in spam identification technologies, and concentrates on Bayesian and related statistical filtering Part one is an introduction to spam filtering. Chapter one reviews the history of spam, although many of the early entries are simply annoyances or chain letters rather than the commercial or fraudulent items considered under the banner today, and the author does not seem to realize that 419 scams predated email by a considerable margin. A look at the development of spam filtering (excluding Bayesian) is presented in chapter two, along with some non-filtering. Bayesian analysis is explained in chapter three, and the statistical filtering basis is outlined in chapter four. The fundamental actuarial core is expanded in part two. Chapter five covers message coding. Tokenization, chunking characters into identifiable items, is examined in chapter six. Tricks spammers use to avoid filters, and the solutions to avoid falling for them, are outlined in chapter seven. Storage and performance issues raised by the data rules required by statistical filters are addressed in chapter eight. Chapter nine looks at aspects of scaling to systems supporting large numbers of users. Part three deals with advanced concepts in statistical filtering. Chapter ten delves into testing which, because of the individual and adaptive nature of Bayesian filtering, presents unique challenges. Tokenization is revisited in chapter eleven, in more advanced forms. Markovian discrimination, with it's examination of stateful entities, is explained in chapter twelve. Having noted many kinds of features in the book, chapter thirteen explores ways to reduce the items used (and data required) while maintaining accuracy. Collaborative rule- building with other users, groups, or systems is reviewed in chapter fourteen. As the preface implies, this is *not* a book for users who just want to install POPFile (although that and other programs are explored in an appendix). For those who are seriously involved in managing and developing spam filtering, however, the book does provide very useful advice, pointers, and research. copyright Robert M. Slade, 2005
Add my review for Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification

Book description:

Join author John Zdziarski for a look inside the brilliant minds that have conceived clever new ways to fight spam in all its nefarious forms. This landmark title describes, in-depth, how statistical filtering is being used by next-generation spam filters to identify and filter unwanted messages, how spam filtering works and how language classification and machine learning combine to produce remarkably accurate spam filters. After reading Ending Spam, you'll have a complete understanding of the mathematical approaches used by today's spam filters as well as decoding, tokenization, various algorithms (including Bayesian analysis and Markovian discrimination) and the benefits of using open-source solutions to end spam. Zdziarski interviewed creators of many of the best spam filters and has included their insights in this revealing examination of the anti-spam crusade. If you're a programmer designing a new spam filter, a network admin implementing a spam-filtering solution, or just someone who's curious about how spam filters work and the tactics spammers use to evade them, Ending Spam will serve as an informative analysis of the war against spammers. TOC Introduction PART I: An Introduction to Spam Filtering Chapter 1: The History of Spam Chapter 2: Historical Approaches to Fighting Spam Chapter 3: Language Classification Concepts Chapter 4: Statistical Filtering Fundamentals PART II: Fundamentals of Statistical Filtering Chapter 5: Decoding: Uncombobulating Messages Chapter 6: Tokenization: The Building Blocks of Spam Chapter 7: The Low-Down Dirty Tricks of Spammers Chapter 8: Data Storage for a Zillion Records Chapter 9: Scaling in Large Environments PART III: Advanced Concepts of Statistical Filtering Chapter 10: Testing Theory Chapter 11: Concept Identification: Advanced Tokenization Chapter 12: Fifth-Order Markovian Discrimination Chapter 13: Intelligent Feature Set Reduction Chapter 14: Collaborative Algorithms Appendix: Shining Examples of Filtering Index About the Author Longtime spam-fighter Jonathan A. Zdziarski maintains DSPAM, a high profile, next-generation spam filter that’s up to 99.985% accurate and has been hailed by Slashdot and Wired News. He lectures widely on the topic of spam and is a foremost researcher in the fields of algorithmic theory and neural networking.