Matevž Gačnik's Weblog

This is an executive email from billg, regarding Microsoft's efforts to stop the cockroach of the internet [Source]. If God would choose between various mailers, I hope he/she would choose Outlook 2003, which will include an efficient spam filter (I hope).

I categorized this to Personal and Other, since it nags me and others.

Towards a spam free future

Email is such an integral part of business and everyday life today that we tend to forget how recently it became popular. The first email program was developed in the early 1970s, but for two decades the technology was hardly used except by computer scientists, researchers and hobbyists.

Not until the mid-1990s, when the growing popularity of personal computers converged with easy access to the Internet, did email become truly pervasive as a way to communicate at work, with family and with friends. Today, email is as easy to use as the telephone, and just as vital for keeping people in touch, and for improving business productivity.

Yet email's popularity has produced one very troubling side effect: spam. Unsolicited commercial email is a spreading plague that feeds off the unique power of the Internet to connect hundreds of millions of computer users around the world, at virtually no cost. Generally unwanted and often pornographic or with fraudulent intent spam is a nuisance and a distraction. Like almost everyone, I receive a lot of spam every day, much of it offering to help me get out of debt or get rich quick. It's ridiculous.

What's more, spam is a drain on productivity, an increasingly costly waste of time and resources for Internet service providers and for businesses large and small. It clogs corporate networks, and is sometimes a vehicle for viruses that can cause serious damage.

Spammers often prey on less sophisticated email users, including children, which can threaten their privacy and personal security. And as everyone struggles to sift spam out of their inboxes, valid messages are sometimes overlooked or deleted, which makes email less reliable as a channel for communication and legitimate e-commerce. Spam is so significant a problem that it threatens to undo much of the good that email has achieved.

At Microsoft, as part of our drive to create a more trustworthy computing environment, we are significantly stepping up our efforts to fight spam and its pollution of the email ecosystem. Although there is no easy fix, we believe that spam can and must be dramatically reduced. We're working toward this goal on many fronts, through technological innovation and in partnership with other leaders in industry and government.

Creating New Anti-Spam Technologies and Strategies

Because spam affects consumer and business users of many Microsoft products and services, we have been working for several years on spam filters, and on tools that enable people to block unwelcome senders and designate others as safe. These tools have become available in recent versions of products such as MSN, Hotmail, Exchange and Outlook.

Recognizing the increasing urgency of the issue, we recently created a new Anti-Spam Technology and Strategy Group that brings together specialists from across the company and integrates all of our anti-spam strategy and R&D efforts.

We are building on advanced work at Microsoft Research in fields such as machine learning the design of systems that learn from data and grow smarter over time. This kind of technology is vital to the fight against spam because every defensive action causes spammers to change their attack. Technology, to be effective, must continuously adapt, without requiring a team of people to examine messages one by one. With machine learning, a "smart" spam filter can automatically adjust to spammers' shifting tactics.

A smart filter can also be customized to suit the preferences of an individual user. This is important because, although a lot of spam is pure junk, not all of it is clearly distinguishable based solely on broad, global criteria. Deciding precisely where to draw the line must ultimately be up to the individual. However, a smart filter can learn from a user's personal preferences to create a unique, anti-spam immune system that is much harder for spammers to work around.

Already, filters on the servers at MSN and Hotmail block more than 2.4 billion messages a day, before they ever reach our customers' inboxes. And to help deal with mail that survives this first hurdle, MSN 8 software includes a smart filter that becomes more effective over time as it learns the characteristics of mail that an individual customer regards as spam. This month, we updated MSN 8 with further improvements in its spam technologies, giving customers an option to block offensive images in email, and adding the ability to filter mail in languages besides English. We will offer more technology advances in a new release of MSN software later this year.

Meanwhile, we are working to create new anti-spam technologies that are even more precise, easier to use, and adaptable. And we are working to integrate them into more of our products, particularly Outlook and Exchange.

To help, we have assembled a massive and still growing database of spam, collected from volunteers among our millions of MSN and Hotmail subscribers. This database will prove invaluable later this year when we release Outlook 2003, which will include a new, smart filter that will access the database to recognize and block spam more effectively. The filter in Outlook 2003 also will be updated frequently and easily, as with Windows Update today.

Exchange 2003 includes a host of anti-spam features, including an Application Programming Interface that enables third-party providers of spam filters to easily supply solutions for Exchange customers. We plan to add our own smart filter and continue building more anti-spam capabilities into the Exchange messaging infrastructure. Our goal is to do everything we can to secure email systems with servers that monitor and control the points of entry.

As we develop new technologies, stemming the tide of spam also requires a multi-faceted approach that includes industry self-regulation, effective and appropriate legislation, and targeted enforcement against the most egregious spammers. It also calls for cooperation among the major players in the email ecosystem. In April, we joined with AOL and Yahoo! in announcing a wide-ranging set of initiatives to fight spam together. Since then, Earthlink and others have joined the effort, which involves promoting business guidelines, best practices and technical standards that can help curb spam sent or received via any online service or computing platform.

Stopping Spam At the Source

Every major provider of email services has rules against spamming. Microsoft puts significant resources into investigating consumer complaints about spam that may have originated from accounts on MSN or Hotmail. We are firm in shutting down those who violate our anti-spam account policies.

There are other challenges. For example, spammers set up many different email accounts to avoid detection, and, once detected, they move to other services. To put an end to this shell game, we are taking steps to prevent spammers from creating fraudulent email accounts in bulk. We also are working with other service providers to share information so that we can keep tabs on roving spammers and shut them down more effectively.

Government policymakers also have a role to play. We support U.S. federal legislation that would strengthen the ability of service providers to shut down spammers by suing them on behalf of customers. And we believe that the use of automated searches to harvest addresses published on the Web and in Internet newsgroups should be banned, making it much more costly and difficult for spammers to assemble mailing lists.

Bringing Spammers into the Sunshine

Government and industry working together also can put an end to spammers' deceptive practices. Spammers go to great lengths to conceal or "spoof" their identities. They relay their mail through multiple servers to hide its origins. They open multiple accounts and change to new ones frequently to avoid drawing the attention of service providers, and to improve the chances of their mail passing through spam filters. They lure unsuspecting readers by faking sender addresses ones that appear to be someone inside the recipient's company, for example.

Microsoft is working with others in the industry to identify and restrict mail that conceals its source. For example, we are working toward a system to verify sender addresses, much as recipients' addresses are verified today. The Internet addresses for all incoming mail servers are published as part of the Domain Name System, the Internet's distributed directory. That's how mail gets to the right destination. If domain administrators could also publish the addresses of their outgoing mail servers, then the receipt of a suspected forgery could trigger a relatively simple, automated verification process. Incoming servers would then be able to confirm whether senders are who they say they are.

To help fight fraudulent or otherwise illegal spam, we are cooperating with other service providers to create better mechanisms for preserving electronic evidence of spammers' activities. And we are coordinating civil lawsuits and other enforcement actions for greatest impact. On June 16, Microsoft filed 15 lawsuits in the United States and the United Kingdom against companies and individuals alleged to be responsible for billions of spam messages sent in violation of state and federal laws.

These efforts would be helped and consumers would benefit from legislation that would include clearer prohibitions against using misleading sender addresses and other false header information.

Isolating Spam

Part of the challenge in curbing spam lies in accurately identifying legitimate commercial email. What would help are guidelines defining, for example, whether and when an email is legitimate based on a previous business relationship between the sender and recipient. By drawing a clear line between spam and legitimate mail, guidelines would enable spam filters to work more precisely, and make it easier for honest businesses to stay on the right side of the line.

Developing such guidelines is the focus of talks involving Microsoft and other technology leaders, responsible marketers and consumer groups. We favor the idea of setting up independent email trust authorities to establish and maintain commercial email guidelines, certify senders who follow the guidelines, and resolve customer disputes. Similar authorities already help in protecting people's privacy online, with organizations such as TRUSTe and BBBOnline providing certification for Web sites and companies that follow guidelines on the use of customers' data.

Self-regulation needs to be supported by strong federal legislation that empowers consumers without threatening the vitality of legitimate e-commerce. Our proposal is to create a regulatory "safe harbor" status for senders who comply with guidelines. The guidelines would be subject to approval by the Federal Trade Commission. Compliance would be confirmed by a self-regulatory body. Senders who do not comply would have to insert an "ADV:" label, for advertisement, in the subject line of all unsolicited commercial e-mail.

Computer users could then customize their spam filters to either accept "ADV:"-labeled mail or automatically delete it. Enabling consumers to regain control of their inboxes in this way would dramatically reduce the volume of spam by creating strong incentives for businesses to make sure their communications are consistent with best-practices guidelines developed by industry itself.

Changing the Landscape, Soon

These and other efforts across many fronts should lead to a world where we are less troubled by spam. As less of it reaches recipients and violators face stiffer sanctions for illegal activities the financial incentives for spammers will decrease, and spamming will lose much of its appeal.

At Microsoft, we're strongly committed to the goal of ending today's spam epidemic.

Bill Gates