Application of a letter code to protect against spam. An overview of the anti-spam system Kaspersky Anti-Spam. Checking internet message headers

- € 55-250 million annually. 60% world mail traffic.
50-75% from all Russian mail traffic. Modern antispam tools filter 85-98% of spam. The global market for antispam filters and services in 2004 was approximately $ 500 million (IDC estimates).
Most antivirus vendors have included anti-spam components in their products. There were several purchases of anti-spam software vendors by anti-virus companies during the year (notably the $ 340 million purchase of BrightMail by Symantec). In Russia, antispam filters have been installed by most of the holders of public mail services and most of the providers, which made it possible to remove the urgency of the problem of spam for their clients. The undoubted leader in Russia in terms of sales and the number of protected mailboxes is the Spamtest technology.
1. PREVENTION The # 1 anti-spam tool is to protect your email address. No spammers will know your address - no spam. Highlight your address on the network, you will have to throw it away and start a new one, it will only be a matter of time. And, as a result, tell all your friends and partners a new address again, and you may lose a number of contacts. To prevent this from happening Get two email addresses. One address for long-term contacts (do not shine it on the network).
Another address for making contacts, using the network (chats, message boards, etc.).
Then there should be no spam on the first address, because it is not known on the network.
When spam goes to the second address, just throw it away and start a new one.
2. CHOOSING A NAME People tend to get the most concise address. Let's say sergey@ mail.ru is cool and what a pity that all simple addresses are already taken. Rest assured that on [email protected] spam is pouring in without stopping. It's cool to have a laconic name for the site, but you still have to tell the email address to everyone personally, let it be from numbers or an original, not a hackneyed word. By the way, for this purpose the leading mail gmail.com registers names of at least 6 characters. All short names have long been included in spam lists.
3. HTML SPECIAL CHARACTERS The simplest and most commonly used method of protecting against spiders is to encode the email address using special HTML characters. Instead of a dog - @ ... But today this method is hopelessly outdated.
Robots can easily find such addresses.
4. JAVASCRIPT On the Anti-Spam Code Generator page, you can generate your own script. Since these scripts for hiding the address are crafted, they are very motley and there are no programs that would be able to fetch email from JavaScript. It is the most reliable address protection on the network today.
5. ANTISPAMMERS But, what if you are lit up, or you are so famous that it is impossible for you not to get noticed, then you cannot do without an anti-spammer. There are many antispam programs that you can download online.
What I do not advise.
I came to the conclusion that all these antispams are small and weak, and a sensible antispam cannot be handled by a person, only a reputable company, such as Gmail.com, can do that. Their spam remains on the server, you can always go in and correct it. So my strong advice: get yourself a mail on Google.
I have never seen a better spam filter, all spam remains on the server, which, if desired, can always be viewed and corrected. Antispammers do not completely solve the problem, but make life easier in the problem.
6. POCKET PC AND WAP Spam has reached this level, but today there are fairly reliable means of protection. Therefore, the development of this issue is not relevant.

The following technologies are used to protect mail servers:

There are two main methods for protecting spam: protecting against spam when the mail is received by the server, and separating spam from the rest of the mail after it is received.

Blacklists. Blacklists contain IP addresses from which spam is sent.

Greylisting or greylisting. The way greylisting works is based on spamming tactics. As a rule, spam is sent in a very short time in large quantities from some server. The job of the greylisting is to deliberately delay the receipt of emails for a while. In this case, the address and time of transfer is entered into the gray list database. If the remote computer is a real mail server, then it must keep the message in the queue and repeat the transfer within five days. Spambots, as a rule, do not save letters in the queue, so after a short time they stop trying to forward the letter. When re-sending a letter from the same address, if the required amount of time has passed since the first attempt, the letter is accepted and the address is entered into the local white list for a sufficiently long period.

DNSBL (DNS blacklist)- lists of hosts stored using the DNS system. The mail server contacts the DNSBL and checks for the presence of the IP address from which it receives the message. If the address is in this list, then it is not accepted by the server, and the corresponding message is sent to the sender

Message limit... Setting a limit on the number of messages.

Program SpamAssasin(SA) allows you to analyze the content of an already delivered message. SpamAssassin comes with a large set of rules that determine which emails are spam and which are not. Most of the rules are based on regular expressions that match the body or header of the message, but SpamAssassin uses other techniques as well. The SpamAssassin documentation calls these rules "tests".

Each test has some "cost". If the message passes the test, this "cost" is added to the total score. The cost can be positive or negative, positive values are called "spam", negative "ham". The message goes through all the tests, the total score is calculated. The higher the score, the more likely the message is spam.

SpamAssassin has a configurable threshold, upon exceeding which a message will be classified as spam. Typically, the threshold is such that a message must match several criteria; triggering only one test is not enough to exceed the threshold.

The following technologies are used to protect sites from spam:

1. Captcha picture... Those. the user is shown arbitrary text that the user must enter to perform any action.

2. Text captchas- the subscriber must enter the answer to the proposed question to confirm his actions.

3. Interactive captcha- a less common, but very useful form of protection. For example, to confirm actions, the user will be asked to solve an easy jigsaw puzzle - for example, to assemble a picture from three or four parts.

Introduction to the problem

We all know what spam is because we have either encountered it or read about it. We all know how spammers collect email addresses. It is also no secret that spam cannot be completely defeated. The problem is how to maximally protect users leaving their contact details on your site with minimal effort.

Previously tested methods of protection

The biggest threat to mailboxes is represented by programs that download sites and take mail addresses from the text of pages. They either download only your site, or roam like search engines all over the web. If your site is small, the protection of this autocorrect text is quite enough:

] + href =) ([""]?) mailto: (+) () @ ".
"() (+. (2,4)) 2 ([>]) ~ i", "1" mailto: [email protected]"
onMouseover = "this.href =" mai "+" lto: 3 "+" 4 "+"% 40 "+" 5 "+" 6 ";" 7 ", $ text);?>

Unfortunately, it won't work if you have a large site. Let's say spectator.ru, the author of which was one of the first to use this method. If I were a spammer, I would have climbed into personal settings, checked the "do not show ears" checkbox, 1000 reviews per page, and caught the cookies by Proxomitron. Then, with a rocking chair or a php script, I would download the pages with comments (substituting cookies with settings) and using a regular expression, I would catch the addresses. I would get a small base for advertising mailing.

There were also a couple of protection methods in which the mailto: link is automatically replaced with a cookie, but the effect remained the same - when you clicked on it, the system client would create a letter to the desired address. Both of them did not stand up to criticism.

Meet the iron grip

Obviously, it is difficult to think of another method of protection besides the already tried one - providing a form on the site to send a message. Let's start designing it. The advantages of this method are obvious: no one will be able to get the addresses for their spam database from your site. It will not work to send messages by hiding their address, as spammers do - the web server will fix its IP address. Lists of public anonymous proxy servers are regularly updated, and it is easy to block access from them.

Form sender

Let's start with him, because this is the hardest part.

When installing a form sender on a website, it is important to protect it from hooligan attacks, which can be no easier than spam. Therefore, we will have to make great efforts in this direction.

First, we will protect ourselves from stupid double clicks and sending many identical requests. The idea is this: the message will not be sent if the user has not opened the page with the form before, and by opening the page with the form, you can send the message only once. This can be done using PHP's built-in sessions. When opening a page with a form, we will start a session in which we will save a variable, say $ flag. The session identifier is displayed as a hidden element at the very end of the form. The user enters a message and submits the form. Upon receiving the form, the script starts a session and checks the existence and value of the $ flag variable. If the variable does not exist, then this is a second click, the letter is not sent and an error message is displayed. If there is a variable, and the form data suits us (the required fields are filled in), the script sends a letter and deletes the session.

Secondly, we will protect ourselves from smart bullies by writing down message logs. If the user submits a correctly filled form, the script will look in the logs and check what is there. So, you need to ban

* send messages to the same address more often than a certain period
* send the same text to different addresses
* and just use the form sender too often - say, no more than 10 messages per day per user

We display the session ID at the very end of the form so that the hacker needs to download the entire form and parse it, which is more difficult than just sending HTTP requests. Naturally, the sender will issue messages about errors in writing a message, a request to indicate a return address, etc.

The resulting sender code was too large to be included in the text. It is archived on the site. It seems that the script is running and sending messages.

Replacing addresses in text

Now the form sender is ready, and you need to replace all emails with links to it. Of course, you shouldn't do this manually. For myself, I wrote a script that automatically replaces addresses with links to the sender.

... Cons: more time for placing links (compensated by a catalog of links), the user, hovering over the link, does not see which address it will go to. (Dmitry Smirnov, "Ideal author's project, hypertextuality")

All the mentioned disadvantages can be easily eliminated if you use a code similar to the one that I will now describe and show.

There is nothing complicated here, if these are links, then "more time for placement" is not required. On my site, I use an engine script that is called by all pages, so it is not a problem to add or call a code that replaces addresses from it. Postal addresses are both written and written directly in the text of the pages, but before being displayed to the user, they are replaced with the desired text. Compiling a database of links or postal addresses is not a problem.

So what does an address substitute do. It looks for "mailto:" links in the text, selects addresses from them, sends a request to the database to count (count (*)) how many addresses from those on the page are in a special table. If there are new addresses on the page, then their number will be greater than the query result. In this case, a query is made, in which the values of addresses are selected, and those already existing in the table are excluded from the list. The remaining list is sent to the table with an INSERT query.

As for the ID addresses, in my opinion, it is better to use something that the site visitor could not pick up. Imagine, the link /email.php?id=10 leads to the sender? What a temptation to put in there 11, 12, etc. and try to send them all a message. Therefore, I decided to use the md5 hash from the addresses as identifiers. Hardly anyone will undertake to select a hash. In the case of a directory of links, you can do without ID, but then you have to select all values from the base, and to replace addresses with their hashes, everything is much easier.

A command of the form

] + href =) "." ([""]?) mailto :( [email protected]+ ".". (2,4)) 2 (. *?>) ~ Ie "," "12" /email.php?email= ". Urlencode (md5 (" 3 "))." "4" " , $ text);?>

... which replaces addresses with their hashes. I did not dare to replace the rest of the addresses in the text with links, but made a simple replacement with addresses like vasya_at_pupkin_dot_ru. The autocorrect code is also in the archive.

Outcome

Hiding email addresses from visitors is pretty easy. The autocorrect mechanism does not require additional effort, and you can write the pages of the site further as if nothing had happened. Difficulties arise when protecting the form sender from web bullies. This protection requires a lot of effort and complex code, so I have not yet used the written code on the site. You can download the archive with a substitute for addresses and a form sender, but I beg you: do not put it on your site in the form in which you downloaded it, I myself do not know how reliably it works.

Hello dear readers of the blog site. Surely you have already encountered spam, even if you did not know exactly what this mess is called. And for this it is not even necessary. Spam with enviable regularity is poured into an ordinary mailbox - these are the very countless advertising brochures that are slipped to you in the hope that you will order or buy something.

So, spam is just is the annoying one, climbing from all sides advertising(but not only advertising - sometimes worse). You didn’t order it, you don’t need it, but it rushes and rushes from all cracks. In this heap, the information you need can easily get lost and you have to spend a lot of time filtering it out.

In our computer age, the main source of spam. And besides advertising spam mailings can be dangerous both for your wallet (phishing, social engineering, wiring) and for your computer (viruses, worms, Trojans).

What does the word Spam mean, what it is and how you can deal with it, you will learn by looking at this small note. Hope it will be interesting (well, definitely useful - I promise you that).

Spam - what is it

WHAT DOES SPAM MEAN IN GERMAN?

Itself Spam word comes from the name of canned meat, which was fiercely advertised after the end of the Second World War (obviously, it was necessary to urgently sell the soldiers' meat rations).

The advertising was so aggressive, all-embracing and ubiquitous that this word (and the associated “sediment”) came to my mind, but this time about the intrusive advertising that appeared along with it in conferences (then still in the feed, if anyone remembers).

The word stuck, especially since the obtrusive advertising did not become less, but rather the opposite. When e-mail gained popularity, unauthorized mass advertising and malicious mailing of letters became commonplace. Such mailings were profitable for spammers, because the necessary information was communicated to a large number of people without special costs.

But e-mail is not limited to... Spam in a personal social networks, on forums, instant messengers, message boards, blog comments, open for editing and adding text by everyone. They also send spam to your phone, for example, by advertising calls or by sending advertising SMS messages.

Where can you find him on the Internet

Email Is just a klondike for spammers. Bulk mailings can sell anything, you can cheat and rob, you can infect computers and send out worms. Databases for mass mailings are collected independently (using programs), or purchased from those who are professionally involved in this.
Forums, comments blogs, wikis and message boards - in fact, everyone is allowed to leave messages and it is difficult for spammers to resist not to sleep. This is not always advertising - often in this way, webmasters try to get a free link to their site so that it stands higher in the search results of Yandex or Google for various queries. This brings them traffic and money.
Social networks and dating sites - spam is very common among incoming personal messages. It is also available in the comments to the messages.
Messengers (like) are also subject to this scourge.
SMS- messages from people unknown to you of an advertising nature. Probably everyone is familiar.
Search spam- a rather specific thing, but familiar to everyone. You had such a thing that you enter a request into Yandex (Google), and in the answers there are entirely sites with some kind of bilibery that have nothing to do with the case. These are the so-called doorways (sites with automatically generated useless texts). They spam the results of search engines, but they make money on visits to them by visitors (in different ways).

Those. all letters in your mailbox (or messages on the forum, blog comments, messages in a personal message) of an advertising or other nature from a sender unknown to you - this is spam... True, the mailings to which you subscribed can also be annoying, but you can still unsubscribe from them (refuse to receive them further).

Spam by itself is annoying and annoying, because it litters where it appears. Most often, this is your inbox, and it can be difficult to separate the wheat from the chaff when there are too many unwanted messages. But spam can also pose a real threat to both you and your computer. Let's deal with this with everything.

Understanding the types of spam (harmless and dangerous)

Now and further, I will mainly talk about spam messages received by e-mail, for it is this channel that is considered especially susceptible to this phenomenon. Everything else is not so launched and is not so actively used by spammers, and this does not apply to all network users. But the problem of their protection from unwanted messages is acute.

Types of harmless spam

Advertising of legal goods and services- in this case, the business owner simply chose spam mailings as one of the channels to attract customers and buyers, because it is not expensive, quickly and brings results. Naturally, he does not really think about the moral (or rather immoral) side of this matter.
Prohibited advertising- there are goods and services that are prohibited by law to advertise, and for them spam mailings can be the main channel for attracting customers. By the way, spammers also advertise their services using mass mailings, because this activity is prohibited by law. This also includes advertising defaming competitors, because it is also prohibited by law (praise yourself, but do not scold others).
Impact on public opinion- very often spam becomes a good choice for those who are trying to manipulate public opinion in the right direction. It can be politics, as well as. Letters in general can be sent out supposedly from some person in order to compromise him, change his opinion about him or use his authority for profit. But personally, this does not pose any danger to you.
Letters that ask for forwarding to others- it can be a variety of so-called "letters of happiness" (send it to 10 friends and you will be happy), or asking to send information to friends for some other reason. Often such letters are used by spammers to collect or replenish the existing base of emails for subsequent mass mailings.

High-risk spam - what it can be

If regular (harmless) spam can only slightly affect your peace of mind and mental health, then the examples below can be very dangerous for your wallet or computer. And this is not a joke.

I myself (the one who teaches and teaches everyone) a couple of years ago "bought" this type of letter (phishing) and parted with several thousand rubles (read ""). I just wound myself up and did everything “on the machine” just to get behind, but when I realized it, it was too late.

- a very effective method of deceiving not only gullible, but also simply busy or inattentive people (there is a hook for anyone). They send you a letter supposedly from your bank, e-money service or somewhere else. In this letter, you are sure to be overwhelmed with something (unsettled) and asked to log in to the site to solve the problem. You log in, but the site will be fake (although it looks like a real one) and the data you provided is immediately used to steal all your money.
Nigerian letters- you will learn that you can get a large amount of money (different pretexts - from unexpected inheritance to helping a prince in exile). You don't believe for a long time, but they convince you. When you believe it, you will be asked to transfer some money for "related expenses." You translate and no one will bother you anymore.
Viruses, worms, trojans- the letter may contain an attached file with a malicious program (or a link leading to a website with a virus). She can immediately cause problems with the operation of the computer, or she will sit quietly in a corner and carefully write down all the passwords, logins and other useful things in the household that you enter. Among other things, worms are also able to send themselves to your friends at the addresses found in the E-mail contacts (as they will then ...).

Spam protection

Where do spammers get databases with email addresses from?

Spammers (those who produce mass mailings) collect email addresses from all available sources. These can be forums, guest books, chats, social networks and other sites where postal addresses can be publicly available.
Hackers manage to get to some databases of addresses stored on sites.
E-mails are collected in most cases by a programmatic method (with the help of search bots - harvesters) and this does not require much effort (only time is not very much). Moreover, similar spam email databases you can not collect, but buy them from those who specialize in this matter ().
Billions of postal addresses are registered all over the world, and therefore you can simply try to generate emails with the help of special programs using appropriate dictionaries. Most likely, many of them will actually exist. Read about how spammers check the validity of addresses below.
There are special worms (viruses) that can send themselves to the database of addresses found on the victim's computer. The database collected in this way will already be cleared of non-working mailboxes.

How do spammers clean databases from inactive email addresses?

Those who collect the base of addresses, in fact, do not care who owns this or that address - they send letters to everyone in a row, because there will still be someone who will react to them (as they say, they are beating the squares).

But nevertheless, in order to optimize costs and increase the return on mailings, it is profitable for them to clear the databases of non-existent addresses. How do they do it? Let's see.

The simplest thing is to place a picture in the letter (it may not even be visible - one pixel in size), which will be downloaded from the site belonging to the spammer when the user opens the letter. If the picture is loaded, it means that the letter has been opened and the Email is valid.
Many e-mail clients (programs for working with e-mail) automatically send a message about reading the letter, which again plays into the hands of spammers.
The letter may contain a link calling to go somewhere, promising mountains of gold. Have passed - consider that your email will now be marked as good. The most unpleasant thing is that such a link can be disguised as an unsubscribe button from the mailing list, which in fact will lead to the opposite effect.

How to reduce the likelihood of your email getting into the spam database?

In general, as soon as your mailbox is "confirmed", spammers won't just get off of you. It is important to understand that no one is immune from spam. But you can significantly reduce the likelihood of getting into such a spam database if you take specific precautions:

You can, of course, not publish your Email anywhere and not tell anyone about it. But in most cases this is difficult to do, so I advise except for the main box have one or two secondary ones that you will use to register on forums, etc. Often they can be useful and that can be obtained without registration at all.
Do not follow the links in spam emails (even if there is an "Unsubscribe" button there - this is a trap) and, if possible, disable automatic image download in your email client program. There is a chance that your email will be counted as inactive and bulk mailings will not be received by a hundred per day.
If you have not registered a mailbox yet or are planning to start a new one (for example, due to the complete clogging of the old with spam), then proceed not from the convenience and ease of memorization, but, on the contrary, make it more authentic and more complicated. You will still send it to your friends electronically, but spam brute-force programs are unlikely to guess it.

What to do if spam doesn't let you breathe anymore?

These were all preventive measures to combat spam (or rather, to reduce its amount). But there is a possibility of effective struggle even in an already completely neglected situation. In this case, it becomes extremely important,.

The fact is that in such large services as or, there are powerful anti-spam filters.

They put all suspicious emails in a separate Spam folder, thereby freeing your Inbox from junk. Yes, there is no perfect spam cutter, and as far as possible, the contents of the "Spam" folder before cleaning it would be better to look diagonally for legitimate correspondence. But it's still much easier than digging through all this garbage all the time.

If you have a mailbox on another service, where the spam cutter is useless (for example, as in), then you should not despair. Get yourself an Email on Gmail or Yandex, and then set up the forwarding of mail to it from your old mailbox. Moreover, these settings can be made as in the old box (i.e., set up forwarding - shown in the screenshot):

In the same way, in the new mailbox, you can configure the collection of mail from your existing Emails (the screenshot shows the settings for collecting mail in Gmail):

The same can be said for mail client programs... Most of them also have a built-in spam cutter.

But in this case, do not forget that the mail service will have its own spam folder, which will also need to periodically look through (or look for messages that should come, but did not reach - for example, confirmation of registration anywhere is often cut by a spam filter), because mail from it will not be sent by default to your computer (although this can be corrected in the settings of the service or client program).

Good luck to you! See you soon on the pages of the blog site