Hello, dear readers of the blog site. Surely you have already encountered spam, even if you did not know exactly what this disgrace is called. And this is not even necessary. Spam pours with enviable regularity into a regular mailbox - these are the same countless advertising booklets that are slipped to you in the hope that you will order or buy something.

So here it is spam is just there is an annoying, climbing from all sides advertising(but not only advertising - it can be worse). You didn’t order it, you don’t need it, but it rushes and rushes from all cracks. In this heap, the information you need can easily be lost and you have to spend a lot of time sifting it out.

In our computer age, the main source of spam is . Besides advertising spam mailings can also be dangerous both for your wallet (phishing, social engineering, wires) and for your computer (viruses, worms, trojans).

What does the word Spam mean, what it is and how you can fight it, you will learn by looking at this short note. I hope it will be interesting (well, certainly useful - I promise you).

Spam - what is it

What does SPAM mean?

Samo spam word comes from the name of canned meat, which was vehemently advertised after the end of the Second World War (obviously, it was necessary to urgently sell the soldiers' meat rations).

Advertising was so aggressive, comprehensive and ubiquitous that this word (and the “sediment” associated with it) was remembered, but already about the intrusive advertising that appeared along with it in conferences (then still in the fidonet, if anyone remembers).

The word has taken root, especially since intrusive advertising has not become less, but rather the opposite. When e-mail gained popularity, then unauthorized mass advertising and malicious mailings of letters became commonplace. Such mailings were beneficial for spammers, because the necessary information was brought to a large number of people without any special costs.

But Email is not limited. They spam in the personal of social networks, on forums, in instant messengers, on bulletin boards, in comments on blogs, it is open for editing and adding text by everyone. They also spam on your phone, for example, by advertising calls or by sending promotional SMS messages.

Where can you find it on the internet?

1. Email- it's just a Klondike for spammers. With the help of mass mailings, you can sell anything, you can cheat and rob, you can infect computers and send worms. Bases for mass mailings are collected independently (with the help of programs), or bought from those who do this professionally.
2. Forums, comments blogs, wikis, and message boards - basically everyone is allowed to post here, and it's hard for spammers to stop spamming. This is not always advertising - often in this way webmasters try to get a free link to their site so that it ranks higher in Yandex or Google search results for various queries. This brings them traffic and money.
3. Social networks and dating sites - spam is very common among incoming personal messages. It is also available in the comments to the posts.
4. Messengers (like ) are also subject to this scourge.
5. SMS- messages from unknown people of an advertising nature. Probably everyone is familiar.
6. Search spam- a rather specific thing, but familiar to everyone. Did you have such that you enter a query into Yandex (Google), and in the answers there are entirely sites with some kind of gibberish that have nothing to do with the case. These are the so-called doorways (sites with automatically generated useless texts). They spam the results of search engines, but they earn on visits to them by visitors (in various ways).

Those. all letters in your mailbox (or messages on the forum, comments on the blog, messages in PM) of an advertising or other nature from an unknown sender - this is spam. True, the mailing lists to which you subscribed can also be annoying, but you can still unsubscribe from them (refuse to receive them further).

By itself, spam is annoying and disturbing, because it litters where it appears. Most often this is your mailbox, and it can be quite difficult to separate the wheat from the chaff when there are too many unwanted messages. But spam can also be a real danger to both you and your computer. Let's deal with all this.

We understand the varieties of spam (harmless and dangerous)

From now on, I will mainly talk about spam email messages, because it is this channel that is considered especially susceptible to this phenomenon. Everything else is not so neglected and not so actively used by spammers, and this does not apply to all network users. And here is the problem of their protection from unwanted messages is acute.

Varieties of harmless spam

1. Advertising of legal goods and services- in this case, the business owner simply chose spam mailings as one of the channels to attract customers and buyers, because it is not expensive, fast and brings results. Naturally, he does not really think about the moral (or rather immoral) side of this matter.
2. Prohibited advertising- there are goods and services that are prohibited by law from advertising, and for them spam mailings can be the main channel for attracting customers. By the way, spammers also advertise their services with the help of mass mailings, because this activity is prohibited by law. This also includes advertising discrediting competitors, because this is also prohibited by law (praise yourself, but do not scold others).
3. Impact on public opinion- very often spam becomes a good choice for those who are trying to manipulate public opinion in the right direction. It can be politics, as well as. Letters in general can be sent ostensibly from some person in order to compromise him, change his opinion about him or use his authority for profit. But personally, this does not pose a danger to you.
4. Letters that ask to be forwarded to others- these can be varieties of so-called "letters of happiness" (send it to 10 friends and you will be happy), or asking to send information to friends for some other reason. Often such emails are used by spammers to collect or replenish an existing email database for subsequent mass mailings.

High-risk spam - what it can be

If normal (harmless) spam can only slightly affect your peace of mind and mental health, then the examples below can be quite dangerous for your wallet or computer. And this is not a joke.

I myself (the one who teaches and teaches everyone) a couple of years ago "bought" a letter of this kind (phishing) and parted with several thousand rubles (read ""). I just wrapped myself up and did everything “on the machine”, just to fall behind, well, and when I realized it, it was too late.

1. - a very effective method of deceiving not only gullible, but also just busy or inattentive people (there is a hook for anyone). They send you a letter supposedly from your bank, electronic money service or from somewhere else. In this letter, you are sure to be taken aback by something (unsettled) and asked to log in to the site to solve the problem. You log in, but the site will be fake (although it looks like a real one as two drops) and the data you provided is immediately used to steal all your money.
2. Nigerian letters- you will find out that you can get a large amount of money (the pretexts are different - from an unexpected inheritance to helping a prince in exile). You do not believe for a long time, but you are convinced. When you believe it, you will be asked to transfer some money for "related expenses." You transfer and no one will bother you anymore.
3. Viruses, worms, trojans- the letter may contain an attached file with a malicious program (or a link leading to a site with a virus). She can immediately cause problems with the computer or sit quietly in a corner and carefully record all the passwords you enter, logins and other things useful in the household. Worms, among other things, also know how to send themselves to your friends at the addresses found in the Email contacts (as they will then be ...).

Spam Protection

Where do spammers get email databases from?

1. Spammers (those who produce mass mailings) collect email addresses from all available sources. These can be forums, guest books, chat rooms, social networks and other sites where email addresses can be publicly available.
2. Hackers manage to get to some databases of addresses stored on websites.
3. The collection of emails is carried out in most cases by a program method (with the help of search bots - harvesters) and this does not require much effort (only time is not very much). Moreover, such email spam databases you don’t have to collect, but buy them from those who specialize in this business ().
4. Billions of postal addresses are registered all over the world, and therefore you can simply try to generate emails using special programs using the appropriate dictionaries. With a high degree of probability, many of them will actually exist. Read below to learn how spammers check the reality of addresses.
5. There are special worms (viruses) that can send themselves to the database of addresses found on the victim's computer. The database collected in this way will already be cleared of non-working mailboxes.

How do spammers clean the databases from inactive email addresses?

Those who collect the database of addresses, in fact, do not care who owns this or that address - they send letters to everyone in a row, because there will still be someone who will respond to them (as they say, they hit the squares).

But still, in order to optimize costs and increase the return on mailings, it is beneficial for them to clear the databases of non-existent addresses. How do they do it? Let's get a look.

1. The simplest thing is to place a picture in the letter (maybe not even visible - one pixel in size), which will be loaded from the site owned by the spammer when the user opens the letter. If the picture is loaded, then the letter was opened and the email is valid.
2. Many mail clients (programs for working with e-mail) automatically send a message about reading the letter, which again plays into the hands of spammers.
3. The letter may contain a link calling to go somewhere, promising mountains of gold. Moved - consider that your Email will now be marked as valid. The most annoying thing is that such a link can be disguised as an unsubscribe button, which in fact will lead to the opposite effect.

How to reduce the likelihood of your email getting into the spam database?

In general, as soon as your mailbox is "confirmed", spammers will not just get off of you. It is important to understand that no one is immune from spam. But it is possible to significantly reduce the likelihood of getting into such a spam database if you take specific precautions:

1. You can, of course, not publish your Email anywhere at all and not tell it to anyone. But in most cases it is difficult to do, so I advise except for the main box have one or two secondary ones that you will use to register on forums and the like. Often they can come in handy and, which can be obtained without registration at all.
2. Don't click on links in spam emails (even if there's an "unsubscribe" button, it's a trap) and, if possible, turn off automatic image downloads in your email client program. There is a chance that your Email will be counted as inactive and mass mailings will not come to it at a hundred per day.
3. If you have not yet registered a mailbox or are planning to start a new one (for example, due to the complete clogging of the old spam), then proceed not from the convenience and ease of remembering it, but on the contrary, make it more authentic and more complicated. You will still send it to your friends electronically, but spammer programs are unlikely to guess it.

What to do if spam no longer lets you breathe?

These were all preventive measures to combat spam (or rather reduce its amount). But there is the possibility of an effective struggle even in an already completely neglected situation. In this case, it becomes extremely important, .

The fact is that in such large services as or, there are powerful anti-spam filters.

They put all suspicious emails in a separate Spam folder, thereby freeing the Inbox folder from garbage. Yes, there is no perfect spam cutter, and as far as possible, the contents of the "Spam" folder before cleaning it would be better to look diagonally for legitimate correspondence. But it's still much easier than digging through all this garbage all the time.

If you have a mailbox on another service, where the spam cutter is useless (for example, as in), then you should not despair. Get yourself an Email on Gmail or Yandex, and then set up mail forwarding to it from your old mailbox. Moreover, these settings can be made as in the old box (i.e., set up forwarding - shown in the screenshot):

Also in the new mailbox, you can configure the collection of mail from the Emails you already have (the screenshot shows the settings for collecting mail in Gmail):

The same can be said about mail client software. Most of them also have a built-in spam cutter.

But in this case, do not forget that the mail service will have its own spam folder, which will also need to be periodically viewed (or searched there for messages that should arrive, but did not reach - for example, confirmation of registration somewhere is often cut by a spam filter), because mail will not be sent from it to your computer by default (although this can be corrected in the settings of the service or client program).

Good luck to you! See you soon on the blog pages site

You may be interested

Account - what is it, how to create or delete it
How to create an email - what is it, how and where to register and which email to choose (mailbox)
What is Email (E-mail) and why is it called email
How to restore a page in VKontakte (in case of loss of access, deletion or blocking)
CAPTCHA (captcha) - what is it and what is it used for Moon Bitcoin (Litecoin, Dash, Dogecoin, Bitcoin Cash) - earn money on cryptocurrency faucets Should I buy image links?
Dog symbol - why is the @ dog icon so called, the history of the appearance of this sign in the email address and on the keyboard

The following technologies are used to protect mail servers:

There are two main methods of spam protection: protection against spam when mail is received by the server, and separation of spam from the rest of the mail after it is received.

Blacklists. Blacklists include IP addresses from which spam is sent.

Greylisting or greylisting. The principle of operation of gray lists is based on the tactics of sending spam. As a rule, spam is sent in a very short time in large quantities from any server. The job of the greylist is to deliberately delay the receipt of emails for some time. The address and forwarding time are entered into the greylist database. If the remote computer is a real mail server, then it must keep the letter in the queue and repeat the forwarding within five days. Spam bots, as a rule, do not keep messages in the queue, so after a short time they stop trying to forward the letter. When resending a letter from the same address, if the required amount of time has passed since the first attempt, the letter is accepted and the address is added to the local whitelist for a sufficiently long period.

DNSBL (DNS blacklist)– lists of hosts stored using the DNS system. The mail server accesses the DNSBL and checks it for the presence of the IP address from which it receives the message. If the address is in this list, then it is not accepted by the server, and the corresponding message is sent to the sender

Message limit. Set a limit on the number of messages.

Program Spamassassin(SA) allows you to analyze the content of an already delivered message. SpamAssassin comes with a large set of rules that determine which emails are spam and which are not. Most of the rules are based on regular expressions that match the message body or header, but SpamAssassin uses other techniques as well. The SpamAssassin documentation refers to these rules as "tests".

Each test has some "cost". If the message passes the test, this "cost" is added to the total score. The cost can be positive or negative, positive values ​​are called spam, negative values ​​are called ham. The message goes through all the tests, the total score is calculated. The higher the score, the more likely the message is spam.

SpamAssassin has a configurable threshold above which an email will be classified as spam. Usually the threshold is such that the email must meet several criteria; just one test failing is not enough to exceed the threshold.

The following technologies are used to protect websites from spam:

1. Image captcha. Those. the user is shown arbitrary text that the user must enter in order to perform any action.

2. Text captchas– the subscriber must enter the answer to the proposed question to confirm his actions.

3. Interactive captcha- a little common, but very useful form of protection. For example, to confirm actions, the user will be asked to solve an easy jigsaw puzzle - for example, to assemble a picture from three or four parts.

- €55-250 million annually. 60% world mail traffic.
50-75% from all Russian mail traffic. Modern anti-spam tools filter 85-98% of spam. The volume of the world market of sales of anti-spam filters and services in 2004 amounted to approximately $500 million (according to IDC).
Most antivirus vendors have included antispam components in their products. During the year, anti-spam companies made several purchases of anti-spam software companies (in particular, the purchase of BrightMail by Symantec for $340 million). In Russia, anti-spam filters have been installed by most of the holders of public mail services and the majority of providers, which made it possible to alleviate the problem of spam for their customers. The undoubted leader in Russia in terms of sales and the number of protected mailboxes is Spamtest technology.
1. PREVENTION The #1 anti-spam tool is to protect your address. Spammers will not know your address - there will be no spam. Light up your address on the network, you will have to throw it away and start a new one, it will only be a matter of time. And, as a result, inform all your friends and partners again of the new address, while losing a number of contacts is possible. To keep this from happening Enter two email addresses. One address for long-term contacts (do not shine it on the network).
Another address for making contacts, using the network (chat rooms, bulletin boards, etc.).
Then there should be no spam on the first address, because it is not known on the network.
When spam goes to the second address, just throw it away and start a new one.
2. NAME CHOICE People tend to get the most concise address. Let's say sergey@mail.ru is cool and what a pity that all the simple addresses are already taken. Be sure to [email protected] spam is pouring in non-stop. It's cool to have a concise site name, but the email address will still have to be reported to everyone personally, whether it be from numbers or an original, not a hackneyed word. By the way, for this purpose, the leading mail gmail.com registers names no shorter than 6 characters. All short names have long been included in the spam lists.
3. HTML SPECIAL CHARACTERS The simplest and most commonly used method of protecting against spiders is to encode an email address using HTML special characters. Instead of a dog @ . But today this method is hopelessly outdated.
Robots can easily find such addresses.
4. JAVASCRIPT On the anti-spam code generator page, you can generate your own script. Since these address hiding scripts are handicraft, they are very diverse and there are no programs that would be able to extract email from JavaScript. Today it is the most reliable address protection on the network.
5. ANTISPAMMERS But, what if you are exposed, or you are so famous that it is impossible for you not to be exposed, then you cannot do without an anti-spammer. There are many anti-spammer programs that you can download online.
What I do not recommend.
I came to the conclusion that all these anti-spams are small and weak, and a person cannot pull sensible anti-spam, only a reputable company, say, such as Gmail.com, can do this. Their spam remains on the server, you can always go in and correct it. So my strong advice: get yourself a mail on Google.
I have not seen the best spam filter, all spam remains on the server, which, if desired, can always be viewed and corrected. Anti-spammers do not completely solve the problem, but make life easier in the problem.
6. POCKET PC & WAP Spam has reached this level, but today there are quite reliable means of protection. Therefore, the development of this issue is not relevant.

Introduction to the problem

We all know what spam is because we have either experienced it or read about it. We all know how spammers collect email addresses. It is also no secret that spam cannot be completely defeated. The problem is how to protect users who leave their contact details on your site as much as possible with minimal effort.

Previously tested methods of protection

The biggest threat to mailboxes comes from programs that download websites and take email addresses from the text of pages. They download either only your site, or roam like search engines throughout the network. If your site is small, the protection of this text autocorrect is enough:

]+href=)([""]?)mailto:(+)()@".
"()(+.(2,4))2([ >])~i", "1" mailto: [email protected]"
onMouseover="this.href="mai" + "lto:3" + "4" + "%40" + "5" + "6";"7", $text); ?>

Unfortunately, it won't work if you have a large site. For example, spectator.ru, whose author was one of the first to use this method. If I was a spammer, I would go into my personal settings, tick off "do not show ears", 1000 reviews per page, and catch cookies with Proxomitron. Then, with a rocking chair or a php script, I would download pages with comments (substituting cookies with settings) and catch the addresses using a regular expression. I would get a small base for advertising mailings.

There were a couple more protection methods in which the mailto: link was automatically replaced with some other one, but the effect remained the same - when you clicked on it, the system client would create a letter to the desired address. Both of them did not withstand criticism.

Meet: hedgehogs

Obviously, it is difficult to come up with another way of protection other than the already tried and tested - providing a form on the site to send a message. Let's design it. The advantages of the method are obvious: no one will be able to get addresses for their spam database from your site. Sending messages by hiding your address, as spammers do, will not work - the web server will fix its IP address. The lists of public anonymous proxy servers are updated regularly and it is easy to block access from them.

Submitter

Let's start with it, because this is the most difficult part.

When installing a form submitter on a site, it is important to protect it from hooligan attacks, which can be no easier than spam. Therefore, we will have to make great efforts in this direction.

First, we will protect ourselves from stupid double clicks and sending many identical requests. The idea is this: the message will not be sent if the user has not previously opened the page with the form, and by opening the page with the form, you can send the message only once. This can be done using PHP's built-in sessions. When opening a page with a form, we will start a session in which we will save a variable, say $flag. The session ID will be displayed as a hidden element at the very end of the form. The user enters a message and submits the form. Upon receiving the form, the script starts the session and checks for the presence and value of the $flag variable. If the variable does not exist, then this is a second click, the letter is not sent and an error message is displayed. If there is a variable, and the form data suits us (the required fields are filled), the script sends an email and deletes the session.

Secondly, we will protect ourselves from smart hooligans by recording message logs. If the user submits a correctly completed form, the script will look in the logs and check what is there. Yes, it should be banned.

* send messages to the same address more often than a certain period
* send the same text to different addresses
* and simply use the form sender too often - say, no more than 10 messages per day per user

We print the session ID at the very end of the form so that the hacker needs to download the entire form and parse it, which is more complicated than just sending HTTP requests. Naturally, the form sender will issue messages about errors in writing a message, a requirement to provide a return address, and so on.

The resulting form submitter code was too large to include in the text. It has been archived on the website. It seems that the script works and sends messages.

Replacing addresses in text

Now the form submitter is ready, and we need to replace all emails with links to it. Of course, you shouldn't do it manually. For myself, I wrote a script that automatically replaces addresses with links to the form sender.

... Cons: more time for placing links (compensated by the catalog of links), the user, hovering over the link, does not see what address he will get to. (Dmitry Smirnov, "Ideal author's project, hypertextuality")

All the disadvantages mentioned are easily eliminated if you use a code similar to the one that I will now describe and show.

There is nothing complicated here, if these are links, then "more time for arrangement" is not required. On my site, I use an engine script that is called by all pages, so it’s not a problem to add code to it or call it from it to replace addresses. Postal addresses are both written and written directly in the text of the pages, but before being displayed to the user, they are replaced with the desired text. Compiling a database of links or email addresses is not a problem.

So what does the address substitute do. It searches for "mailto:" in the link text, selects addresses from them, sends a query to the database to count (count(*)) how many addresses from those on the page are in a special table. If there are new addresses on the page, then their number will be greater than the result of the query. In this case, a query is made in which the address values ​​are selected, and those already existing in the table are excluded from the list. The rest of the list is sent to the table with an INSERT query.

As far as ID addresses go, I think it's better to use something that the site visitor couldn't pick up. Can you imagine, the link /email.php?id=10 leads to the form submitter? What a temptation to substitute 11, 12, etc. there. and try to send them all a message. Therefore, I decided to use the md5 hash from addresses as identifiers. Hardly anyone will undertake to pick up a hash. In the case of a directory of links, you can get by with the ID, but then you have to select all the values ​​from the database, and replacing addresses with their hashes is much easier.

Executing a command like

]+href=)". "([""]?)mailto:( [email protected]+". ".(2,4))2(.*?>)~ie", ""12"/email.php?email=". urlencode(md5("3")). ""4"" , $text); ?>

...which replaces addresses with their hashes. I did not dare to replace the rest of the addresses in the text with links, but made a simple replacement with addresses like vasya_at_pupkin_dot_ru. The autosubstitute code is also in the archive.

Outcome

Hiding email addresses from visitors is pretty easy. The autocorrect mechanism does not require any additional effort, and you can write pages on the site as if nothing had happened. Difficulties arise when protecting the form submitter from web bullies. This protection requires a lot of effort and complex code, so for the time being I did not use the written code on the site. You can download an archive with an address substitute and a form sender, but I ask you very much: do not put it on your site in the form in which you downloaded it, I myself do not know how reliable it works.

A modern spam mailing list is distributed in hundreds of thousands of copies in just a few tens of minutes. Most often, spam goes through malware-infected user computers - zombie networks. What can be done to counter this pressure? The modern IT security industry offers many solutions, and there are various technologies in the arsenal of anti-spammers. However, none of the existing technologies is a magical "silver bullet" against spam. There is simply no universal solution. Most modern products use several technologies, otherwise the effectiveness of the product will not be high.

The most known and widespread technologies are listed below.

Blacklists

They are DNSBL (DNS-based Blackhole Lists). This is one of the oldest anti-spam technologies. Block mail coming from the IP servers listed.

• Pros: Blacklist 100% cuts off mail from a suspicious source.
• Minuses: They give a high level of false positives, so use with caution.

Mass control (DCC, Razor, Pyzor)

The technology involves the detection of mass messages in the mail flow, which are absolutely identical or differ slightly. To build a workable "mass" analyzer, huge mail flows are required, so this technology is offered by large manufacturers with significant volumes of mail that they can analyze.

• Pros: If the technology worked, then it was guaranteed to determine the mass mailing.
• Minuses: Firstly, a “large” mailing may not be spam, but quite legitimate mail (for example, Ozon.ru, Subscribe.ru send out thousands of almost identical messages, but this is not spam). Secondly, spammers are able to “break through” such protection with the help of intelligent technologies. They use software that generates different content - text, graphics, etc. - in every spam email. As a result, mass control does not work.

Checking internet message headers

Spammers write special programs to generate spam messages and distribute them instantly. At the same time, they make errors in the design of headers, as a result, spam does not always comply with the requirements of the RFC mail standard, which describes the format of headers. These errors can be used to detect a spam message.

• Pros: The process of spam recognition and filtering is transparent, regulated by standards and quite reliable.
• Minuses: Spammers are learning fast, and there are fewer and fewer errors in spam headers. Using only this technology will allow you to delay no more than a third of all spam.

Content filtering

Also one of the old, proven technologies. The spam message is checked for the presence of spam-specific words, text fragments, pictures, and other typical spam features. Content filtering began with the analysis of the message subject and those parts of it that contained text (plain text, HTML), but now spam filters check all parts, including graphic attachments.

As a result of the analysis, a text signature can be built or the “spam weight” of the message can be calculated.

• Pros: Flexibility, the ability to quickly "fine" settings. Systems based on this technology easily adapt to new types of spam and rarely make mistakes in distinguishing between spam and normal mail.
• Minuses: Updates are usually required. The filter is configured by specially trained people, sometimes by entire anti-spam laboratories. Such support is expensive, which affects the cost of the spam filter. Spammers invent special tricks to circumvent this technology: they introduce random "noise" into spam, which makes it difficult to find spam characteristics of the message and evaluate them. For example, they use non-alphabetic characters in words (for example, the word viagra may look like this when using this technique: vi_a_gra or [email protected]@) generate variable colored backgrounds in images, and so on.

Content filtering: bayes

Statistical Bayesian algorithms are also designed for content analysis. Bayesian filters do not need constant tuning. All they need is pre-training. After that, the filter adjusts to the topics of letters that are typical for this particular user. Thus, if a user works in the education system and conducts trainings, then personally messages on this topic will not be recognized as spam. For those who do not need offers to attend the training, the statistical filter will classify such messages as spam.

• Pros: Individual setting.
• Minuses: Works best on individual mail flow. Setting up Bayes on a corporate server with heterogeneous mail is a difficult and thankless task. The main thing is that the end result will be much worse than for individual boxes. If the user is lazy and does not train the filter, then the technology will not be effective. Spammers specifically work to bypass Bayesian filters, and they succeed.

Greylisting

Temporary refusal to receive a message. The failure comes with an error code that all mail systems understand. After some time, they resend the message. And programs that send spam, in this case, do not resend the letter.

• Pros: Yes, this is also a solution.
• Minuses: Delay in mail delivery. For many users, this solution is unacceptable.