Hello, dear readers of the blog site. Today I want to look at status codes and HTTP headers, which are included as components in the server response and provide valuable information about the operation of the site. Well, let’s look at what tools allow you to check them.

This material will be a logical continuation of the previous article, where I presented general information about, which serve as nothing more nor less than a “vehicle” for transmitting hypertext (), which is precisely the content of any page of a web resource.

If, when making requests to the server, each page of your site responds with the correct code, this will be a great contribution to its successful promotion. Conversely, code that does not correspond to the state of the web page can greatly ruin the life of the webmaster and initiate a decline in positions. Therefore, I advise you not to neglect this aspect and pay due attention to it, at least in general terms by reading this article.

Server response and its components that can affect SEO

In an article explaining the essence of data transfer via HTTP protocol(HTTPS), the link to which is given at the beginning of the publication, I wrote about how communication occurs in principle, which is based on the scheme "client request - server response".

Let me briefly remind you how this is done. The browser, after the user enters the URL of the page in the address bar, contacts the nearest DNS server, where lists of all domains () are stored, as well as their corresponding IP addresses (every device on the Internet has, including servers where sites “live” ).

Having received the required IP, the browser sends to the corresponding IP server GET request to get the content you need. The server software processes the request and sends a response that includes the content of the web page in the form of HTML code, which is then modified by the web browser to display the content of the page in a human-readable form.

But, as they say, not a single browser... In a similar way, any client program that is equipped with the necessary functionality for this can “conduct a dialogue” with the server, including robots search engines . The principles of the mechanism of such interaction for various applications are absolutely identical, the only difference is in the details.

One of the nuances is that the main task web browser is to display the content of the page required by the user. For search bots, the function of displaying content on the monitor screen is not relevant at all. They use the information always contained in the server response for their own selfish purposes, namely, as an additional factor used when evaluating a resource page.

To practically check the server’s response to a request from a Yandex search engine robot, you can use a special tool where you enter the URL of the page being examined, and also select the desired bot from the drop-down list (in addition to the main one, there are robots for mirrors, images, video searches, and others) :

Below I will tell you in more detail what useful things can be gleaned from this data. After all, if we understand this, we can find out which path to take in terms of SEO optimization site pages. Well, let's pay attention to others online services, through which you can check the server response code and view the contents of HTTP headers.

HTTP status codes - 200, 301, 302, 403, 404, 500 and others

The status code that comes in the server response determines the status of the site's web page for which the client application sends a request to the server. For example, HTTP 200 OK means that all the contents of the page have been transferred and will be available for viewing.

For successful promotion the main thing is that in each specific case the status code is correct and corresponds to the current state of affairs. For example, if the address has been changed on a permanent basis for one reason or another, then the server response should indicate the presence in relation to the page being examined (in the screenshot below, the URL of the page to which the redirection was made is indicated as the “Location” value):

A practical example is constant redirection, which creates duplicate pages with the same content, which without appropriate measures to eliminate them can lead to crash. Before continuing our reasoning, let’s see what codes generally exist, which are divided into five groups:

1. 1XX— informational, in which the server reports on the process of processing the request.

2. 2XX— HTTP codes informing about successfully transferred data. I have already mentioned 200 OK, the rest are its derivatives.

3. 3XX- forwarding various types from one URL to another. For example, if 301 means that the page address has been changed permanently, then 302 indicates a temporary redirect. Unlike a permanent 302 redirect, it is not a signal to search engines to transfer the weight of the page from the old address, so in practice it is used only in exceptional situations when it is the most optimal solution.

4. 4XX— HTTP error codes in the request from the client. For example, the well-known status code 404 means that there is no document at that address on the host.

5. 5XX— an error on the server, as a result of which the page cannot be provided.

More detailed list status codes provided in the server's HTTP response can be obtained by visiting the corresponding Wikipedia page.

The importance of the correct status of web resource pages is very difficult to overestimate. Therefore, from time to time try to check the server response codes for the pages of your site, this can protect you from many troubles.

There have been cases, for example, when the server responds with HTTP code 404 instead of the expected 200, because in reality web pages are accessible and open perfectly. If such a situation, God forbid, arises when the server responds to a request from the same Yandex robot, then it is likely that these pages will drop out of the index, which will be very disappointing.

But even if such force majeure occurs, timely checking the status code will help to detect this problem in time and correct its consequences with minimal costs time and effort that you may need for other important things to optimize your site.

If you have a standard virtual hosting, then contacting your technical support often turns out to be the best solution. If your resource is located on a dedicated server, then you will most likely have to solve the problem yourself, but the main thing is that you not only know about its existence, but also “where it comes from.”

If you look at the screenshot above, where the server response is given, you will see that just below the line with the status code there is an explanation that includes information about the server response time, the site’s IP address, encoding and page size:

Particularly interesting server response time, which is integral part. This metric is one of the ranking factors, so we have a vested interest in how to reduce it.

What should the response time be? Google, for example, defines a maximum limit of 200 ms (milliseconds), but, of course, the lower the better. How to increase server response speed? First, try to carry out some activities on , it is quite possible that installing a caching plugin will solve the problem.

It is possible that the actions you take will help little, since a lot depends on the settings and capacities software the server itself. Then it makes sense to contact the hosting server administrator. If you do not receive a clear answer, and the server response time greatly exceeds the limit indicated above, you should think about at least changing the provider.

HTTP headers and their meaning

In this light, we will consider examples of answers to search engine robot queries, since they interest us in the first place. For clarity, first I present a screenshot with HTTP headers corresponding to the page URL with a status of 200 OK:

Server— name and version of the web server. IN in this example this is nginx, which, due to its low resource consumption and configuration flexibility, solves the problem of optimizing the operation of the main Apache server and is used in conjunction with it.

Date— date and time of return of the contents of the requested page.

Content-Length— volume of transmitted content in bytes ().

Connection- connection. The keep-alive parameter means that after the document is issued, the connection to the server is not broken and additional requests can be sent.

Vary- this header allows you to issue correct document if there are several versions of it. It is relevant, for example, when using page compression technology, when both compressed and uncompressed versions are stored in the cache. With an Accept-Encoding response, the cache will contain different versions of the requested page for different client applications (agents).

Cache-Control— caching management. In our example, this header reflects the type of cache in which the document is located (public) and the time during which it should be in the cache (max-age). The value public indicates that this operation applies to files stored in the public cache. The max-age parameter gives the time in seconds.

X-Hyper-Cache is a special header that many WordPress users will probably immediately identify with. Undoubtedly, it concerns the work, which I consider to be perhaps the best in its class. The value "hit - gzip" indicates that the cached page has gzip compression applied.

Content-Encoding— a method of encoding (in a general sense) the page content transmitted in the response. In our example, gzip compression was applied. This is a signal client program (User Agent) unpack the contents for its correct perception.

And now I will note the response headers, the contents of which webmasters should pay attention to Special attention, as it can have a major impact on promotion. Moreover, if you use site content management, with the help of which HTML pages are generated “on the fly,” then with a high degree of probability, if one web page has a problem, the others will also suffer.

Content-Type— content type, which in this example is HTML code in UTF-8 encoding. Incorrectly specifying the encoding can lead to difficulties in the perception of text by users and PS bots, and this can lead to the page not being included in the index.

After all, if your encoding is set incorrectly, then instead of adequate Russian text, the same users will see incomprehensible “crazy words” on the page, which will not increase the prestige of your website.

Last-Modified— date of the last modification of the web page. If the client (in our case, the Yandex robot) received this header from the server with the date the content was updated, then the next time it accesses the URL of the same page, it will send it to the server as part of the request If-Modified-Since.

The web server will allocate a period of time latest changes until the time specified in the If-Modified-Since header. If during this period the page has not been changed in any way, the server will send a response with an HTTP code 304 Not Modified, and in this case the contents of the page will not be sent. If editing took place, then the robot will receive the code 200 OK along with the modified content.

This mechanism, if configured correctly, allows you to provide constantly updated information. After all, the relevance of the data is important here, which is ensured by the correct implementation of time checking last update. After all, if configured incorrectly (if the date specified in Last-Modified does not change), the robot may simply receive the code 304 Not Modified (instead of 200 OK with new version document), although the content has been edited several times.

How can you check whether Last-Modified is working correctly for the server on which your site is located? Let's try to figure it out on a specific example.

On the same Yandex service, the link to which I already offered above, there is a special option that allows you to add an If-Modified-Since request and specify the date and time you need (in GMT format, that is, Greenwich, relative to the Moscow time zone this is - 3 hours) up to minutes, which will determine the time interval for checking for updates:

Take a look at the 10th screenshot up from here, which shows the result of the check against the URL of one of my blog pages (where all sections of the server response are marked). There, in part of the headers, a certain Last-Modified value is given, that is, the date of the last update. Now I include the If-Modified-Since indicator in the request and check the server response:

As you can see, a 304 Not Modified code was received without the content of the web page, which is absolutely true for this situation, since the content really has not been updated in this period. Next, for testing, I added a small piece of text in this article.

Then I again sent a request from the Yandex robot to the server, which, with the caching mechanism working correctly (after refreshing the page, is present in the cache latest version) should return a 200 OK response with new content, which is what happened:

For complete reassurance, you can also view the contents of the Content-Lenght header, which shows that the volume of content has increased slightly (18443 versus 18437 before editing). This is true, since I just added a bit of text. Likewise, you can check that the headers are configured correctly for your server.

Location is another title that I would like to note for completeness of information on this topic. It appears in the server response if the robot sends a request for a web page from which the permanent redirection was made(HTTP code 301):

The new address to which the redirect was placed will be present in the Location header. There is no page content in the response, which is quite logical, but the explanation that follows the 301 Moved Permanently response code indicates the size of the page to the URL of which the redirect is being made.

Checking the server response in online services

Further, for the sake of completeness, it would be useful to note online services, which allow you to check the server's HTTP response. On the Internet, I liked this one (Checkmy.ru), which has decent functionality. Let’s now check the server’s response on it, but to a Google robot request for a change:

After activating the process, just below you will receive a response with all the layouts:

The Checkmy service offers users not only the choice of the application (User Agent) from which the request will be sent, but also the use of the If-Modified-Since and Accept-Encoding headers, which were discussed above.

In addition, if a response contains a redirect code, the number of redirects will be indicated (ideally, it should be the only one). Several consecutive redirections already give reason to think twice, since this is not the best option to optimize the resource.

The site also has such a feature as a browser bookmark, which will provide a quick check of any web page you go to. To do this, simply scroll down the page to the desired location by clicking on the link “ Fast access» from the top menu. Then, using the left mouse button, grab the button "Checkmy", move it to your browser's bookmarks bar:

In conclusion, I would like to mention another service, with the help of which you can successfully carry out a mass check of the server response for 200 URLs at once, and there is the ability to download ZIP archive with URLs. And for dessert, a video about what the 404 Soft code is and why it is dangerous for webmasters:

Impossible without knowing the server's responses.

Example:

404 Not found

Further actions depend exactly on what response code the server or page gave. Due to the fact that the set of codes is standard for all sites/pages/servers, the actions when issuing a particular code will also be standard.

Today there are 5 main classes of response code:

1xx: Informational (Russian Informational) - the request was correctly received, but its processing was not completed.

2xx: Success (Russian: Successfully) - the request was correctly received and successfully processed.

3xx: Redirection (Russian: Redirection) - redirection codes to other pages.

4xx: Client Error (Russian: Client error) - error on the client side.

5xx: Server Error (Russian: Server error) - error on the server side.

Now let's look at some of the IANA status codes individually.

Server response 1XX

100 Continue Server Code

100 Continue reports that communication with the server has already been established, the server has accepted the correct request, and data is now being exchanged between the server and the client. This code is temporary, i.e. he is always followed by another. Code 100 is internal and is not an error code. Those. “The door is open, read what you need, when you finish, close it.” Code 100 may not be generated if the user has already received part of the data from the server.

101 Switching Protocols

This code is also not erroneous. Generated when switching from one protocol to another. For example, when requesting switching from old version HTTP to a newer one.

This is one of the simplest server codes. It means that a request was received from the user to switch the type of protocol used on the web server, and the server agreed to this.

102 Processing

In a sense, this is an analogue of code 100. It is generated when processing a request may take a long time. For these purposes, the wait timer is reset and waiting for further commands occurs as usual. It is also not an error code.

Server response 200 OK

It rightfully occupies the very first place in importance and popularity, because This is what the server gives if the user’s request is successfully and correctly processed.

Server response 301

It is also one of the common response codes. It reports that the requested page at a given address is no longer available, and then redirects to another address. A 301 redirect can be used, for example, when “moving” a site from the HTTP protocol to HTTPS (usually this is implemented through the .htaccess file, available on Apache servers).

Server response 302

This code indicates that the location of the requested page has been temporarily changed. Information about the new location of the requested document must also be provided. This code was originally used as the main redirection method.

Server response 404

That's it, the only people who didn't see the 404 server response error were those who weren't born yet and those who died before the creation of the Internet. This code indicates that the requested document is not available on the site for some reason. The server response error code 404 should only be returned if there has never been a document at the user-specified address. If a document was previously available at this address, and then it was removed from the site, then the server should return a code 410, not 404.

Fake 404 pages

Most webmasters do not pay any attention to 404 pages, however, this can seriously harm the ranking of the site. It's a paradox, but the page with the 404 File message Not Found does not always give a 404 code. Such pages are usually called “Soft 404”. The reasons for this are simple - for some reason the page returns a code other than 404 and 410 - for example, 200. This is quite possible if the page has already been created, but there is no content on it yet.

Server response 500

All 5xx series codes indicate that the server is unable to complete processing of the request. Along with the code, an explanatory hint (with a reason) should appear in English.

500 Internal Server Error

Code 500 is given in case of any internal server error, with the exception of other errors of class 5xx. Such an error can be given when the link is generated on the server immediately at the time of the request. The simplest example- internal search on the site: there is physically no document on the requested link.

Server response 502

Code 502 can be displayed in cases where the server plays the role of a gateway or proxy, but it was not possible to “find a common language” between it and the upstream server, i.e., in fact, this is simply a data exchange error.

Server response 550

If error 550 occurs, you need to check how correctly the MX records are written in order to eliminate these server response errors.

The output will be a table.

You need to make sure that it contains the necessary entries for your mail to work:

IMPORTANT! Mixing MX records is not allowed, i.e. the table in the output should only contain those MX records that are needed specifically for your mail. If necessary, you need to adjust the records by correcting errors and/or removing unnecessary items.

How to get server (page) response codes via Yandex

Step 1. Check the server response code to the site page that should be in the search.

We open any page of your website located in search results Yandex, then from address bar copy its URL.

Now we go to the Yandex service (http://webmaster.yandex.ru/server-response.xml), with which you can look at the site through the eyes of a robot and check the server response speed in the Yandex panel.

Simply paste the URL of the page we are interested in into the text field and click on the “Check” button. In this case, we received a 200 OK code, indicating that the page is working normally.

Step 2. Check the server’s response to a obviously non-existent page.

In the same service, enter domain_name/some_crocozyabr

In this case, we received a 301 Moved Permanently response. This indicates that the page address is incorrect and the page is being redirected to the correct address.

How else can I find out the server (site) response codes?

As an alternative, you can punch the response code using the http://mainspy.ru service. It works similarly to the Yandex service: insert the URL of interest and click “Check”. The response code in this case is in the very first line:

Bertal, unlike Mainspy, allows you to look at the page not only through the eyes of a Yandex bot, but also through the eyes of search robots Bing and Google, and as a bonus, it can emulate popular browsers. For convenience, let's look at the same pages through the eyes of GoogleBot. In this case, the response code is highlighted in green.

Bulk checking of server (site) responses online

Mass checking of response codes can be useful for finding broken sites on which links were purchased (through exchanges or directly - it doesn’t matter).

Dimax.biz - http://backlinks-checker.dimax.biz/tools/proverka_otveta_servera.php - this is one of the best checkers. The only negative is that in the free mode you can make no more than 2 requests of 50 links each. For more “serious” volumes you will have to use a paid PRO tariff. At the output we get a list sorted by response code. In this case, there is no need for sorting, because There are only 2 addresses in the list, and both give code 200.

Urlitor is another service for bulk verification of response codes. The good thing about the service is that the test results are tabulated to make it easier to understand. By the way, the links in the table are clickable.

How to check the speed (time) of the site server response?

It’s impossible to count how many such services have already been created. Let's look at some of them.

This is an English-language tool that analyzes speed in all parameters. With its help, you can find out the speed in seconds, how much the tested page weighs, and also get an assessment and recommendations for improving it. Advantage of this service is that everyone is analyzed separate element. This analysis allows you to find out what exactly is slowing down the loading of a particular page and/or the site as a whole.

Which Loads Faster

The main feature of this service is that it analyzes the loading time of two resources simultaneously. This allows you to find out which of the two resources is faster. The only negative is with different connections and in different browsers the result may vary.

Google PageSpeed ​​Insights

Google PageSpeed ​​Insights is also one of the most powerful tools to measure the speed of the mobile and desktop versions. The assessment is made on a 100-point scale. 85 points or more is a good indicator. Plus, as a bonus, it gives recommendations for improvement.

Long server response

A response that lasts more than half a second is usually called “long.” Therefore, when loading the site for a long time, you may see a message in the browser “the timeout for a response from the server has been exceeded.” There can be many reasons for a long response:

Complex logic for providing data

The server does not have time to process incoming requests in a timely manner due to their large number

The queries themselves (either complex, or unoptimized, or both)

Queries to a large number of external resources

Large number of executable files

The web server itself takes a long time to process the request.

The most painful areas of server performance:

Web server used (Apache, IIS).

A number of web servers can create delays even when serving static files, because... At the architectural level, they are not designed to process a large number of requests and because of this, there may be a message that the timeout for a response from the server has been exceeded. Therefore, for the normal operation of the web server, it makes sense to use nginx (and in conjunction with Apache, php-fpm, as well as other application servers for processing server-side calculations).

Using OpCache.

Reduce server response time by caching executable code (site scripts) - it allows you to use a ready-made result instead of translating PHP instructions into binary code each time. But this caching has nothing in common with caching the results of executing PHP scripts.

Database queries.

The second step to server performance is setting up tables (indexes) in the database and structuring them to facilitate query processing. This also includes recalculating intermediate results and caching the most frequently used results into separate tables. This will reduce the consumption of server resources several times and help reduce server response time.

Complex data processing logic.

The third step is to simplify the server logic. Essentially, it's just eliminating unnecessary operations and profiling the execution time of server-side scripts.

Access to third party services.

Requests to third-party services written in the code of server scripts are a “common story” that can bring many surprises, since the performance of the services from which data is requested is almost never checked by anyone. But the response time third party service directly affects server response time. Therefore, it is best to use only internal sources in server queries, which can be monitored for performance quality at any time, or request data from the client in a deferred mode.

Why web server response speed affects promotion.

Firstly, because loading speed is one of the ranking factors (although not decisive). Google openly states that less than 1% of websites rank for page speed. BUT…

Secondly, if the page takes too long to load, the user will simply close it. This user behavior is usually called “refusal”. By the way, “refuses” have a direct impact on positions in search results. The higher the download speed, the lower the failure rate and, as a result, the higher the positions.

Timed out waiting for a response from the server.

First, it is important to understand the cause of the failure. Those. the user enters the address, and the browser at this moment sends a group of requests, and also starts a countdown stopwatch for each of them. If after a specified time the browser does not receive a response to its request, then the user will see such an unpleasant picture.

There can be several main reasons for the failure:

• It is impossible to connect to the site due to unstable operation of its servers;
• Broken browser settings or clutter;

Problems with the Internet connection on the user's side;

The resource is blocked.

What to do to solve?

If the failure is single, reload the page using the Ctrl+F5 combination. You may need to reload the page several times. If it doesn’t help, check your Internet connection.

Network Settings.

1. Some sites are sometimes capricious. For dynamic IP, the solution will be simple - reboot the router by turning off the power.

2. A slow connection sometimes causes the ERR_CONNECTION_TIMED_OUT error. Internet speed can be checked using Yandex Internetometer. If the speed is too low, you should contact your Internet provider.

3. You need to check “Network Properties” for the presence of extraneous DNS addresses. If there are such addresses, delete them (after rewriting them somewhere, just in case) and check the system for viruses using anti-virus software installed on the PC - NOD32, Kaspersky, AdwCleaner, MalwareBytes, Dr.Web, etc. It is best to use Live downloaders for these purposes.

4. Check the settings of the router itself. The MTU parameter is most often lost. It is impossible to give universal recommendations for setting up a router, because... this directly depends on both the router model and the Internet provider. Typically MTU values ​​are 1500, 1460, 1476.

What should the server response time be?

And straight away the specific numbers:

The highest conversion rates are for pages that load completely in 1.8 and 2.7 seconds for desktop and mobile versions respectively

The lowest bounce rate is for pages that load completely in 1 and 0.7 seconds for desktop and mobile versions, respectively.

These figures are taken from a study by Akamai Technologies.

So, you have checked the site for loading speed. But how to react to the results?

<1 секунды - идеал

1-2 seconds - almost ideal

3-5 seconds - tolerable, but it makes sense to finish it

5-10 seconds - bad, need to finish it urgently

≥10 seconds - very bad, you need to finish it EMERGENCY

However, we must not forget one ultra-important rule - the download speed should be higher than that of competitors. Research from The New York Times has shown that a difference of 0.25 seconds can be enough to make visitors prefer a faster site. And before you can blink an eye (in the most literal sense), the user will leave you for a competitor.

Reducing the server response

Graphics optimization.

We said earlier that some checkers also provide optimization recommendations. Among them you can find the addresses of pictures that can be optimized by reducing them.

Use browser cache.

The browser will download the images to its cache. Consequently, re-downloading images from the server will no longer be required, which will save a lot of time on downloading.

Enable compression.

Relevant if gzip is used. As a result, the volume of data is reduced by 4, or even 5 times. The smaller the volume of transmitted data, the less time it takes to transfer it.

Reduce server response time.

Using the Pingdom service, you can calculate how long it takes the server to send a response code. The ideal time is no more than 0.2 seconds.

These instructions will help significantly speed up the site. However, there is a risk of harming functionality or appearance. Therefore, before each action, it is imperative to make backups of the source files. It also doesn’t hurt to consult with technical specialists.

We've released a new book, Social Media Content Marketing: How to Get Inside Your Followers' Heads and Make Them Fall in Love with Your Brand.

Response code 200 is one of the types of HTTP codes that informs the user that the request was successfully processed. Based on the status, the server can provide the body and header of the message.

More videos on our channel - learn internet marketing with SEMANTICA

Let's give an example. You sent a parcel to another city. The post office gave you a tracking number. Using it, you can see what’s wrong with your shipment—it left the sorting center in your city, or it arrived in another. Here it was handed to the addressee. Each time the system gives you a status in response to a request.

How it works

First, let's look at it. So, the user opens a browser and makes a request to an Internet resource. After this, the browser receives a response from the host, where a three-digit code is indicated. By a combination of numbers you can determine what situation is currently observed on the host.

HTTP is a special protocol for exchanging data between different (the user's browser and the web server where the site itself is located). That is, the browser sends a request to the server it is interested in, this could be an action or document, and then receives a response. If the response to the request is positive, the server response code 200 is displayed and the file download begins. If it is negative, that is, the requested page was not found or there are problems with the service, an error message appears.

What does code 200 mean for proper site indexing?

The 2xx server response category is the “Success” category. This category notifies users of a positive result. In particular, the “200 OK” code tells the user that his request was completed successfully. For example, the client requested certain data. A server response of 200 means that this data is displayed in the header or message.

Today, all search engines index resources and links that provide a response code of 200 to queries. The search engine understands this this way: the page really exists, which means it can be included in the index database. If you want a search engine to index a particular page, make sure that it produces a response code of 200.

It is important to check whether non-existent pages are returning code 200. This is possible even when you visually see “404 - page not found” on the screen. The cause of this problem may be incorrect configuration of the site. If you don’t want problems with promoting your resource, check all types of pages for the correct server response. This way you can identify pages that are only pretending to be necessary.

How to check response codes

To do this, you can use one of the many programs that are available on the Internet. Some perform mass checks for all pages of the site, while others require you to enter each URL. Choose a service based on your tasks.

In fact, there are a large number of server response codes, but the most common are the following:

• If at first the page responded to the request with code 200, was successfully indexed, but then it was deleted, when you go to it, code 404 (not found) will be displayed.
• If you use a temporary redirect (302), then both addresses will be included in the index.
• If a web page uses a permanent redirect, you will receive a response with code 301. And the search engine will only index the final address with the required code.

If you assign a 301 redirect to a page, it will later be removed from the index database, and its weight may be transferred to the page to which the redirect is directed. However, re-indexing is a lengthy process; in some cases, Yandex completes it within a year. Therefore, it is better to immediately edit the pages correctly and configure them to work correctly before indexing.

The 404 page is designed to inform the user that the url (page address) he specified does not exist.
Such incorrect URLs can also be called “broken links”.
Many sites make their 404 pages for the convenience of their users. These are often beautiful and interesting pages that make the user smile instead of being disappointed that the page address is incorrect.
There is an important technical component to creating a 404 page that will greatly affect your sites ranking in search engines if everything is not set up correctly.

If you are interested in creating a 404 page, then you need to consider three points:
1) Redirect from all incorrectly entered urls to the 404 page in .htaccess.
2) Correct server response after redirection (http page code should be 404, not 200).
3) Closing the 404 page from indexing in robots.txt

I would like to note right away that all of the above was written for self-written sites, mainly in PHP. There are plugins for WordPress to configure the same. But in this article we will look at how everything looks in reality. %)

Redirecting incorrect URLs to page 404

The first thing you do is create the 404 page itself so that you have somewhere to send people %).
Url redirection is configured in the .htaccess file
Just enter the line:
ErrorDocument 404 http://mysite.com/404.php
Where “mysite.com” is your domain, and http://mysite.com/404.php is the path to the real page. If your site is in html, then the line will look like:
ErrorDocument 404 http://mysite.com/404.html
The verification is very simple. After uploading the .htaccess file with the above line to the hosting, do a check by entering a obviously non-existent URL (broken link), for example: http://mysite.com/$%$%
If a redirect to the page you created occurs, then everything is working.
So, the entire .htaccess file, where ONLY a 404 redirect is configured, will look like this:
____________________________
RewriteEngine on
ErrorDocument 404 http://mysite.com/404.html
____________________________

Correct server response (http page code)

It is very important that when redirecting there is a correct server response, namely 404 Not Found.
This needs to be explained separately.

When requested, any url is assigned a status (http page code).
For all existing pages, this is: HTTP/1.1 200 OK
For redirected pages: HTTP/1.1 302 Found
If the page doesn't exist, it should be HTTP/1.1 404 Not Found

That is, no matter what URL is entered, it is assigned a status, a certain server response code.
You can check the server’s response on a resource such as bertal.ru or SEARCH CONCOLE GOOGLE – Scan/View as GOOGLE bot.
When you did not have a redirect via .htaccess to the 404 page, then any non-existent URL entered by the user, as well as broken links, received the response “HTTP/1.1 404 Not Found”

After you have set up a redirect to your 404 author page via .htaccess, as described above, then entering a broken link (invalid url that obviously does not exist), such as http://mysite.com/$%$% , the server response will:
- first HTTP/1.1 302 Found (redirection),
- and then HTTP/1.1 200 OK (page exists).

Check through bertal.ru.
What does this mean? This will mean that Google can add all broken links to its database (index), like existing pages with the contents of a 404 page. In fact, duplicate pages. And this is incredibly harmful for search engine optimization.

In this case, you need to do two things:
1) Set up the correct server response on the 404 page.
2) Block the 404 page from indexing. This is done through the robots.txt file

Configuring HTTP/1.1 404 Not Found server response for non-existent pages

The server response is customized thanks to the php function at the very beginning of the page:

Write it at the beginning of the 404 file.
As a result, we should receive a response to the broken link:

Close 404 page from indexing

You can block a page from indexing in the rodots.txt file. Be careful with this tool, because through this file your site essentially communicates with search robots!
The full text of the rodots.txt file, where ONLY the 404 page indexing is closed, looks like this:
____________________________
User-agent: *
Disallow:
Disallow: /404.php
____________________________

Code notes: "/404.php" means the path to the page. If on your site the 404.php page (or 404.html, respectively) is located in some folder, then the path will look like:
/holder/404.php
where "holder" is the name of the folder.

That's all about the 404 page. Check the operation of the page, redirections of broken links, and server responses.
I repeat: All of the above is for self-written sites. If you're using WordPress, you might want to look for a decent 404 error plugin.

452

HTTP status and error codes are like a short message from the server that appears at the top of the web page. It's not actually part of the web page. This message, returned when the server is contacted, lets you know how things were when the server received a request to view the page.

These messages are returned every time the browser interacts with the server, even if you don't see them. HTTP status codes are an invaluable tool for diagnosing and correcting errors encountered in a site's configuration.

This article provides the most common status codes and error codes.

Where do they come from?

Every time you click on a link or enter a URL and click " Enter", the browser sends a request to the server. It receives and processes the request, and then sends back the requested resources along with an HTTP header.

Status codes are delivered to the browser in the HTTP header. Although you don't see them. But when something goes wrong, the user is shown a status code in the browser. This is the server's way of saying: " Is there something wrong. Here is the code that explains what exactly».

Google HTTP Status Code 404

To see status codes that the browser doesn't normally display, you'll need special tools. There are corresponding extensions available for popular browsers such as Chrome and Firefox. There are also many services for displaying headers, such as Web Sniffer.

To see the HTTP status code using one of these tools, look for the line at the top of the report that says “Status: HTTP/1.1”. After it is the status code returned by the server.

HTTP status code classes

HTTP status codes are divided into 5 classes:

• 100: Information codes indicating that the request initiated by the browser is ongoing.
• 200: successful request codes. Returned when the browser request has been successfully received, recognized, and processed by the server.
• 300: Redirect codes are returned when the requested resource is replaced with a new one.
• 400: http errors that occur on the client side and indicate that there is a problem with the request.
• 500: Server error codes indicating that the request was accepted, but an error on the server prevented it from being completed.

List of HTTP status codes

There are over 40 different server status codes. But the ones you will encounter regularly are less than a dozen. The following is a list of HTTP status codes:

Status code 200

200: "Everything is fine." This is the code that is returned when a web page or resource behaves exactly as expected.

Status codes 300

301: " The requested resource has been moved permanently" This code is returned when a web page or resource is replaced by another resource. It is used to permanently redirect URLs.

302: This is an http error " The requested resource was moved but was found" This code is used to indicate that the requested resource was found, but not in the expected location. It is used to temporarily redirect URLs.

304: " The requested resource has not changed since the last time it was accessed" Reports that the resources stored in the browser cache have not changed. It is used to speed up the delivery of web pages by reusing previously downloaded resources.

Status codes 400

http error 403: " Access to this resource is denied" Returned when the user attempts to open a resource for which they do not have access rights. For example, an unauthorized user attempting to view password-protected content may result in a 403 error.

404: " The requested resource was not found" The most common error message. Means that the requested resource does not exist and the server does not know if it ever existed.

405: " Method not allowed" Generated when the hosting server (source server) supports the received method, but the target resource is missing.

406: " Inappropriate answer" The requested resource is only capable of generating content that is unacceptable according to the Accept headers sent in the request.

408: " The server timed out waiting for the rest of the request from the browser to arrive" Generated when the server aborts processing after timing out a complete request from the browser. In other words, the server did not receive the complete request sent by the browser. One possible reason could be network congestion causing packet loss between the browser and the server.

410: " The requested resource is missing and will not be returned" Similar to the 404 Not Found code, except that the 410 status code indicates that this status is expected on an ongoing basis.

429: This is an http error " Too many requests" Generated by the server when the user has sent too many requests in a given period of time ( speed limit). Sometimes the error can be caused by bots trying to access the site. In this case, you may need to change your WordPress admin login URL.

429 too many requests

499: " The client closed the request" Returned by NGINX when the client closes the request while NGINX is still processing it.

Status codes500

500: "N and the server encountered an error and the request could not be completed" Generic http code, also called " Internal Server Error" Something went wrong on the server and the requested resource was not delivered. This code is generated by third-party plugins when the PHP code or database connection fails.

Error establishing a connection to the database

501: "Not implemented." This error indicates that the server does not support the functionality required to complete the request. The error is almost always related to the server itself, and to resolve it you need to contact the hosting provider's support service.