The new owners of a separate or regional network may at first not attach much importance to counting the traffic that they consume from the global network. But after receiving the first receipt for Internet services, the user realizes the importance of such an abstract, at first glance, concept as incoming Megabytes.

What it is

Speaking in simple words, traffic is the volume of information that a user sends and receives while browsing the Internet. Accordingly, a distinction is made between outgoing and incoming traffic.

There are several ways to waste incoming data:

How to find out the spent traffic

A computer user can quickly find out the amount and statistics of traffic consumption using the standard Start menu. Having made the transition to the "Control Panel". Then, choosing the option “ Network connections», You can see two icons with the necessary information:

"Main connection": the window contains data about external traffic. local network»The open window displays data on the activity of incoming and outgoing data, as well as statistics on the movement of fixed traffic packets. This information applies only to the internal network flow when using local resources.

But standard means often display inaccurate information, and the values ​​of some indicators can be quite incomprehensible to inexperienced users. Quite easy to use are programs designed to accurately measure traffic (such as NetWorx). Most of them are available for free download.

In special counter programs, there is step by step setup algorithm of work, which allows you to view the dynamics of incoming data at any time (up to a bit), set a warning function about exceeding the limit and track statistics for a week, month, year.

Measuring traffic waste is no less important for Android, whose users are online almost around the clock. Most quick way set control - setting the "Traffic control" option in the standard menu. It is possible to set separate restrictions on data acceptance via Wi-Fi or mobile network, their transmission can be disabled altogether.

Special applications (such as Data Usage or Internet Speed ​​Meter Lite) are designed not only for accounting, but also to save a limited amount of information. The Android user independently creates a template for displaying consumption, setting the desired period of time, warnings about exceeding the norm.

For example, Data Usage performs control of the transmitted information units, both via mobile and Wi-Fi network(which is relevant for frequent users public Internet, which have a limit on free access). Restrictions can be enabled for each network separately.

Video: ways to save mobile traffic

Savings methods

Android users can prevent device applications from independently connecting to the Internet. Updates should only be installed with the consent of the user. However, canceling auto-sync may negatively affect the performance of some standard programs(weather forecast or postal agent).

As mentioned above, each user on a separate line needs to either install a counter program, or at least know where this information can be found in the standard menu.

It is better to start economical use of the Internet on a computer with excessive cleaning active programs... The Process Hacker application scans the computer in operating mode and helps the user to eliminate traffic “eaters” on his own.

Photo: Scanning Firewall Programs

System programs damage traffic with automatic calls. You can control their activity using the free Firewall, which controls the flow of information and blocks unwanted access to the network for programs.

Determining where precious megabytes go is half the battle. Rational use of all services of the global network should become a habit. Programs such as uTorrent.exe should not run at startup and run idle.

How is internet traffic measured?

The smallest unit of measure of the received information is Bit. Depending on the situation and the consumed volume, the consumed data can be counted in Bytes, Kilobytes, Megabytes. The most common unit is Megabyte (MB).

Average sizes of the most popular files:

• three dozen web pages or 400 text pages: 1 MB;
• 5 pictures High Quality: 1 MB;
• one audio file: 3-12 Mb;
• one video clip: 30-200MB, film: 600-1400MB.

In conclusion, it should be said that monitoring and calculating Internet traffic allows you not only to avoid the need to pay inflated bills, but also to save significantly, without limiting yourself at the same time in using the capabilities of the Internet.

Any administrator sooner or later receives instructions from the management: “calculate who goes online and how much they download”. For providers, it is supplemented with the tasks "let in whoever you need, take payment, restrict access." What to count? How? Where? There is a lot of fragmentary information, they are not structured. Let's save a novice admin from tedious searches by equipping him general knowledge, and useful links to the materiel.
In this article I will try to describe the principles of organizing the collection, accounting and control of traffic in the network. We will consider the problematics of the question, and list possible ways retrieving information from network devices.

This is the first theoretical article in a series of articles devoted to the collection, accounting, management and billing of traffic and IT resources.

Internet access structure

In general, the structure of network access is as follows:

• External resources - the Internet, with all sites, servers, addresses and other things that do not belong to the network that you control.
• An access device is a router (hardware or PC-based), switch, VPN server, or hub.
• Internal resources - a set of computers, subnets, subscribers whose work in the network must be taken into account or controlled.
• Management or accounting server - a device on which a specialized software... It can be functionally combined with a software router.

In this structure, network traffic passes from external resources to internal ones, and back through the access device. It sends traffic information to the management server. The control server processes this information, stores it in the database, displays it, issues commands for blocking. However, not all combinations of access devices (methods), and collection and control methods are compatible. The various options will be discussed below.

Network traffic

First, you need to determine what is meant by "network traffic", and what useful statistical information can be extracted from the stream of user data.
IP version 4 remains the dominant protocol for interworking. The IP protocol complies with the 3rd layer of the OSI model (L3). Information (data) between the sender and the receiver is packed into packets - with a header and a "payload". The header determines where the packet comes from and where it goes (sender and receiver IP addresses), packet size, payload type. The bulk of network traffic consists of UDP and TCP payload packets - these are Layer 4 (L4) protocols. In addition to addresses, the headers of these two protocols contain port numbers that determine the type of service (application) that transfers data.

To transmit an IP packet over wire (or radio) network devices are forced to "wrap" (encapsulate) it into a layer 2 protocol (L2) packet. The most common protocol of this type is Ethernet. The actual transmission "to the wire" is at the 1st level. Usually, the access device (router) does not analyze packet headers at a level higher than 4th (except for smart firewalls).
Information from the fields of addresses, ports, protocols and length counters from L3 and L4 headers of data packets is the "source material" that is used in accounting and traffic management. The actual amount of information transmitted is found in the Length field of the IP header (including the length of the header itself). By the way, due to the fragmentation of packets due to the MTU mechanism, the total amount of transmitted data is always larger than the payload size.

The total length of the IP and TCP / UDP fields of the packet of interest to us in this context is 2 ... 10% of the total packet length. If you process and store all this information in batches, there won't be enough resources. Fortunately, the vast majority of traffic is structured in such a way that it consists of a set of "conversations" between external and internal network devices, the so-called "flows". For example, within one transfer operation email(SMTP protocol) a TCP session is opened between the client and the server. It is characterized by a constant set of parameters (Source IP, Source TCP Port, Destination TCP Port)... Instead of processing and storing information on a per-batch basis, it is much more convenient to store flow parameters (addresses and ports), as well as additional information - the number and sum of the lengths of transmitted packets in each direction, optional session duration, router interface indices, ToS field value, and so on. This approach is beneficial for connection-oriented protocols (TCP), where you can explicitly intercept when the session ends. However, even for non-session-oriented protocols, it is possible to perform aggregation and logical completion of the stream recording by, for example, a timeout. Below is an excerpt from the SQL database of its own billing system that logs information about traffic flows:

It should be noted the case when the access device performs address translation (NAT, masquerading) to organize access to the Internet for computers on the local network using one external public IP address. In this case, a special mechanism substitutes IP addresses and TCP / UDP ports of traffic packets, replacing internal (not routable on the Internet) addresses according to its own dynamic table broadcast. In such a configuration, it must be remembered that for the correct accounting of data on the internal hosts of the network, the collection of statistics should be done in a way and in the place where the translation result does not yet "depersonalize" internal addresses.

Methods for collecting traffic / statistics information

It is possible to capture and process information about passing traffic directly on the access device itself (PC-router, VPN-server), from this device transmitting it to a separate server (NetFlow, SNMP), or "from the wire" (tap, SPAN). Let's analyze all the options in order.

PC router

Let's consider the simplest case - an access device (router) based on a PC running Linux.

How to set up such a server, address translation and routing, written a lot... We are interested in the next logical step - information on how to obtain information about the traffic passing through such a server. There are three common ways:

• interception (copying) of packets passing through the server's network card using the libpcap library
• interception of packets passing through the built-in firewall
• using third-party tools for converting per-packet statistics (obtained by one of the two previous methods) into a stream of aggregated information netflow

Libpcap

In the first case, a copy of the packet passing through the interface, after passing the filter (man pcap-filter), can be requested client program on the server, written using this library. The packet comes with a Layer 2 header (Ethernet). It is possible to limit the length of the captured information (if we are only interested in the information from its header). Examples of such programs are tcpdump and Wireshark. There is a libpcap implementation for Windows. In the case of using address translation on a PC router, such interception can be performed only on its internal interface connected to local users. On the external interface, after translation, the IP packets do not contain information about the internal hosts of the network. However, with this method, it is impossible to take into account the traffic generated by the server itself on the Internet (which is important if it runs web or Post service).

Libpcap requires external support operating system, which currently boils down to installing a single library. In this case, the application (user) program that collects packages must:

• open the required interface
• specify the filter through which to pass received packets, the size of the captured part (snaplen), the size of the buffer,
• set the promisc parameter, which switches the network interface to the mode of capturing all packets passing by in general, and not only those addressed to the MAC address of this interface
• set a function (callback) called for each received packet.

When a packet is transmitted through the selected interface, after passing the filter, this function receives a buffer containing Ethernet, (VLAN), IP, etc. headers, overall size before snaplen. Since the libcap library copies packages, it cannot be blocked from passing through it. In this case, the program for collecting and processing traffic will have to use alternative methods, for example, calling a script to put the specified IP address in the traffic blocking rule.

Firewall

Capturing data passing through the firewall allows you to take into account both the traffic of the server itself and the traffic of network users, even when address translation is running. The main thing in this case is to correctly formulate the capture rule and put it in the right place. This rule activates the transfer of the packet towards the system library, from where the traffic accounting and control application can receive it. For Linux OS, iptables is used as the firewall, and the interceptors are ipq, netfliter_queue, or ulog. For FreeBSD OC - ipfw with tee or divert rules. In any case, the firewall mechanism is supplemented by the ability to work with a user program in the following way:

• User program - the traffic handler registers itself in the system using system call, or a library.
• A user program or external script installs a rule in the firewall that “wraps” the selected traffic (according to the rule) inside the handler.
• For each passing packet, the handler receives its contents in the form of a memory buffer (with IP headers, etc. After processing (accounting), the program must also tell the operating system kernel what to do next with such a packet - to discard or forward it. pass the modified package to the kernel.

Since the IP packet is not copied, but sent to the software for analysis, it becomes possible to "throw" it, and, consequently, to completely or partially restrict traffic of a certain type (for example, to a selected subscriber of the local network). However, if the application stops responding to the kernel about its decision (hangs, for example), traffic through the server is simply blocked.
It should be noted that the described mechanisms, with significant volumes of transmitted traffic, create an excessive load on the server, which is associated with the constant copying of data from the kernel to the user program. This drawback is deprived of the method of collecting statistics at the OS kernel level, with the issuance of application program aggregated statistics over the NetFlow protocol.

Netflow

This protocol was developed by Cisco Systems to export traffic information from routers for traffic accounting and analysis. The most popular version 5 now provides the recipient with a structured data stream in the form of UDP packets containing information about the passed traffic in the form of so-called flow records:

The amount of information about traffic is several orders of magnitude less than the traffic itself, which is especially important in large and distributed networks. Of course, it is impossible to block the transmission of information when collecting statistics on netflow (unless additional mechanisms are used).
Currently, the further development of this protocol is becoming popular - version 9, based on the flow record template structure, implementation for devices from other manufacturers (sFlow). Recently, the IPFIX standard has been adopted, which allows statistics to be transmitted over the protocols of deeper layers (for example, by the type of application).
The implementation of netflow sources (agents, probe) is available for PC routers, both in the form of utilities operating according to the mechanisms described above (flowprobe, softflowd), and directly built into the OS kernel (FreeBSD: ng_netgraph, Linux :). For software routers, the netflow statistics stream can be received and processed locally on the router itself, or sent over the network (transmission protocol - over UDP) to the receiving device (collector).


The collector program can collect information from many sources at once, being able to distinguish their traffic even with overlapping address spaces. With the help of additional tools, such as nprobe, it is also possible to carry out additional data aggregation, split streams or protocol conversion, which is important when managing a large and distributed network with dozens of routers.

The netflow export functions are supported by Cisco Systems, Mikrotik routers, and several others. Similar functionality (with other export protocols) is supported by all major network equipment manufacturers.

Libpcap "outside"

Let's complicate the task a little. What if your access device is a third-party hardware router? For example, D-Link, ASUS, Trendnet, etc. On it, most likely, it is impossible to put an additional software tool data retrieval. Alternatively, you have a smart access device, but it is not possible to configure it (there are no rights, or it is controlled by your provider). In this case, you can collect traffic information directly at the interface of the access device with the internal network, using the "hardware" means of copying packets. In this case, you will definitely need a stand-alone server with a dedicated network card to receive copies of Ethernet packets.
The server must use the mechanism for collecting packets using the libpcap method described above, and our task is to send a data stream to the input of a dedicated network card that is identical to the one leaving the access server. To do this, you can use:

• Ethernet hub: A device that simply forwards packets between all of its ports indiscriminately. In modern realities, it can be found somewhere in a dusty warehouse, and it is not recommended to use this method: unreliable, low speed(there are no hubs at 1 Gbps)
• Ethernet is a switch with the ability to mirror (mirroring, SPAN ports. Modern intelligent (and expensive) switches allow copying all traffic (incoming, outgoing, both) of another physical interface, VLAN, including remote (RSPAN) to a specified port
• Hardware splitter, which may require installation to collect two network cards instead of one - and this is in addition to the main, system.

Naturally, you can configure the SPAN port on the access device itself (router), if it allows it - Cisco Catalyst 6500, Cisco ASA. Here is an example of such a configuration for a Cisco switch:
monitor session 1 source vlan 100! where do we get the packages
monitor session 1 destination interface Gi6 / 3! where do we issue packages

SNMP

What if there is no router under our control, there is no desire to communicate with netflow, we are not interested in the details of our users' traffic. They are simply connected to the network through a managed switch, and we just need to roughly estimate the amount of traffic going to each of its ports. As you know, network devices with the ability remote control support, and can display the counters of packets (bytes) passing through network interfaces... It will be correct to use the standardized SNMP remote management protocol to poll them. Using it, you can simply get not only the values ​​of the specified counters, but also other parameters, such as the name and description of the interface, the MAC addresses visible through it, and other useful information... This is done as utilities command line(snmpwalk), graphical SNMP browsers, and more sophisticated network monitoring programs (rrdtools, cacti, zabbix, whats up gold, etc.). But, this method has two significant drawbacks:

• traffic can be blocked only by completely disabling the interface, using the same SNMP
• SNMP traffic counters refer to the sum of the lengths of Ethernet packets (unicast, broadcast and multicast separately), while the rest of the previously described means give values ​​relative to IP packets. This creates a noticeable discrepancy (especially on short packets) due to the overhead caused by the length of the Ethernet header (however, this can be roughly dealt with: L3_bytes = L2_bytes - L2_packets * 38).

VPN

Separately, it is worth considering the case of user access to the network by explicitly establishing a connection to the access server. A classic example is the good old dial-up, an analogue of which in modern world are VPN services remote access(PPTP, PPPoE, L2TP, OpenVPN, IPSEC)


The access device not only routes users' IP traffic, but also acts as a dedicated VPN server and terminates logical tunnels (often encrypted) within which user traffic is transmitted.
To account for such traffic, you can use all the tools described above (and they are well suited for deep analysis by port / protocol), as well as additional mechanisms that provide VPN access controls. First of all it will be about the RADIUS protocol. His work is a rather complex topic. We will briefly mention that the control (authorization) of access to the VPN server (RADIUS client) is controlled by special application(RADIUS server), which has a base ( text file, SQL, Active Directory) of valid users with their attributes (connection speed limits, assigned IP addresses). In addition to the authorization process, the client periodically transmits accounting messages to the server, information about the status of each current running VPN session, including the counters of transmitted bytes and packets.

Conclusion

Let's summarize all the above methods of collecting traffic information together:

Let's summarize a little. In practice, there are a large number of methods for connecting the network you manage (with clients or office subscribers) to the external network infrastructure, using a number of access means - software and hardware routers, switches, VPN servers. However, in almost any case, you can come up with a scheme when information about the traffic transmitted over the network can be sent to the software or hardware tool for its analysis and control. It is also possible that this tool will allow you to carry out feedback with an access device, applying intelligent algorithms to limit access for individual clients, protocols and more.
This concludes the analysis of the materiel. Of the unanswered topics remained:

• how and where the collected traffic data goes
• traffic accounting software
• what is the difference between billing and a simple "counting room"
• how you can impose traffic restrictions
• accounting and limiting visited websites

YouTube video hosting is one of the most admired by users social networks and in general this kind of services. It does not exclude the possibility of creating (opening) and broadcasting your own channel. But! Somehow it is necessary to attract spectators to it. Now we will talk about the very first step in creating a performance. own project... How to make a beautiful YouTube channel header? Few people think about such a design, but this is, in fact, a manifestation of some kind of marketing move to attract a future audience.

How to make a YouTube header: general guidelines

If you approach the issue with the utmost seriousness, the created channel should first of all attract a potential visitor with a picture or animation that is displayed in a thumbnail or when entering a resource.

Therefore, the question of how to do cool hat for a YouTube channel, should start with just that choice. Among the most frequently preferred graphic elements of design, the background and, in fact, the picture, most often dominate. But even here not everything is as simple as it might seem at first glance. The fact is that it will not be possible to install a selected image from the Internet or saved on a hard disk to a channel.

Presented video hosting requirements

The hosting itself has its own limitations regarding, first of all, the size of the image (animation) and the file uploaded for this. Image or video parameters should correspond to the recommended size of 2560 x 1440 pixels per inch (although slight deviations are possible). In addition, the requirements clearly state that the size of the uploaded file should not exceed 4 MB.

One can disagree with this, since many users (and this is confirmed by their reviews) quite calmly download graphic objects volume of about 10 MB, and that's okay.

Background and image size issues

In the question of how to make a 3D header for a YouTube channel or deviate from the design standards, you will have to recall your skills in owning graphic or video editors that allow you to produce three-dimensional modeling (and sometimes even 3D animation). But as for precisely the so-called headers, you can limit yourself to the knowledge of creating and editing animated GIF files.

3D animation is not an easy task, so an ordinary user can use many special template schemes.

Using templates

We continue to look at how to make a header for a YouTube channel. Further - directly about the primary scheme, which is taken as a basis when creating any channel. It is a gray box with blue lines crossing it vertically and horizontally. Such lines delimit the fields responsible for viewing the installed header on different devices(from the TV panel high resolution to the screen mobile phone). This function markup is activated or deactivated Ctrl+ H.

What to do with all this? Yes, just open it in the same Photoshop editor and insert your image, logo, text or any other element into the required fields, after which the file is saved in standard JPEG formats or PNG.

How to make an animated YouTube header and background issues

For animation, you should initially use even the simplest program that can work with the Flash format, in which most of the primitive online toys are represented. How to make animation using it is another question.

How to make a header for a YouTube channel seems to be clear in general terms. So, let's say she's ready. Now it needs to be installed on the channel. Now on the agenda is the placement of the background. To do this, having entered under your registration record, you should move the cursor to the place where this design element will be located, and click in the upper right corner of the pencil image or the add art button.

From the menu that appears, choose to change the design, after which a window will appear in which you need to drag the banner or finished image... In the case of using templates, you do not need to crop anything additionally (otherwise it may degrade the quality). Then all that remains is to press the select button, and the background will appear in the channel header.

Naturally, all these operations must be performed only when entering under your own registration record, as already mentioned above.

Something in the end

Please note: when considering how to make a header for a YouTube channel, you should take into account that when you enter the channel, sound may be played. In particular, this applies only to those cases when the header uses animation or video linked to the audio track.

Professional audio processing is also not easy, but in the simplest case, you can use editors that can combine overlay audio and video materials, such as Sound Forge or Adobe Audition. But ideally, it's better to edit the sound initially, and only then overlay it on the video. When editing in sound programs, it is advisable to use the normalization function (this will save you from the appearance of peak loads and equalize the sound along the entire playback line). For a primitive combination of video and audio, even a non-professional can use the simplest program"VideoMASTER".

With regard to audio preprocessing, if you absolutely do not understand these processes, you can use the universal application AAMS (Auto Audio Mastering System), which will change the sound of the track in accordance with the selected template from its own list or make editing based on the analysis of any composition you choose.

However, this is not the most important thing. The essence of creating a header, as well as filling a channel, is to be creative, and not use template schemes, although with their help you can learn a lot, but only at first. And then - a flight of imagination ...

No, of course, you can turn to special online services that can do all this for you. But can this be called a creative process?