aboutsummaryrefslogtreecommitdiffstats
path: root/doc/src/internals.xml
blob: 88da43a5351d63ed73f03df913516adfc46496aa (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
<para>
There are three basic parts of daemon: UDP listener, message processor and
message writer.
All these parts run in separate threads:
the listener receives messages as fast as possible and passes them to the
message processor,
the message processor performs time-consuming tasks and
message writer performs asynchronous output to files.
</para>
<para>
Datagrams are received by the UDP listener.
The listener emits raw messages (struct raw_message) which contain content
of datagram, sender address and reference to a source described
in configuration file.
</para>
<para>
Raw messages are passed to the main processing thread via queue.
Message processing involves the following tasks:
	<itemizedlist>
		<listitem>
			<para>
parse datagram:
pick out PRI, TIMESTAMP, HOSTNAME, TAG an CONTENT according to
<ulink url="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</ulink>;
			</para>
		</listitem>
		<listitem>
			<para>
convert CONTENT's encoding if specified;
			</para>
		</listitem>
		<listitem>
			<para>
determine sender host name if <option>usedns</option> option is set or just
convert IP address to string;
the result is saved in hostname cache to speed up subsequent resolutions;
			</para>
		</listitem>
		<listitem>
			<para>
multiplex message to logpaths and apply filters in logpaths;
in other words, messages are multiplexed to logpaths through filters;
messages in logpaths are represented with references to message structure
and message structure contains reference count;
			</para>
		</listitem>
	</itemizedlist>
Because destination file name may be a pattern, further multiplexing is
performed. Messages with similar HOSTNAME, TAG and CONTENT are coalesced.
</para>
<para>
Log rotation is initiated at process startup or at writing thread startup.
</para>
<para>
Old log files are deleted by the purger which is launched at process startup
or by the writing thread after file is closed.
	<screen>
                   mark message
+-------------------------------------------------+
|                                                 |
|    +----------------+    +-----+ message +--------------+
+--->|internal sources|--->|queue|-------->| - - - - -    |
     | (log_internal) |    +-----+         |           \  |
     +----------------+                    | main loop  \ |
                                           |             \| message
+------------+ raw message +-----+         | +------+    /|--------->
|UDP listener|------------>|queue|-------->|-|parser|- -  |
+------------+             +-----+         | +------+     |
                                           +--------------+

    +------+    +-----------------+    +-----------+
--->|filter|--->|   multiplexer   |-+->|destination|+
    |      |    |charset converter| |  |           ||
    +------+    +-----------------+ +->+-----------+|+
                         ^          +-> +-----------+|
                         |               +-----------+
                     +-------+
                     |logpath|+
                     +-------+|+
                      +-------+|
                       +-------+


   file         +-----------+    +-----+        +--------------+
destination: >--|multiplexer|-+->|queue|+  ---> |writing thread|+
                +-----------+ +->+-----+|+ ---> +--------------+|+
                      ^       +-> +-----+| --->  +--------------+|
                      |            +-----+        +--------------+
              +----------------+
              |filename pattern|
              +----------------+


   relay        +--------+
destination: >--|sendto()|
                +--------+
	</screen>
</para>