18 Kvě
2016
18 Kvě
'16
19:33
Hi,
Linux 4.6 was just released, and it includes a new feature called
"Kernel Connection Multiplexor":
http://kernelnewbies.org/Linux_4.6#head-d86a7a8affd7cefef85fff400e39403718b…
1.5. Kernel Connection Multiplexor, a facility for accelerating
application layer protocols
This release adds Kernel Connection Multiplexor (KCM), a facility
that provides a message-based interface over TCP for accelerating
application layer protocols. The motivation for this is based on the
observation that although TCP is byte stream transport protocol with
no concept of message boundaries, a common use case is to implement
a framed application layer protocol running over TCP. Most TCP
stacks offer byte stream API for applications, which places the
burden of message delineation, message I/O operation atomicity, and
load balancing in the application.
With KCM an application can efficiently send and receive application
protocol messages over TCP using a datagram interface. The kernel
provides necessary assurances that messages are sent and received
atomically. This relieves much of the burden applications have in
mapping a message based protocol onto the TCP stream. KCM also make
application layer messages a unit of work in the kernel for the
purposes of steerng and scheduling, which in turn allows a simpler
networking model in multithreaded applications. In order to
delineate message in a TCP stream for receive in KCM, the kernel
implements a message parser based on BPF, which parses application
layer messages and returns a message length. Nearly all binary
application protocols are parseable in this manner, so KCM should be
applicable across a wide range of applications.
DNS-over-TCP is definitely amenable to this scheme, since messages are
framed with a 2-byte message length value. It also sounds like it can be
combined with recvmmsg():
Q: What about the problem of a connections with very slow rate of
incoming data? As a result your application can get storms of very
short reads. And it actually happens a lot with connection from
mobile devices and it is a problem for servers handling a lot of
connections.
A: The storm of short reads will occur regardless of whether KCM is used
or not. KCM does have one advantage in this scenario though, it will
only wake up the application when a full message has been received,
not for each packet that makes up part of a bigger messages. If a
bunch of small messages are received, the application can receive
messages in batches using recvmmsg.
Maybe this could help speed up a DNS server, or even improve resistance
against slowloris style TCP attacks.
--
Robert Edmonds
edmonds(a)debian.org
19 Kvě
19 Kvě
11:05
Hello Robert,
thank you for pointing this out. I've just noticed the KCM feature in
the Linux kernel, however I haven't thought about advantages it could
bring for us. I agree, that this could help us quite a lot with TCP.
Cheers,
Jan
On 18.5.2016 19:33, Robert Edmonds wrote:
Hi,
Linux 4.6 was just released, and it includes a new feature called
"Kernel Connection Multiplexor":
http://kernelnewbies.org/Linux_4.6#head-d86a7a8affd7cefef85fff400e39403718b…
1.5. Kernel Connection Multiplexor, a facility for accelerating
application layer protocols
This release adds Kernel Connection Multiplexor (KCM), a facility
that provides a message-based interface over TCP for accelerating
application layer protocols. The motivation for this is based on the
observation that although TCP is byte stream transport protocol with
no concept of message boundaries, a common use case is to implement
a framed application layer protocol running over TCP. Most TCP
stacks offer byte stream API for applications, which places the
burden of message delineation, message I/O operation atomicity, and
load balancing in the application.
With KCM an application can efficiently send and receive application
protocol messages over TCP using a datagram interface. The kernel
provides necessary assurances that messages are sent and received
atomically. This relieves much of the burden applications have in
mapping a message based protocol onto the TCP stream. KCM also make
application layer messages a unit of work in the kernel for the
purposes of steerng and scheduling, which in turn allows a simpler
networking model in multithreaded applications. In order to
delineate message in a TCP stream for receive in KCM, the kernel
implements a message parser based on BPF, which parses application
layer messages and returns a message length. Nearly all binary
application protocols are parseable in this manner, so KCM should be
applicable across a wide range of applications.
DNS-over-TCP is definitely amenable to this scheme, since messages are
framed with a 2-byte message length value. It also sounds like it can be
combined with recvmmsg():
Q: What about the problem of a connections with very slow rate of
incoming data? As a result your application can get storms of very
short reads. And it actually happens a lot with connection from
mobile devices and it is a problem for servers handling a lot of
connections.
A: The storm of short reads will occur regardless of whether KCM is used
or not. KCM does have one advantage in this scenario though, it will
only wake up the application when a full message has been received,
not for each packet that makes up part of a bigger messages. If a
bunch of small messages are received, the application can receive
messages in batches using recvmmsg.
Maybe this could help speed up a DNS server, or even improve resistance
against slowloris style TCP attacks.
22:06
Hello again.
I'm reducing my enthusiasm for KCM after doing some more research. If
I understand the KCM design correctly, then our authoritative name
server won't benefit its features that much.
The multiplexor bridges sockets for application layer messages
processing with connected TCP sockets. The architecture expects all TCP
sockets attached to one multiplexor to be connected to the same
destination. So the KCM is doing kind of a load balancing both for
message processing and network I/O.
Hence we can't use the KCM for listening sockets (it would be awesome if
we could) as we need already established connections. And we won't
benefit message distribution as we can easily process multiple queries
on a single TCP connection one-by-one. Out-of-order processing makes no
sense for authoritative server. And I think the overhead of the KCM
setup would be much higher than the current DNS messages assembly in
user space.
As for the resistance against various attacks... I don't know. I don't
have any strong opinion on that (yet).
Regards,
Jan
On 19.5.2016 11:05, Jan Včelak wrote:
Hello Robert,
thank you for pointing this out. I've just noticed the KCM feature in
the Linux kernel, however I haven't thought about advantages it could
bring for us. I agree, that this could help us quite a lot with TCP.
Cheers,
Jan
On 18.5.2016 19:33, Robert Edmonds wrote:
> Hi,
>
> Linux 4.6 was just released, and it includes a new feature called
> "Kernel Connection Multiplexor":
>
> http://kernelnewbies.org/Linux_4.6#head-d86a7a8affd7cefef85fff400e39403718b…
>
> 1.5. Kernel Connection Multiplexor, a facility for accelerating
> application layer protocols
>
> This release adds Kernel Connection Multiplexor (KCM), a facility
> that provides a message-based interface over TCP for accelerating
> application layer protocols. The motivation for this is based on the
> observation that although TCP is byte stream transport protocol with
> no concept of message boundaries, a common use case is to implement
> a framed application layer protocol running over TCP. Most TCP
> stacks offer byte stream API for applications, which places the
> burden of message delineation, message I/O operation atomicity, and
> load balancing in the application.
>
> With KCM an application can efficiently send and receive application
> protocol messages over TCP using a datagram interface. The kernel
> provides necessary assurances that messages are sent and received
> atomically. This relieves much of the burden applications have in
> mapping a message based protocol onto the TCP stream. KCM also make
> application layer messages a unit of work in the kernel for the
> purposes of steerng and scheduling, which in turn allows a simpler
> networking model in multithreaded applications. In order to
> delineate message in a TCP stream for receive in KCM, the kernel
> implements a message parser based on BPF, which parses application
> layer messages and returns a message length. Nearly all binary
> application protocols are parseable in this manner, so KCM should be
> applicable across a wide range of applications.
>
> DNS-over-TCP is definitely amenable to this scheme, since messages are
> framed with a 2-byte message length value. It also sounds like it can be
> combined with recvmmsg():
>
> Q: What about the problem of a connections with very slow rate of
> incoming data? As a result your application can get storms of very
> short reads. And it actually happens a lot with connection from
> mobile devices and it is a problem for servers handling a lot of
> connections.
>
> A: The storm of short reads will occur regardless of whether KCM is used
> or not. KCM does have one advantage in this scenario though, it will
> only wake up the application when a full message has been received,
> not for each packet that makes up part of a bigger messages. If a
> bunch of small messages are received, the application can receive
> messages in batches using recvmmsg.
>
> Maybe this could help speed up a DNS server, or even improve resistance
> against slowloris style TCP attacks.
2904
days inactive
2905
days old
2 comments
2 participants
participants (2)
-
Jan Včelak -
Robert Edmonds