28 Kvě
2014
28 Kvě
'14
10:01
Hello,
I'm trying to understand if Knot can solve our performance problem with big scale
dynamic DNS deployment. Currently we use BIND and the performance of Dynamic Updates is
insufficient for our requirements. Doing some research I understood that BIND processes
updates sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with updates. But reads is not an issue in our
deployment. My question is, does Knot able to process multiple updates in parallel (and
not sequentially) and provide a better scale for _updates_?
As far as I understand it depends on data structure hierarchy implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex@jajah.com<mailto:alex@jajah.com>
28 Kvě
28 Kvě
10:23
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve our
performance problem with big
scale dynamic DNS deployment. Currently we use BIND and the performance of
Dynamic Updates is insufficient for our requirements. Doing some research I
understood that BIND processes updates sequentially and can't really benefit
from SMP.
I understand how RCU allows parallel reads with
updates. But reads is not an
issue in our deployment. My question is, does Knot able to process multiple
updates in parallel (and not sequentially) and provide a better scale for _
updates _?
As far as I understand it depends on data structure
hierarchy implemented in
Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
11:29
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel updates
which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve our
performance problem
with big scale dynamic DNS deployment. Currently we use BIND and the
performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads is
not an issue in our deployment. My question is, does Knot able to
process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data structure
hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
5 Čen
5 Čen
15:28
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it
comes to DDNS, and it's partly because of RCU, since for every update,
we have to do a partial copy of the zone. We'll be doing a lot of
improvements to Knot's internal structures during summer, and we'll keep
DDNS performance in mind, so hopefully things will get better soon.
As of now, we do not plan to parallelize DDNS processing. We've done
some measurements, and we're confident we can easily achieve performance
of ~5k QPS in single thread only. We'll let you know once we have some
working code and we can discuss it further then. I'm sorry I don't have
anything better for you, but hopefully that will change with the 1.6.0
release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and the
performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads is
not an issue in our deployment. My question is, does Knot able to
process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
15:38
Thanks Jan,
5k of dynamic updates does the trick for us, at least for one year.
We'll do stress tests and will share the results to the forum as well.
Looking forward for the 1.6.0 release.
BR, Alex.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Thursday, June 05, 2014 4:28 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it comes to DDNS, and
it's partly because of RCU, since for every update, we have to do a partial copy of
the zone. We'll be doing a lot of improvements to Knot's internal structures
during summer, and we'll keep DDNS performance in mind, so hopefully things will get
better soon.
As of now, we do not plan to parallelize DDNS processing. We've done some
measurements, and we're confident we can easily achieve performance of ~5k QPS in
single thread only. We'll let you know once we have some working code and we can
discuss it further then. I'm sorry I don't have anything better for you, but
hopefully that will change with the 1.6.0 release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and the
performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads
is not an issue in our deployment. My question is, does Knot able to
process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
18 Čen
18 Čen
11:33
Hi Jan,
What does this code in update.c do? We're experiencing this error.
if (pthread_mutex_trylock(&zone->ddns_lock) != 0) {
qdata->rcode = KNOT_RCODE_SERVFAIL;
log_zone_error("Failed to process UPDATE for "
"zone %s: Another UPDATE in progress.\n",
zone->conf->name);
return NS_PROC_FAIL;
}
Is it a global lock on the whole zone, meaning that only one concurrent update can run and
the rest are rejected?
So should we force KNOT to a single thread?
What about using lock instead of trylock and increase a number of threads?
Thanks, Alex.
-----Original Message-----
From: knot-dns-users-bounces(a)lists.nic.cz [mailto:knot-dns-users-bounces@lists.nic.cz] On
Behalf Of Alex Massover
Sent: 05 June 2014 16:38
To: Jan Kadlec
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Thanks Jan,
5k of dynamic updates does the trick for us, at least for one year.
We'll do stress tests and will share the results to the forum as well.
Looking forward for the 1.6.0 release.
BR, Alex.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Thursday, June 05, 2014 4:28 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it comes to DDNS, and
it's partly because of RCU, since for every update, we have to do a partial copy of
the zone. We'll be doing a lot of improvements to Knot's internal structures
during summer, and we'll keep DDNS performance in mind, so hopefully things will get
better soon.
As of now, we do not plan to parallelize DDNS processing. We've done some
measurements, and we're confident we can easily achieve performance of ~5k QPS in
single thread only. We'll let you know once we have some working code and we can
discuss it further then. I'm sorry I don't have anything better for you, but
hopefully that will change with the 1.6.0 release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and the
performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads
is not an issue in our deployment. My question is, does Knot able to
process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
12:05
Hello Alex,
1.5.0-rc1 does not have this limitation, updates are stored in a queue
there.
As for this code, we could change it to pthread_mutex_lock(), but that
way updates could block all the answering threads. This is somewhat okay
for hidden master scenario, but not okay in general. We could use queue
in the 1.4.x as well, but since it's already in 1.5.x, would you be
wiling to give that a try?
Oh, and just to clarify one thing, even with 1.5, DDNS will be *really,
really* slow with big zones, nowhere near BIND's performance (think 1
qps of actual processed updates with 100M zone :( - queuing itself will
be much faster, but I suppose it's the actual processing (i.e. when the
changes are visible) that's important to you). 5k qps were only
possible when we turned RCU and some other stuff off, but sadly, a
proper solution will come no sooner than in 1.6. If your BIND setup is
no longer suitable, we could try to hack some code that would speed up
the updates using our current code, but I'd rather focus on the proper
solution that does not have similar design problems to our current
solution.
Regards,
Jan
On Wed, 2014-06-18 at 09:33 +0000, Alex Massover wrote:
Hi Jan,
What does this code in update.c do? We're experiencing this error.
if (pthread_mutex_trylock(&zone->ddns_lock) != 0) {
qdata->rcode = KNOT_RCODE_SERVFAIL;
log_zone_error("Failed to process UPDATE for "
"zone %s: Another UPDATE in progress.\n",
zone->conf->name);
return NS_PROC_FAIL;
}
Is it a global lock on the whole zone, meaning that only one concurrent update can run
and the rest are rejected?
So should we force KNOT to a single thread?
What about using lock instead of trylock and increase a number of threads?
Thanks, Alex.
-----Original Message-----
From: knot-dns-users-bounces(a)lists.nic.cz [mailto:knot-dns-users-bounces@lists.nic.cz] On
Behalf Of Alex Massover
Sent: 05 June 2014 16:38
To: Jan Kadlec
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Thanks Jan,
5k of dynamic updates does the trick for us, at least for one year.
We'll do stress tests and will share the results to the forum as well.
Looking forward for the 1.6.0 release.
BR, Alex.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Thursday, June 05, 2014 4:28 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it comes to DDNS,
and it's partly because of RCU, since for every update, we have to do a partial copy
of the zone. We'll be doing a lot of improvements to Knot's internal structures
during summer, and we'll keep DDNS performance in mind, so hopefully things will get
better soon.
As of now, we do not plan to parallelize DDNS processing. We've done some
measurements, and we're confident we can easily achieve performance of ~5k QPS in
single thread only. We'll let you know once we have some working code and we can
discuss it further then. I'm sorry I don't have anything better for you, but
hopefully that will change with the 1.6.0 release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and the
performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads
is not an issue in our deployment. My question is, does Knot able to
process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital
Architect
M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
12:22
I see, thanks, we'll try 1.5.0.
BTW, does it have the same bug with NAPTR ddns updates? I can't figure out as the code
changed since 1.4.6.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Wednesday, June 18, 2014 1:05 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
1.5.0-rc1 does not have this limitation, updates are stored in a queue there.
As for this code, we could change it to pthread_mutex_lock(), but that way updates could
block all the answering threads. This is somewhat okay for hidden master scenario, but not
okay in general. We could use queue in the 1.4.x as well, but since it's already in
1.5.x, would you be wiling to give that a try?
Oh, and just to clarify one thing, even with 1.5, DDNS will be *really,
really* slow with big zones, nowhere near BIND's performance (think 1 qps of actual
processed updates with 100M zone :( - queuing itself will be much faster, but I suppose
it's the actual processing (i.e. when the changes are visible) that's important to
you). 5k qps were only possible when we turned RCU and some other stuff off, but sadly, a
proper solution will come no sooner than in 1.6. If your BIND setup is no longer suitable,
we could try to hack some code that would speed up the updates using our current code, but
I'd rather focus on the proper solution that does not have similar design problems to
our current solution.
Regards,
Jan
On Wed, 2014-06-18 at 09:33 +0000, Alex Massover wrote:
Hi Jan,
What does this code in update.c do? We're experiencing this error.
if (pthread_mutex_trylock(&zone->ddns_lock) != 0) {
qdata->rcode = KNOT_RCODE_SERVFAIL;
log_zone_error("Failed to process UPDATE for "
"zone %s: Another UPDATE in progress.\n",
zone->conf->name);
return NS_PROC_FAIL;
}
Is it a global lock on the whole zone, meaning that only one concurrent update can run
and the rest are rejected?
So should we force KNOT to a single thread?
What about using lock instead of trylock and increase a number of threads?
Thanks, Alex.
-----Original Message-----
From: knot-dns-users-bounces(a)lists.nic.cz
[mailto:knot-dns-users-bounces@lists.nic.cz] On Behalf Of Alex
Massover
Sent: 05 June 2014 16:38
To: Jan Kadlec
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Thanks Jan,
5k of dynamic updates does the trick for us, at least for one year.
We'll do stress tests and will share the results to the forum as well.
Looking forward for the 1.6.0 release.
BR, Alex.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Thursday, June 05, 2014 4:28 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it comes to DDNS,
and it's partly because of RCU, since for every update, we have to do a partial copy
of the zone. We'll be doing a lot of improvements to Knot's internal structures
during summer, and we'll keep DDNS performance in mind, so hopefully things will get
better soon.
As of now, we do not plan to parallelize DDNS processing. We've done some
measurements, and we're confident we can easily achieve performance of ~5k QPS in
single thread only. We'll let you know once we have some working code and we can
discuss it further then. I'm sorry I don't have anything better for you, but
hopefully that will change with the 1.6.0 release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and
the performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads
is not an issue in our deployment. My question is, does Knot able
to process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital Architect M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
3809
days inactive
3830
days old
7 comments
3 participants
participants (3)
-
Alex Massover -
Jan Kadlec -
Ondřej Surý