9 Bře
2013
9 Bře
'13
11:51
On a server with 16 GB of RAM, my instance of BIND can load my 5174
zones into memory and use around 13 GB.
Knot didn't do so well. At some point while trying to XFR-in these
zones, it hit the memory limit and the Linux out-of-memory killer came
along and killed it.
When I started it up again, it began loading zones in from the disk, but
then appeared to go into some kind of loop, and the CPU usage was 100%.
This is usually a sign that it is stuck in some kind of loop or
deadlock. The only want to stop it is with a KILL signal (TERM doesn't
work). The log didn't output anything.
How can I help debug this?
Do you have any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
Regards,
Anand
11 Bře
11 Bře
15:33
On 9 March 2013 11:51, Anand Buddhdev <anandb(a)ripe.net> wrote:
On a server with 16 GB of RAM, my instance of BIND can
load my 5174
zones into memory and use around 13 GB.
Knot didn't do so well. At some point while trying to XFR-in these
zones, it hit the memory limit and the Linux out-of-memory killer came
along and killed it.
When I started it up again, it began loading zones in from the disk, but
then appeared to go into some kind of loop, and the CPU usage was 100%.
This is usually a sign that it is stuck in some kind of loop or
deadlock. The only want to stop it is with a KILL signal (TERM doesn't
work). The log didn't output anything.
I believe this is worked on in another thread.
How can I help debug this?
Do you have any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
We estimate the Knot will take around 4-5x the size of the zone on the disk,
but there are other variables and it depends on the content of the zone as well.
Also, the memory requirements may be double that during transfers, I
think it's described
in the manual.
We have a new parser that would make compiling obsolete and some
memory optimizations
that would make it much leaner in preparations for the next release,
so that will hopefully help.
Kind regards,
Marek
Regards,
Anand
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
17:35
On 11/03/2013 15:33, Marek Vavruša wrote:
Hi Marek,
Okay, thanks for this. I'll look at the manual in a bit.
Do you have
any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
We estimate the Knot will take around 4-5x the size of the zone on the disk,
but there are other variables and it depends on the content of the zone as well.
Also, the memory requirements may be double that during transfers, I
think it's described
in the manual. We have a new parser that would make compiling
obsolete and some
memory optimizations
that would make it much leaner in preparations for the next release,
so that will hopefully help.
This sounds promising. Which release is it going to be in?
Regards,
Anand
12 Bře
12 Bře
11:21
On 11 March 2013 17:35, Anand Buddhdev <anandb(a)ripe.net> wrote:
On 11/03/2013 15:33, Marek Vavruša wrote:
Hi Marek,
Okay, thanks for this. I'll look at the manual in a bit.
The new parser and memory optimizations are going to 1.3.
I'll try my best to solve the thundering herd problem with bootstrap
as well to the 1.3.
Kind regards,
Marek
Do you
have any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
We estimate the Knot will take around 4-5x the size of the zone on the disk,
but there are other variables and it depends on the content of the zone as well.
Also, the memory requirements may be double that during transfers, I
think it's described
in the manual. We have a new parser that would make compiling
obsolete and some
memory optimizations
that would make it much leaner in preparations for the next release,
so that will hopefully help.
This sounds promising. Which release is it going to be in?
Regards,
Anand 
20 Bře
20 Bře
23:35
Hi Marek,
On Mar 11, 2013, at 15:33 , Marek Vavruša wrote:
On 9 March 2013 11:51, Anand Buddhdev
<anandb(a)ripe.net> wrote:
I'm also very interested in this.
In a recent test I loaded a three large zones (one very large) into a BIND9 server with
16GB memory w/o problems. Load time about 5 min and memory usage about 13GB. NSD3 hit out
of memory, but succeeded with 32GB memory (load time around 20 min). Knot 1.2rc3 hit out
of memory also with 32GB so I went for 64GB which succeded. However, Knot took even longer
to load than NSD (and as I only have three zones, it is not the thundering herd situation)
and the final memory usage was close to 55GB!
That's a lot more memory than BIND9 and also a lot longer load time. Once loaded
everything is fine, but the memory consumption is really a concern at the moment.
Regards,
Johan
On a server with 16 GB of RAM, my instance of
BIND can load my 5174
zones into memory and use around 13 GB.
Knot didn't do so well. At some point while trying to XFR-in these
zones, it hit the memory limit and the Linux out-of-memory killer came
along and killed it.
When I started it up again, it began loading zones in from the disk, but
then appeared to go into some kind of loop, and the CPU usage was 100%.
This is usually a sign that it is stuck in some kind of loop or
deadlock. The only want to stop it is with a KILL signal (TERM doesn't
work). The log didn't output anything.
I believe this is worked on in another thread.
How can I help debug this?
Do you have any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
We estimate the Knot will take around 4-5x the size of the zone on the disk,
but there are other variables and it depends on the content of the zone as well.
Also, the memory requirements may be double that during transfers, I
think it's described
in the manual.
We have a new parser that would make compiling obsolete and some
memory optimizations that would make it much leaner in preparations for the next release,
so that will hopefully help.
22 Bře
22 Bře
15:58
Hi Johan,
I agree, we have tested current 1.3 prerelease and rough estimate is
that it takes about half of what
1.2 needs to load a zone, or slightly more than that. Of course it
depends on the contents and other factors. It would also be faster to
load and doesn't require zone precompilation.
Kind regards,
Marek
On 20 March 2013 23:35, Johan Ihrén <johani(a)johani.org> wrote:
Hi Marek,
On Mar 11, 2013, at 15:33 , Marek Vavruša wrote:
On 9 March 2013 11:51, Anand Buddhdev
<anandb(a)ripe.net> wrote:
I'm also very interested in this.
In a recent test I loaded a three large zones (one very large) into a BIND9 server with
16GB memory w/o problems. Load time about 5 min and memory usage about 13GB. NSD3 hit out
of memory, but succeeded with 32GB memory (load time around 20 min). Knot 1.2rc3 hit out
of memory also with 32GB so I went for 64GB which succeded. However, Knot took even longer
to load than NSD (and as I only have three zones, it is not the thundering herd situation)
and the final memory usage was close to 55GB!
That's a lot more memory than BIND9 and also a lot longer load time. Once loaded
everything is fine, but the memory consumption is really a concern at the moment.
Regards,
Johan
On a server with 16 GB of RAM, my instance of
BIND can load my 5174
zones into memory and use around 13 GB.
Knot didn't do so well. At some point while trying to XFR-in these
zones, it hit the memory limit and the Linux out-of-memory killer came
along and killed it.
When I started it up again, it began loading zones in from the disk, but
then appeared to go into some kind of loop, and the CPU usage was 100%.
This is usually a sign that it is stuck in some kind of loop or
deadlock. The only want to stop it is with a KILL signal (TERM doesn't
work). The log didn't output anything.
I believe this is worked on in another thread.
How can I help debug this?
Do you have any numbers on how much RAM Knot will require given a bunch
of zones? This would allow me to estimate how much RAM I will need in a
server for the zones I have.
We estimate the Knot will take around 4-5x the size of the zone on the disk,
but there are other variables and it depends on the content of the zone as well.
Also, the memory requirements may be double that during transfers, I
think it's described
in the manual.
We have a new parser that would make compiling obsolete and some
memory optimizations that would make it much leaner in preparations for the next release,
so that will hopefully help.
4294
days inactive
4307
days old
5 comments
3 participants
participants (3)
-
Anand Buddhdev -
Johan Ihrén -
Marek Vavruša