9 Lis
2016
9 Lis
'16
16:23
Hola,
As recently a variety of people claimed to me that 'running DNSSEC is
not scary'.... I was like, lets try again after having tried it ~10
years ago and it failing miserably.
DNSSEC auto-maintain style looks to be better; still not as nice as
running dual-nsd's in master mode; but we'll live with those moving parts.
It ran fine for a bit, till I noticed the signatures of the zone had
expired and noticed the master simply did not bother to update the sigs
anymore. So much for 'automatic' mode.
Restarting it caused a nice crash:
Debian provided jessie-backports 2.3.1-1~bpo8:
```
knotd[6679]: *** Error in `/usr/sbin/knotd': double free or corruption
(out): 0x00007f4244042e80 ***
```
Then was like... lets try the latest edition:
Debian provided unstable 2.3.2-1:
```
knotd[11892] general protection ip:7fb8f7f0f218 sp:7fb8ce1cc3b0 error:0
in libc-2.24.so[7fb8f7e98000+195000]
````
yes, that is on a newer libc, hence different style error message it seems.
The 2.3.1 edition was able to report:
```
error: [example.com] changes from journal applied 1 -> 1 (invalid parameter)
````
before crashing out, the 2.3.2 just borks out.
Unfortunately there are no dbgsym packages for those editions, thus
can't easily dig what goes wrong where without having to resort to
manually building it all.
I could also not find a way to signup to:
https://gitlab.labs.nic.cz/users/sign_in
to be able to file a ticket about this.
Any extra details that one should be providing outside of the above
(link to that list is welcome ;) )
Should I attempt knot-nightly?
Greets,
Jeroen
PS: News on https://labs.nic.cz/en/ ends in April 2016...
9 Lis
9 Lis
16:33
Hi Jeroen,
just a quick reply - the debugging packages are now in separate
archive: https://wiki.debian.org/AutomaticDebugPackages
Could you try installing debugging symbols and run this under
gdb and/or valgrind?
Also would you be able to provide the /var/lib/knot and /etc/knot
for inspection?
After backing-up /var/lib/knot you could try bisecting /var/lib/knot
by deleting journal .db files one by one and/or timers database.
We would be interested in seeing a corrupted .db file if you can
detect which one is causing the troubles.
Cheers,
--
Ondřej Surý -- Technical Fellow
--------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Milesovska 5, 130 00 Praha 3, Czech Republic
mailto:ondrej.sury@nic.cz https://nic.cz/
--------------------------------------------
----- Original Message -----
From: "Jeroen Massar"
<jeroen(a)massar.ch>
To: "knot-dns-users" <knot-dns-users(a)lists.nic.cz>
Sent: Wednesday, 9 November, 2016 16:23:24
Subject: [knot-dns-users] Segfault in knot 2.3.1-1~bpo8+ and 2.3.2-1
Hola,
As recently a variety of people claimed to me that 'running DNSSEC is
not scary'.... I was like, lets try again after having tried it ~10
years ago and it failing miserably.
DNSSEC auto-maintain style looks to be better; still not as nice as
running dual-nsd's in master mode; but we'll live with those moving parts.
It ran fine for a bit, till I noticed the signatures of the zone had
expired and noticed the master simply did not bother to update the sigs
anymore. So much for 'automatic' mode.
Restarting it caused a nice crash:
Debian provided jessie-backports 2.3.1-1~bpo8:
```
knotd[6679]: *** Error in `/usr/sbin/knotd': double free or corruption
(out): 0x00007f4244042e80 ***
```
Then was like... lets try the latest edition:
Debian provided unstable 2.3.2-1:
```
knotd[11892] general protection ip:7fb8f7f0f218 sp:7fb8ce1cc3b0 error:0
in libc-2.24.so[7fb8f7e98000+195000]
````
yes, that is on a newer libc, hence different style error message it seems.
The 2.3.1 edition was able to report:
```
error: [example.com] changes from journal applied 1 -> 1 (invalid parameter)
````
before crashing out, the 2.3.2 just borks out.
Unfortunately there are no dbgsym packages for those editions, thus
can't easily dig what goes wrong where without having to resort to
manually building it all.
I could also not find a way to signup to:
https://gitlab.labs.nic.cz/users/sign_in
to be able to file a ticket about this.
Any extra details that one should be providing outside of the above
(link to that list is welcome ;) )
Should I attempt knot-nightly?
Greets,
Jeroen
PS: News on https://labs.nic.cz/en/ ends in April 2016...
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
16:45
On 2016-11-09 16:33, Ondřej Surý wrote:
Hi Jeroen,
just a quick reply - the debugging packages are now in separate
archive: https://wiki.debian.org/AutomaticDebugPackages
Ah, installing those.
Could you try installing debugging symbols and run
this under
gdb and/or valgrind?
Will go for valgrind with gdb trap ;)
Also would you be able to provide the /var/lib/knot
and /etc/knot
for inspection?
I am just using a test domain for this, thus yep no prob, can drop those
somewhere.
After backing-up /var/lib/knot you could try bisecting
/var/lib/knot
by deleting journal .db files one by one and/or timers database.
There is only two domains, and I am quite sure which one it is.
We would be interested in seeing a corrupted .db file
if you can
detect which one is causing the troubles.
Digging up next ;)
Greets,
Jeroen
17:11
On 2016-11-09 16:45, Jeroen Massar wrote:
Yep, removing it and stuff starts up again.
See the db file attached. Not so big (105k only) as it is a test domain
with almost nothing in there, thus should be perfect for testing /
debugging where this went wrong.
The knotd config for this is simple (relevant pieces below).
It has something special which might cause the issue: "zonefile-sync:
-1", this as I do not want it to overwrite the zone files that I put
there (they exist in git), get updated and a reload issued to let knot
know to update them.
A "zonefile-alt-path" config variable would be a good thing if one wants
to really store them btw.
Thus maybe me changing the file and then reloading it caused the db to
go inconsistent?
Noting also these two error messages:
error: [secluded.ch] changes from journal applied 1 -> 1 (invalid parameter)
error: [secluded.ch] zone event 'load' failed (invalid parameter)
If you need more data, or something to test, yell ;)
Greets,
Jeroen
--
policy:
- id: default
nsec3: on
template:
- id: default
dnssec-signing: on
dnssec-policy: default
# Do not store DNSSEC edition
zonefile-sync: -1
# Verify contents
semantic-checks: on
# DNS
acl: acl_cluster
# Zone files
file: "/etc/knot/zones/%s"
# Ignore SOA, just use filetime
serial-policy: "unixtime"
zone:
- domain: secluded.ch
On 2016-11-09 16:33, Ondřej Surý wrote:
Find attached, otherwise email will mess up the formatting anyway.
That is against 2.3.2-1 from Debian unstable.
Hi Jeroen,
just a quick reply - the debugging packages are now in separate
archive: https://wiki.debian.org/AutomaticDebugPackages
Ah, installing those.
Could you try installing debugging symbols and
run this under
gdb and/or valgrind?
Will go for valgrind with gdb trap ;) Also would you
be able to provide the /var/lib/knot and /etc/knot
for inspection?
I am just using a test domain for this, thus yep no prob, can drop those
somewhere.
After backing-up /var/lib/knot you could try
bisecting /var/lib/knot
by deleting journal .db files one by one and/or timers database.
There is only two domains, and I am quite sure which one it is. We would be
interested in seeing a corrupted .db file if you can
detect which one is causing the troubles.
Digging up next ;) 
18:00
Hi Jeroen, Ondřej, all,
On 16-11-09 17:11:14, Jeroen Massar wrote:
The knotd config for this is simple (relevant pieces
below).
It has something special which might cause the issue: "zonefile-sync:
-1", this as I do not want it to overwrite the zone files that I put
there (they exist in git), get updated and a reload issued to let knot
know to update them.
If this turns out to be the root cause, I would appreciate if this could
be fixed quickly, as my config looks the same in this regard.
If I'm able to help somehow, please let me know.
Thanks in advance and all the best,
Georg
1 Pro
1 Pro
2:58
Hi Jeroen, Ondřej, all,
On 16-11-09 16:23:24, Jeroen Massar wrote:
It ran fine for a bit, till I noticed the signatures
of the zone had
expired and noticed the master simply did not bother to update the sigs
anymore. So much for 'automatic' mode.
Restarting it caused a nice crash:
Debian provided jessie-backports 2.3.1-1~bpo8:
```
knotd[6679]: *** Error in `/usr/sbin/knotd': double free or corruption
(out): 0x00007f4244042e80 ***
```
Then was like... lets try the latest edition:
Debian provided unstable 2.3.2-1:
```
knotd[11892] general protection ip:7fb8f7f0f218 sp:7fb8ce1cc3b0 error:0
in libc-2.24.so[7fb8f7e98000+195000]
````
Any news on that?
TIA,
Georg
14:45
Hi Georg,
Yesterday, we have fixed the double free. But this issue still requires some testing,
because automatic signing with disabled zone file synchronization brings some
problems.
How do you update the zone file and what type of changes do you do?
Do you simply reload the server or restart?
Daniel
On 12/01/2016 02:58 AM, georg(a)riseup.net wrote:
Hi Jeroen, Ondřej, all,
On 16-11-09 16:23:24, Jeroen Massar wrote:
It ran fine for a bit, till I noticed the
signatures of the zone had
expired and noticed the master simply did not bother to update the sigs
anymore. So much for 'automatic' mode.
Restarting it caused a nice crash:
Debian provided jessie-backports 2.3.1-1~bpo8:
```
knotd[6679]: *** Error in `/usr/sbin/knotd': double free or corruption
(out): 0x00007f4244042e80 ***
```
Then was like... lets try the latest edition:
Debian provided unstable 2.3.2-1:
```
knotd[11892] general protection ip:7fb8f7f0f218 sp:7fb8ce1cc3b0 error:0
in libc-2.24.so[7fb8f7e98000+195000]
````
Any news on that?
TIA,
Georg
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
8 Pro
8 Pro
12:36
Hi Daniel.
Sorry, I've overlooked your mail.. :(
On 16-12-01 14:45:07, Daniel Salzman wrote:
Yesterday, we have fixed the double free. But this
issue still
requires some testing, because automatic signing with disabled zone
file synchronization brings some problems.
Sounds great, thanks for that!
How do you update the zone file and what type of
changes do you do?
Do you simply reload the server or restart?
If setting up a machine with knot I'm using puppet to do the install and
to provide "base files" containing essential information like SOA etc.
for the configured zones. After this, I'm using DDNS updates exclusively
to add and delete records, nothing fancy. Sounds quite safe, doesn't it?
Thanks for your work,
all the best,
Georg
12:57
Hi Georg,
On 12/08/2016 12:36 PM, georg(a)riseup.net wrote:
Hi Daniel.
Sorry, I've overlooked your mail.. :(
No problem :-)
On 16-12-01 14:45:07, Daniel Salzman wrote:
Yes,
your use case is quite safe. The only thing that you have to
ensure is the maximum journal size. Because the changes are cumulating.
In the upcoming 2.4 release, the new zone journal will be able
to merge the oldest changes to reduce the journal occupation.
Best,
Daniel
Yesterday, we have fixed the double free. But
this issue still
requires some testing, because automatic signing with disabled zone
file synchronization brings some problems.
Sounds great, thanks for that!
How do you update the zone file and what type of
changes do you do?
Do you simply reload the server or restart?
If setting up a machine with knot
I'm using puppet to do the install and
to provide "base files" containing essential information like SOA etc.
for the configured zones. After this, I'm using DDNS updates exclusively
to add and delete records, nothing fancy. Sounds quite safe, doesn't it? Thanks for your work,
all the best,
Georg
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
12:32
Hi Jeroen,
The reported crash should be fixed in Knot 2.3.3.
As you have zone file synchronization disabled, you must keep the SOA record
in the zone file unchanged. Otherwise the zone journal can not be applied.
The SOA serial will be updated automatically during the following zone re-signing.
Regards,
Daniel
On 11/09/2016 04:23 PM, Jeroen Massar wrote:
Hola,
As recently a variety of people claimed to me that 'running DNSSEC is
not scary'.... I was like, lets try again after having tried it ~10
years ago and it failing miserably.
DNSSEC auto-maintain style looks to be better; still not as nice as
running dual-nsd's in master mode; but we'll live with those moving parts.
It ran fine for a bit, till I noticed the signatures of the zone had
expired and noticed the master simply did not bother to update the sigs
anymore. So much for 'automatic' mode.
Restarting it caused a nice crash:
Debian provided jessie-backports 2.3.1-1~bpo8:
```
knotd[6679]: *** Error in `/usr/sbin/knotd': double free or corruption
(out): 0x00007f4244042e80 ***
```
Then was like... lets try the latest edition:
Debian provided unstable 2.3.2-1:
```
knotd[11892] general protection ip:7fb8f7f0f218 sp:7fb8ce1cc3b0 error:0
in libc-2.24.so[7fb8f7e98000+195000]
````
yes, that is on a newer libc, hence different style error message it seems.
The 2.3.1 edition was able to report:
```
error: [example.com] changes from journal applied 1 -> 1 (invalid parameter)
````
before crashing out, the 2.3.2 just borks out.
Unfortunately there are no dbgsym packages for those editions, thus
can't easily dig what goes wrong where without having to resort to
manually building it all.
I could also not find a way to signup to:
https://gitlab.labs.nic.cz/users/sign_in
to be able to file a ticket about this.
Any extra details that one should be providing outside of the above
(link to that list is welcome ;) )
Should I attempt knot-nightly?
Greets,
Jeroen
PS: News on https://labs.nic.cz/en/ ends in April 2016...
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
2905
days inactive
2934
days old
9 comments
4 participants
participants (4)
-
Daniel Salzman -
georg＠riseup.net -
Jeroen Massar -
Ondřej Surý