[dscng-dev] XML update bug report

Kostas Zorbadelos kzorba at otenet.gr
Wed Aug 1 10:58:02 CEST 2012


Using update_from_xml.py to import data from dsc xml files directly to
the database I bumped into the following bug:

-bash-4.1$ python /www/dnsmon.eng.otenet.gr/docroot/dscng/update_from_xml.py -d /usr/local/dsc/data/infra-resolver/athe-dns2/incoming/2012-08-01/1343809920.dscdata.xml 
INFO:root:Using (slower) direct database access

[(u'_709_51_1', 4002), ('gr/public_html/caps', 3987), (u'36qo', 3993), (u'q81t6w7g', 4000), (u'27dm', 3991), (u'_700_53_8', 3994), (u'46wc', 3995), (u'o20z4t3k', 4001), (u'd12j4n3l', 4005), (u'j89d7l7j', 3990), (u'_470_00_2', 3986), (u'84ch', 3989), (u'59xr', 4004), ('tw ', 3985), (u'_137_67_0', 4003), (u'95mg', 3992), (u'_103_33_8', 3997), (u'_259_83_6', 3996), (u'102-0249cddc', 3988), (u'k60a7p4y', 3998), ('\xc8ur', 4006), (u'37ex', 3999)]
^^^^^^^^^^^^
Above output is the value of to_insert list in write_dynamic_remaps_to_db() (File dsc_storage/psql/data_types.py)

/www/dnsmon.eng.otenet.gr/docroot/dscng/dsc_storage/psql/data_types.py:249: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  for rec in sorted(to_insert):
Traceback (most recent call last):
  File "/www/dnsmon.eng.otenet.gr/docroot/dscng/update_from_xml.py", line 326, in <module>
    main()
  File "/www/dnsmon.eng.otenet.gr/docroot/dscng/update_from_xml.py", line 311, in main
    importer.import_xml_file(fname)
  File "/www/dnsmon.eng.otenet.gr/docroot/dscng/update_from_xml.py", line 162, in import_xml_file
    self.update_callback(self._current_bundle)
  File "/www/dnsmon.eng.otenet.gr/docroot/dscng/dsc_storage/psql/storage.py", line 922, in insert_update_bundle
    self.dtm.write_dynamic_remaps_to_db()
  File "/www/dnsmon.eng.otenet.gr/docroot/dscng/dsc_storage/psql/data_types.py", line 249, in write_dynamic_remaps_to_db
    for rec in sorted(to_insert):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc8 in position 0: ordinal not in range(128)

Any thoughts?
Included attached are xml data files from dsc in my test
infrastructure. This bug prevents the db update of all xml files in my 
environment comming from a name server accepting traffic. In case the
nameserver is idle the xml files are processed OK.

Will fall back to using import_dsc_dat_files.py for DSCng DB update and
see what happens.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1343809920_problem.dscdata.xml
Type: application/xml
Size: 84881 bytes
Desc: not available
URL: <http://lists.nic.cz/pipermail/dscng-dev/attachments/20120801/ee8c0581/attachment-0002.xml>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1343809620_noproblem.dscdata.xml
Type: application/xml
Size: 4883 bytes
Desc: not available
URL: <http://lists.nic.cz/pipermail/dscng-dev/attachments/20120801/ee8c0581/attachment-0003.xml>


More information about the dscng-dev mailing list