[dscng-dev] import_dsc_dat_files.py assertion error

Bedrich Kosata bedrich.kosata at nic.cz
Thu Jun 14 14:25:07 CEST 2012


Hi Thomas,

thanks for reporting these problems.
The first problem would probably be a rounding error where something 
like 1.00000000001 is considered higher than 1 and thus the assertion 
fails. I will fix it asap.
As to the second problem, this is probably a result of the previous 
crash - there is some statistics that is written to the db at the end of 
an import and because of the crash it is out of sync (this is one of the 
things that need to be improved before a preview release).
Please run the debug.py script in the root directory and let me know if 
it helps.

Best regards

Beda



On 06/14/2012 01:53 PM, Thomas Dupas wrote:
> FYI, in case somebody else ever has the same issue.
> There seems to be a conflict if dsc is still running/writing at the same time.
> Run the script again 2-3 times untill it completes correctly.
>
> The dashboard still doesn't work here though
> "Overall traffic: Error occured when fetching data: Internal Server Error
> Rcode chart: Error occured when fetching data: Internal Server Error"
>
> Any clues to what tables I should check?
> apache error log contains:
>
> "DEBUG:dsc_storage:Getting available timespan: 0.001486
> DEBUG:dsc_storage:Getting available timespan: 0.001532
> DEBUG:dsc_storage:SELECT: SELECT server_id,SUM(count)
>              FROM dscng_data2d
>              WHERE time = '2012-06-14T13:00:00+02:00'::timestamptz AT TIME ZONE 'UTC' AND minute = 59 AND count != -1
>              AND data_type_id = 16
>              GROUP BY server_id;
> DEBUG:dsc_storage:Getting available timespan: 0.000695
> DEBUG:dsc_storage:Query preparation: 0.0105571746826
> DEBUG:dsc_storage:SELECT time AT TIME ZONE 'UTC', minute,value1,SUM(count) FROM dscng_data2d WHERE time AT TIME ZONE 'UTC'>= '2012-06-14 12:59:00+02:00' AND time AT TIME ZONE 'UTC'<= '2012-06-14 13:59:00+02:00' AND data_type_id = 25 AND count != -1  GROUP BY time,minute,value1 ORDER BY time,minute;
> DEBUG:dsc_storage:Query execution: 0.00430297851562
> DEBUG:dsc_storage:Result processing: 0.00108504295349
> DEBUG:dsc_storage:Time points: 0
> ERROR:django.request:Internal Server Error: /dscng/json/data_type_detail_dt/
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
>      response = callback(request, *callback_args, **callback_kwargs)
>    File "/var/www/dscng/dsc/plots/views.py", line 380, in get_data_type_detail_dt_json
>      data_dict.append({'count': data[0].get(subtype, 0),
> IndexError: list index out of range
> ERROR:django.request:Internal Server Error: /dscng/json/overall_traffic/
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
>      response = callback(request, *callback_args, **callback_kwargs)
>    File "/var/www/dscng/dsc/plots/views.py", line 319, in get_overall_traffic_json
>      qps = storage.get_overall_traffic(timepoint) / 60.0
> TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'
> "
>
> Br,
>
> Thomas
>
> ________________________________________
> Van: dscng-dev-bounces at lists.nic.cz [dscng-dev-bounces at lists.nic.cz] namens Thomas Dupas [thomas at dupas.be]
> Verzonden: donderdag 14 juni 2012 12:21
> Aan: dscng-dev at lists.nic.cz
> Onderwerp: [dscng-dev] import_dsc_dat_files.py assertion error
>
> Hi,
>
> I'm using the latest git version, where the import_dsc_dat_files.py errors out on the very last moment.
> At first sight it can't handle the 99.9% done / 0.1% remaining to 100% done / 0% remaining transition
>
> "# DSCng - importing original DSC .dat files
>
> * Importing into database 'dsc'
> * Logging into 'dscng_import-2012-06-14_112109.log'
>
> #################### 0:23:19, 3.29 s/dir, 1052 kB/s, 99.9%, ~0:00:01 to goTraceback (most recent call last):
>    File "import_dsc_dat_files.py", line 1547, in<module>
>      main()
>    File "import_dsc_dat_files.py", line 1544, in main
>      show_progress=True, stat_out_stream=stat_out_stream)
>    File "import_dsc_dat_files.py", line 1447, in import_dirs
>      log_to_progressbar(format_percent_bar(done_part) + " " + \
>    File "import_dsc_dat_files.py", line 1479, in format_percent_bar
>      assert 0<= percent<= 1
> AssertionError"
>
> When querying some tables in the database I can already see some data, but I can't output anything useful in the webinterface.
> I only see the list of servers in the left column
>
> Br,
>
> Thomas Dupas
> _______________________________________________
> dscng-dev mailing list
> dscng-dev at lists.nic.cz
> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev
> _______________________________________________
> dscng-dev mailing list
> dscng-dev at lists.nic.cz
> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev



More information about the dscng-dev mailing list