[dscng-dev] Error reading corrupt dat files

Bedrich Kosata bedrich.kosata at nic.cz
Fri Jun 15 10:45:46 CEST 2012


Hi John,

the fix should be already in git, could you please test it on your data?

Best regards

Beda

On 06/14/2012 07:29 PM, Bedrich Kosata wrote:
> Hi John,
>
> no problem, the information about the bug is valuable on its own. I will
> have a look at the code tomorrow and push a fix into the git repo.
>
> Best regards
>
> Beda
>
> On 06/14/2012 05:57 PM, John Dickinson wrote:
>> Hi,
>>
>> hmm that seems to put the count out of scope. Adding a line count=0
>> before the if statement seems to fix it. As you may be gathering I am
>> not a python developer and I am sure you can correct my mistakes :)
>>
>> Thanks
>> John
>>
>> On Jun 14, 2012, at 4:29 PM, John Dickinson wrote:
>>
>>> Hi again
>>>
>>> I now have an error on the next line of the parse_data_file_2d
>>> function so I think the patch should be along the lines of
>>>
>>> diff --git a/import_dsc_dat_files.py b/import_dsc_dat_files.py
>>> index c54bc4b..439952a 100644
>>> --- a/import_dsc_dat_files.py
>>> +++ b/import_dsc_dat_files.py
>>> @@ -752,8 +752,13 @@ value for 'else'. This is probably an error.")
>>> if remap1 is None and data[i].isdigit():
>>> value1 = local_int(data[i])
>>> else:
>>> - value1 = value_to_int(data[i], dim1)
>>> - count = local_int(data[i+1])
>>> + try:
>>> + value1 = value_to_int(data[i], dim1)
>>> + count = local_int(data[i+1])
>>> + except:
>>> + print "Error reading line from 2D data file"
>>> + i += 2
>>> + continue
>>> row.append(((value1,), count))
>>> i += 2
>>> to_insert.append((rec_dt, row))
>>>
>>> The error on this occasion occurred parsing a client_port_range.dat
>>> and was: Import was unsuccessful - error: list index out of range
>>>
>>> There are far too many numbers in that file to find the fault :)
>>>
>>> Thanks
>>> John
>>>
>>> On Jun 14, 2012, at 2:26 PM, Bedrich Kosata wrote:
>>>
>>>> Hi John,
>>>>
>>>> thanks for both the bug report and the fix. I will have a look at it
>>>> and patch potential other places where it would be needed.
>>>>
>>>> Best regards
>>>>
>>>> Beda
>>>>
>>>> On 06/14/2012 03:14 PM, John Dickinson wrote:
>>>>> Hi,
>>>>>
>>>>> I hit an error importing a corrupt data file. 1 line in the file
>>>>> was truncated and contained no value.
>>>>>
>>>>> Here is a quick fix, but I suspect that it needs to be repeated in
>>>>> all the parse_data_file_Xd functions.
>>>>>
>>>>> diff --git a/import_dsc_dat_files.py b/import_dsc_dat_files.py
>>>>> index c54bc4b..d6a8357 100644
>>>>> --- a/import_dsc_dat_files.py
>>>>> +++ b/import_dsc_dat_files.py
>>>>> @@ -752,7 +752,12 @@ value for 'else'. This is probably an error.")
>>>>> if remap1 is None and data[i].isdigit():
>>>>> value1 = local_int(data[i])
>>>>> else:
>>>>> - value1 = value_to_int(data[i], dim1)
>>>>> + try:
>>>>> + value1 = value_to_int(data[i], dim1)
>>>>> + except:
>>>>> + print "Error reading line from 2D data file"
>>>>> + i += 2
>>>>> + continue
>>>>> count = local_int(data[i+1])
>>>>> row.append(((value1,), count))
>>>>> i += 2
>>>>>
>>>>> regards
>>>>> John
>>>>>
>>>>>
>>>>> ---
>>>>> jad at sinodun.com
>>>>>
>>>>> http://sinodun.com
>>>>>
>>>>> Sinodun Internet Technologies Ltd.
>>>>> Stables 4, Suite 11,
>>>>> Howbery Park,
>>>>> Wallingford,
>>>>> Oxfordshire,
>>>>> OX10 8BA,
>>>>> U.K.
>>>>>
>>>>> +44 (0)1491 834957
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> dscng-dev mailing list
>>>>> dscng-dev at lists.nic.cz
>>>>> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev
>>>>
>>>> _______________________________________________
>>>> dscng-dev mailing list
>>>> dscng-dev at lists.nic.cz
>>>> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev
>>>
>>> _______________________________________________
>>> dscng-dev mailing list
>>> dscng-dev at lists.nic.cz
>>> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev
>>
>
> _______________________________________________
> dscng-dev mailing list
> dscng-dev at lists.nic.cz
> https://lists.nic.cz/cgi-bin/mailman/listinfo/dscng-dev

-- 
Bedrich Kosata
CZ.NIC Labs <http://labs.nic.cz>


More information about the dscng-dev mailing list