[Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
Hello,

something broke in sdfdataset with fpc 3.2.0 (a segfault that doesn't
occur if I compile the same program with fpc 3.0.4) and I'm trying to
debug it.

First I tried to compile just the fcl with debug information

cd fpc-3.2.0/packages
OPT=-gl make
sudo make install

but lazarus then complained that it couldn't find the DB unit (?), so I
recompiled everything:

cd fpc-3.2.0
make distclean
OPT=-gl make all
sudo make install

but even so I'm missing some debug info. This is what I see in the call
stack windows when I put a breakpoint on TFixedFormatDataSet.GetRecord
(file fpc-3.2.0/packages/fcl-db/src/sdf/sdfdata.pp)


#0 GETRECORD(0x7fffe3949370, 0x0, GMNEXT, false) at
fcl-db/src/sdf/sdfdata.pp:546
#1 ?? at :0
#2 ?? at :0
#3 ?? at :0
#4 ?? at :0
#5 ?? at :0
#6 ?? at :0

that's with "copy all", the content of the call stack window is
different, i.e. (only the index, location and line columns for brevity)

0 sdfdata.pp (fcl-db/src/sdf/)  546
1 :7FFFFFFFDF58
2 :7FFFFFFFE0A0
3 :00000001
4 :7FFFE3949370
5 :7FFFFFFFE129
6 :00000000


Any hint?

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 15/7/20 a les 17:20, Luca Olivetti via lazarus ha escrit:

> Hello,
>
> something broke in sdfdataset with fpc 3.2.0 (a segfault that doesn't
> occur if I compile the same program with fpc 3.0.4) and I'm trying to
> debug it.
>
> First I tried to compile just the fcl with debug information
>
> cd fpc-3.2.0/packages
> OPT=-gl make
> sudo make install
>
> but lazarus then complained that it couldn't find the DB unit (?), so I
> recompiled everything:
>
> cd fpc-3.2.0
> make distclean
> OPT=-gl make all
> sudo make install
>
> but even so I'm missing some debug info. This is what I see in the call
> stack windows when I put a breakpoint on TFixedFormatDataSet.GetRecord
> (file fpc-3.2.0/packages/fcl-db/src/sdf/sdfdata.pp)
>
>
> #0 GETRECORD(0x7fffe3949370, 0x0, GMNEXT, false) at
> fcl-db/src/sdf/sdfdata.pp:546
> #1 ?? at :0
> #2 ?? at :0
> #3 ?? at :0
> #4 ?? at :0
> #5 ?? at :0
> #6 ?? at :0
>
> that's with "copy all", the content of the call stack window is
> different, i.e. (only the index, location and line columns for brevity)
>
> 0 sdfdata.pp (fcl-db/src/sdf/)  546
> 1 :7FFFFFFFDF58
> 2 :7FFFFFFFE0A0
> 3 :00000001
> 4 :7FFFE3949370
> 5 :7FFFFFFFE129
> 6 :00000000
>
>
> Any hint?

For the record, the segfault happens here


#0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963
#1 ?? at :0
#2 fpc_ansistr_decr_ref(<error reading variable: Cannot access memory at
address 0x30>) at ../inc/astrings.inc:148
#3 ?? at :0


i.e. the call stack is even more wacky, but I know it comes from
TFixedFormatDataSet.GetRecord

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 15/7/20 a les 17:31, Luca Olivetti via lazarus ha escrit:

> El 15/7/20 a les 17:20, Luca Olivetti via lazarus ha escrit:
>> Hello,
>>
>> something broke in sdfdataset with fpc 3.2.0 (a segfault that doesn't
>> occur if I compile the same program with fpc 3.0.4) and I'm trying to
>> debug it.
>>
>> First I tried to compile just the fcl with debug information
>>
>> cd fpc-3.2.0/packages
>> OPT=-gl make
>> sudo make install
[...]

>> Any hint?

The optimization! I had to use "-g -gl -O-" to get the full call stack.

>
> For the record, the segfault happens here
>
>
> #0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963

The memory is hosed, I still don't know why.
I managed to reproduce the problem with a simple lazarus program,
however a plain pascal program  that does the same doesn't trigger the it.

I'm attaching the lazarus project as well as the simple pascal program
if somebody could look into it...


Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus

sdfsegfault.lpr (1K) Download Attachment
project1.zip (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 15/7/20 a les 18:26, Luca Olivetti via lazarus ha escrit:

>> For the record, the segfault happens here
>>
>>
>> #0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963
>
> The memory is hosed, I still don't know why.
> I managed to reproduce the problem with a simple lazarus program,
> however a plain pascal program  that does the same doesn't trigger the it.
>
> I'm attaching the lazarus project as well as the simple pascal program
> if somebody could look into it...


I reported it here

https://bugs.freepascal.org/view.php?id=37370

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 16/7/20 a les 8:55, Luca Olivetti via lazarus ha escrit:

> El 15/7/20 a les 18:26, Luca Olivetti via lazarus ha escrit:
>
>>> For the record, the segfault happens here
>>>
>>>
>>> #0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963
>>
>> The memory is hosed, I still don't know why.
>> I managed to reproduce the problem with a simple lazarus program,
>> however a plain pascal program  that does the same doesn't trigger the
>> it.
>>
>> I'm attaching the lazarus project as well as the simple pascal program
>> if somebody could look into it...
>
>
> I reported it here
>
> https://bugs.freepascal.org/view.php?id=37370

I found that reverting

https://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&revision=43245

(in fact I just took sdfdata.pp from 3.0.4)

fixes the segfault.

Michael, what do you think?

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list


On Thu, 16 Jul 2020, Luca Olivetti wrote:

> El 16/7/20 a les 8:55, Luca Olivetti via lazarus ha escrit:
>> El 15/7/20 a les 18:26, Luca Olivetti via lazarus ha escrit:
>>
>>>> For the record, the segfault happens here
>>>>
>>>>
>>>> #0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963
>>>
>>> The memory is hosed, I still don't know why.
>>> I managed to reproduce the problem with a simple lazarus program, however
>>> a plain pascal program  that does the same doesn't trigger the it.
>>>
>>> I'm attaching the lazarus project as well as the simple pascal program if
>>> somebody could look into it...
>>
>>
>> I reported it here
>>
>> https://bugs.freepascal.org/view.php?id=37370
>
> I found that reverting
>
> https://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&revision=43245
>
> (in fact I just took sdfdata.pp from 3.0.4)
>
> fixes the segfault.
>
> Michael, what do you think?
Seems like a bug somewhere. I'll need to look at it. Probably has something
to do with the default codepage: Lazarus changes that.

Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 9:51, Michael Van Canneyt ha escrit:

>>> I reported it here
>>>
>>> https://bugs.freepascal.org/view.php?id=37370
>>
>> I found that reverting
>>
>> https://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&revision=43245 
>>
>>
>> (in fact I just took sdfdata.pp from 3.0.4)
>>
>> fixes the segfault.
>>
>> Michael, what do you think?
>
> Seems like a bug somewhere. I'll need to look at it. Probably has something
> to do with the default codepage: Lazarus changes that.

What worries me is the fact that it is so easy to mess up the heap. What
other dragons are lurking ahead in 3.2.0?

For the record, I tried to set the codepage of the component to 'utf-8'
and it still segfaults, so I don't think it's because of the default
codepage.

Also the fact that some of the TFieldDefs.Add are procedure and some
functions doesn't seem right.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list


On Fri, 17 Jul 2020, Luca Olivetti wrote:

> El 17/7/20 a les 9:51, Michael Van Canneyt ha escrit:
>
>>>> I reported it here
>>>>
>>>> https://bugs.freepascal.org/view.php?id=37370
>>>
>>> I found that reverting
>>>
>>> https://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&revision=43245 
>>>
>>> (in fact I just took sdfdata.pp from 3.0.4)
>>>
>>> fixes the segfault.
>>>
>>> Michael, what do you think?
>>
>> Seems like a bug somewhere. I'll need to look at it. Probably has something
>> to do with the default codepage: Lazarus changes that.
>
> What worries me is the fact that it is so easy to mess up the heap. What
> other dragons are lurking ahead in 3.2.0?

Looking at the bugtracker, there could be several.
I think we may need to release a 3.2.2 after a relatively short amount of time.

> For the record, I tried to set the codepage of the component to 'utf-8' and
> it still segfaults, so I don't think it's because of the default codepage.

We'll see.
This is the only relevant change in the component, so that is the most likely candidate.
What is more puzzling is that it only happens in Lazarus.

> Also the fact that some of the TFieldDefs.Add are procedure and some
> functions doesn't seem right.

They are simply overloaded versions, but all boil down to the same.
Some of these exist for Delphi compatibility. I doubt this is the cause.

Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 10:26, Michael Van Canneyt via lazarus ha escrit:

>> For the record, I tried to set the codepage of the component to
>> 'utf-8' and it still segfaults, so I don't think it's because of the
>> default codepage.
>
> We'll see. This is the only relevant change in the component, so that is
> the most likely candidate.

In fact, if you look at the bug report, it's enough to use the old
version of Fielddefs.Add (i.e. the one without the encoding) to "fix"
the problem, but it's not because of the default (I changed it to
'utf-8', or maybe it's actually 'utf-8' that causes the problem).

> What is more puzzling is that it only happens in Lazarus.

For the record, in the simple program I tried with 100000 iterations and
it still doesn't segfaults.

>
>> Also the fact that some of the TFieldDefs.Add are procedure and some
>> functions doesn't seem right.
>
> They are simply overloaded versions, but all boil down to the same. Some
> of these exist for Delphi compatibility. I doubt this is the cause.

I'm not saying it's the cause, it just seemed odd.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 11:46, Luca Olivetti via lazarus ha escrit:

>
> In fact, if you look at the bug report, it's enough to use the old
> version of Fielddefs.Add (i.e. the one without the encoding) to "fix"
> the problem, but it's not because of the default (I changed it to
> 'utf-8', or maybe it's actually 'utf-8' that causes the problem).

Yes, setting it to '437' or 'iso8859-1' avoids the segfault.

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list


On Fri, 17 Jul 2020, Luca Olivetti via lazarus wrote:

> El 17/7/20 a les 11:46, Luca Olivetti via lazarus ha escrit:
>
>>
>> In fact, if you look at the bug report, it's enough to use the old
>> version of Fielddefs.Add (i.e. the one without the encoding) to "fix"
>> the problem, but it's not because of the default (I changed it to
>> 'utf-8', or maybe it's actually 'utf-8' that causes the problem).
>
> Yes, setting it to '437' or 'iso8859-1' avoids the segfault.

Haha... The mystery is unraveling :-)

Note that these are single-byte codepages.
Are you using fixed-length of CSV ? Because in the case of fixed-length,
UTF8 makes no sense...

Michael.
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 12:12, Michael Van Canneyt via lazarus ha escrit:

>
>
> On Fri, 17 Jul 2020, Luca Olivetti via lazarus wrote:
>
>> El 17/7/20 a les 11:46, Luca Olivetti via lazarus ha escrit:
>>
>>>
>>> In fact, if you look at the bug report, it's enough to use the old
>>> version of Fielddefs.Add (i.e. the one without the encoding) to "fix"
>>> the problem, but it's not because of the default (I changed it to
>>> 'utf-8', or maybe it's actually 'utf-8' that causes the problem).
>>
>> Yes, setting it to '437' or 'iso8859-1' avoids the segfault.
>
> Haha... The mystery is unraveling :-)
>
> Note that these are single-byte codepages.
> Are you using fixed-length of CSV ? Because in the case of fixed-length,
> UTF8 makes no sense...

I'm not using the CSV at all, I load an empty file and fill the dataset
by other means (Dataset.Fields[f].AsString:=....).
However I put a breakpoint on every SetAsString and none is
triggered....(the assumption here is that a buffer of length n*4 is
actually of length n, that would explain the heap corruption and I
wanted to see where it happens).
I any case it doesn't matter if it makes sense or not, it should work,
maybe with a wrong encoding but definitely it shouldn't segfault.


Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 12:32, Luca Olivetti via lazarus ha escrit:

> However I put a breakpoint on every SetAsString and none is
> triggered....

And that's an issue with the debugger I think :-(

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 12:45, Luca Olivetti via lazarus ha escrit:
> El 17/7/20 a les 12:32, Luca Olivetti via lazarus ha escrit:
>
>> However I put a breakpoint on every SetAsString and none is triggered....
>
> And that's an issue with the debugger I think :-(
>
Sorry, it was a PEBKAC, I was debugging under windows where the rtl/fcl
has no debug information.
Back to linux...

Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Debugging the rtl/fcl - segfault in sdfdataset

Free Pascal - Lazarus mailing list
El 17/7/20 a les 12:49, Luca Olivetti via lazarus ha escrit:

> El 17/7/20 a les 12:45, Luca Olivetti via lazarus ha escrit:
>> El 17/7/20 a les 12:32, Luca Olivetti via lazarus ha escrit:
>>
>>> However I put a breakpoint on every SetAsString and none is
>>> triggered....
>>
>> And that's an issue with the debugger I think :-(
>>
> Sorry, it was a PEBKAC, I was debugging under windows where the rtl/fcl
> has no debug information.
> Back to linux...

I found the problem and added a patch to the bugreport.
This is no C folks, "Move" should be banned from the compiler, though I
use it myself ;-)

https://bugs.freepascal.org/view.php?id=37370#c124122


Bye
--
Luca Olivetti
Wetron Automation Technology http://www.wetron.es/
Tel. +34 93 5883004 (Ext.3010)  Fax +34 93 5883007
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus