[Lazarus] Can some fix this "make" issue? Pi related.

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[Lazarus] Can some fix this "make" issue? Pi related.

Anthony Walter
I believe this problem effects a lot of people using the Raspberry Pi, and possibly other platforms as well.

The problem:

Recently I been using "make all" on Lazarus sources, or trying to rebuild the IDE, such as when installing or removing packages, and experiencing build fails.

This will happen on the Raspberry Pi when 128MB or more memory is assigned to the GPU, a setting which must be configured to allow for video playback and/or GLES to run on the Pi. For those of you not experiencing this issue you may have left the GPU size at its default 64MB setting.

When a system such as the Pi has less than 700MB (approximately) of free memory the "make all" or rebuilding the IDE task will fail with errors such as" switching to an external resource compiler" or some other mysterious message.

Temporarily lowering the GPU memory, rebooting the computer, rebuilding Lazarus, changing the GPU memory back, then rebooting the computer again fixes the problem, but this is a very awkward solution. This is especially awkward considering you may need to do this anytime you want to add or remove component in Lazarus under the conditions I've described.

My question is can someone look into this issue so that building Lazarus in environments where free memory is than 700MB does not result in Lazarus build failures?
 

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mattias Gaertner
On Fri, 1 Apr 2016 00:39:15 -0400
Anthony Walter <[hidden email]> wrote:

>[...]
> My question is can someone look into this issue so that building Lazarus in
> environments where free memory is than 700MB does not result in Lazarus
> build failures?

Does it compile when you directly call the compiler?

Mattias

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Anthony Walter
Calling the compiler directly works on most any project/program. I am not sure how to build Lazarus (with my preferred packages) using lazbuild or fpc. The "make useride" command fails exactly like I described in my original post. As a side note, I am able to have GPU memory set to 128MB or more and have "make all" succeed if I have no window manage running and instead ssh into the pi and execute the "make all" command from a remote tty session. Again this is because running without a window manager frees up enough memory for "make all" to succeed.

Here is what my Pi looks like with a ssh session and 64MB reserved for the GPU. Note the free memory is 854MB:

pi@retropie:~ $ free -m
             total       used       free     shared    buffers     cached
Mem:           925        571        354          6         20        479
-/+ buffers/cache:         70        854
Swap:           99          0         99

Here is what it looks like with and ssh session, 128MB reserved for the GPU, mate desktop running, and Lazarus opened:

pi@retropie:~ $ free -m
             total       used       free     shared    buffers     cached
Mem:           862        413        449         16         32        199
-/+ buffers/cache:        181        681
Swap:           99          0         99

Rebuilding the IDE will actually fail at this point. Note the free memory is actually 681MB.

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Henry Vermaak
In reply to this post by Anthony Walter
On Fri, Apr 01, 2016 at 12:39:15AM -0400, Anthony Walter wrote:
> My question is can someone look into this issue so that building Lazarus in
> environments where free memory is than 700MB does not result in Lazarus
> build failures?

I've come across ld running out of memory when linking the lazarus
executable on my (very) old ARM laptop.  You can try to pass
--no-keep-memory and --reduce-memory-overheads to ld, at the cost of
performance.  I can't see how the build system has anything to do with
it.

Henry

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mattias Gaertner
In reply to this post by Anthony Walter
On Fri, 1 Apr 2016 03:25:47 -0400
Anthony Walter <[hidden email]> wrote:

> Calling the compiler directly works on most any project/program. I am not
> sure how to build Lazarus (with my preferred packages) using lazbuild or
> fpc. The "make useride" command fails exactly like I described in my
> original post.

make useride calls "lazbuild --build-ide=", which calls the compiler.
You can call "./lazbuild --build-ide= --max-process-count=1" directly
to avoid calling make and use at most one process.

Make sure to strip lazbuild to avoid loading debugging and compile it
with optimizations to use less memory.

Have you tried building lazbuild with cmem?

If this does not suffice, call the compiler directly using the command
shown by lazbuild.

Is it possible to add virtual memory to the Pi?

Mattias

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mark Morgan Lloyd
In reply to this post by Henry Vermaak
Henry Vermaak wrote:

> On Fri, Apr 01, 2016 at 12:39:15AM -0400, Anthony Walter wrote:
>> My question is can someone look into this issue so that building Lazarus in
>> environments where free memory is than 700MB does not result in Lazarus
>> build failures?
>
> I've come across ld running out of memory when linking the lazarus
> executable on my (very) old ARM laptop.  You can try to pass
> --no-keep-memory and --reduce-memory-overheads to ld, at the cost of
> performance.  I can't see how the build system has anything to do with
> it.

It's always worked fine for me on things like NSLU-2 provided that the
overall memory (i.e. RAM + swap) is at least 512Mb. It will be
unfortunate if recent builds of FPC and/or Lazarus have pushed this
requirement upwards, but so far I've not seen any problems on a
Raspberry Pi (both 1 and 2) running Raspbian or Debian.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Henry Vermaak
On Fri, Apr 01, 2016 at 09:12:10AM +0000, Mark Morgan Lloyd wrote:

> Henry Vermaak wrote:
> >On Fri, Apr 01, 2016 at 12:39:15AM -0400, Anthony Walter wrote:
> >>My question is can someone look into this issue so that building Lazarus in
> >>environments where free memory is than 700MB does not result in Lazarus
> >>build failures?
> >
> >I've come across ld running out of memory when linking the lazarus
> >executable on my (very) old ARM laptop.  You can try to pass
> >--no-keep-memory and --reduce-memory-overheads to ld, at the cost of
> >performance.  I can't see how the build system has anything to do with
> >it.
>
> It's always worked fine for me on things like NSLU-2 provided that the
> overall memory (i.e. RAM + swap) is at least 512Mb. It will be unfortunate
> if recent builds of FPC and/or Lazarus have pushed this requirement upwards,
> but so far I've not seen any problems on a Raspberry Pi (both 1 and 2)
> running Raspbian or Debian.

Yes, I remember that the slug needed swap.  Swap is painfully slow,
though, so I'd recommend reducing ld's memory usage instead.  I wonder
if anyone has tried the gold linker.  It was supposed to be faster, I
don't know about memory usage.

Henry

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

wkitty42
In reply to this post by Anthony Walter
On 04/01/2016 03:25 AM, Anthony Walter wrote:

> Here is what my Pi looks like with a ssh session and 64MB reserved for the GPU.
> Note the free memory is 854MB:
>
> pi@retropie:~ $ free -m
>               total       used       free     shared    buffers     cached
> Mem:           925        571        354          6         20        479
> -/+ buffers/cache:         70        854
> Swap:           99          0         99
>
> Here is what it looks like with and ssh session, 128MB reserved for the GPU,
> mate desktop running, and Lazarus opened:
>
> pi@retropie:~ $ free -m
>               total       used       free     shared    buffers     cached
> Mem:           862        413        449         16         32        199
> -/+ buffers/cache:        181        681
> Swap:           99          0         99
>
> Rebuilding the IDE will actually fail at this point. Note the free memory is
> actually 681MB.

ummm... shouldn't the compiling process use as much swap as needed if real
memory it too tight (aka not enough)??

the swap size is what attracted my attention because normally swap is the same
or double what installed real memory is... eg: 16G RAM means that 16G to 32G of
swap is configured...

--
  NOTE: No off-list assistance is given without prior approval.
        *Please keep mailing list traffic on the list* unless
        private contact is specifically requested and granted.

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mark Morgan Lloyd
In reply to this post by Henry Vermaak
Henry Vermaak wrote:

>> It's always worked fine for me on things like NSLU-2 provided that the
>> overall memory (i.e. RAM + swap) is at least 512Mb. It will be unfortunate
>> if recent builds of FPC and/or Lazarus have pushed this requirement upwards,
>> but so far I've not seen any problems on a Raspberry Pi (both 1 and 2)
>> running Raspbian or Debian.
>
> Yes, I remember that the slug needed swap.  Swap is painfully slow,
> though, so I'd recommend reducing ld's memory usage instead.  I wonder
> if anyone has tried the gold linker.  It was supposed to be faster, I
> don't know about memory usage.

Lazarus took a week to build, but would do so reliably. That sort of
performance takes me back to my mainframe days :-)

It does occur to me that this could be something to do with the way that
the linker has been built, i.e. not strictly a Lazarus (or makefile
etc.) problem.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mark Morgan Lloyd
In reply to this post by wkitty42
[hidden email] wrote:

> ummm... shouldn't the compiling process use as much swap as needed if
> real memory it too tight (aka not enough)??

Yes, and that's outside the remit of Lazarus etc.

> the swap size is what attracted my attention because normally swap is
> the same or double what installed real memory is... eg: 16G RAM means
> that 16G to 32G of swap is configured...

That's at the discretion of whoever built the distro, it's only a
convention. Check the output of  swapon -s  and monitor console output
for kernel messages suggesting there might be a problem.

In the case of Raspbian the swap is in a file rather than a separate
partition, and I've got a vague recollection that it has to be
explicitly enabled the first time the system is run.

If swap isn't working then something at the system level is broken. Once
I'd hit on the right binaries and sources (initially, a patched 2.6.4)
I've had no significant problems on either Raspbian or pukka Debian
running on either a RPi 1 or 2 (3 not tested yet), with swap on either
card or a USB-connected Flash stick.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Sven Barth
In reply to this post by Henry Vermaak

Am 01.04.2016 11:20 schrieb "Henry Vermaak" <[hidden email]>:
>
> On Fri, Apr 01, 2016 at 09:12:10AM +0000, Mark Morgan Lloyd wrote:
> > Henry Vermaak wrote:
> > >On Fri, Apr 01, 2016 at 12:39:15AM -0400, Anthony Walter wrote:
> > >>My question is can someone look into this issue so that building Lazarus in
> > >>environments where free memory is than 700MB does not result in Lazarus
> > >>build failures?
> > >
> > >I've come across ld running out of memory when linking the lazarus
> > >executable on my (very) old ARM laptop.  You can try to pass
> > >--no-keep-memory and --reduce-memory-overheads to ld, at the cost of
> > >performance.  I can't see how the build system has anything to do with
> > >it.
> >
> > It's always worked fine for me on things like NSLU-2 provided that the
> > overall memory (i.e. RAM + swap) is at least 512Mb. It will be unfortunate
> > if recent builds of FPC and/or Lazarus have pushed this requirement upwards,
> > but so far I've not seen any problems on a Raspberry Pi (both 1 and 2)
> > running Raspbian or Debian.
>
> Yes, I remember that the slug needed swap.  Swap is painfully slow,
> though, so I'd recommend reducing ld's memory usage instead.  I wonder
> if anyone has tried the gold linker.  It was supposed to be faster, I
> don't know about memory usage.

FPC does not support the gold linker (though I don't remember right now what the problems were).

Regards,
Sven


--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Mark Morgan Lloyd
In reply to this post by Mark Morgan Lloyd
Mark Morgan Lloyd wrote:

> [hidden email] wrote:
>
>> ummm... shouldn't the compiling process use as much swap as needed if
>> real memory it too tight (aka not enough)??
>
> Yes, and that's outside the remit of Lazarus etc.
>
>> the swap size is what attracted my attention because normally swap is
>> the same or double what installed real memory is... eg: 16G RAM means
>> that 16G to 32G of swap is configured...
>
> That's at the discretion of whoever built the distro, it's only a
> convention. Check the output of  swapon -s  and monitor console output
> for kernel messages suggesting there might be a problem.
>
> In the case of Raspbian the swap is in a file rather than a separate
> partition, and I've got a vague recollection that it has to be
> explicitly enabled the first time the system is run.
>
> If swap isn't working then something at the system level is broken. Once
> I'd hit on the right binaries and sources (initially, a patched 2.6.4)
> I've had no significant problems on either Raspbian or pukka Debian
> running on either a RPi 1 or 2 (3 not tested yet), with swap on either
> card or a USB-connected Flash stick.

Using an up-to-date Debian Jessie on an RPi 2, 1Gb RAM, no swap, and
with process accounting enabled,  make clean bigide  completed without
problems.

Unfortunately I've got minimal experience of the accounting stuff and
suspect that it's showing me the memory used by the most recent
invocation of each program rather than the maximum used, e.g. ld is
showing about 25Mb which is way smaller than experience would suggest is
correct.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Anthony Walter
I am running mate as the desktop environment which may account for a greater than normal memory usage than the default Raspbian environment, though it's not too much more.

I will say this though, increasing the swap file from the default of 100MB to 1024MB seems to alleviate the problem.

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Can some fix this "make" issue? Pi related.

Marco van de Voort
In reply to this post by Henry Vermaak
On Fri, Apr 01, 2016 at 09:23:23AM +0100, Henry Vermaak wrote:
> I've come across ld running out of memory when linking the lazarus
> executable on my (very) old ARM laptop.  You can try to pass
> --no-keep-memory and --reduce-memory-overheads to ld, at the cost of
> performance.  I can't see how the build system has anything to do with
> it.

Maybe trying to disable smartlinking also helps.

--
_______________________________________________
Lazarus mailing list
[hidden email]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus