[Lazarus] Multi-line msgid in PO file

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
I've had a problem with multi-line translations not working and realised
that an extra '\n' was appended to the last line of the msgid in the PO
file causing the translation lookup to fail.  I'm assuming that this is
a bug, but thought I'd ask here first.

I've attached a simple patch.

Henry

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus

multiline-newline.patch (613 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list


30.01.2019 20:10, Henry Vermaak via lazarus пишет:
> I've had a problem with multi-line translations not working and realised
> that an extra '\n' was appended to the last line of the msgid in the PO
> file causing the translation lookup to fail.  I'm assuming that this is
> a bug, but thought I'd ask here first.

Multiline translations are used in Lazarus and seem to work?

Please send a test project.

--
Best regards,
 Maxim Ganetsky                  mailto:[hidden email]
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
On Wed, Jan 30, 2019 at 08:19:47PM +0300, Maxim Ganetsky via lazarus wrote:
> 30.01.2019 20:10, Henry Vermaak via lazarus пишет:
> > I've had a problem with multi-line translations not working and realised
> > that an extra '\n' was appended to the last line of the msgid in the PO
> > file causing the translation lookup to fail.  I'm assuming that this is
> > a bug, but thought I'd ask here first.
>
> Multiline translations are used in Lazarus and seem to work?

That's perhaps because lazarus doesn't use .mo files?

> Please send a test project.

OK, see attached.  Steps to reproduce:

- Build project.

- Run and check hover hint on form, it's in English.

- Edit po/fr.po and remove the "\n" on the last line of the multiline
  hint for both msgid and msgstr.

- Re-create the .mo file:

msgfmt -o locale/fr/LC_MESSAGES/multiline.mo po/fr.po

- Run again and the hint will now be in French(*).

Henry

* Or some approximation thereof :)

--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus

multiline.tgz (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
31.01.2019 14:01, Henry Vermaak via lazarus пишет:
> On Wed, Jan 30, 2019 at 08:19:47PM +0300, Maxim Ganetsky via lazarus wrote:
>> 30.01.2019 20:10, Henry Vermaak via lazarus пишет:
>>> I've had a problem with multi-line translations not working and realised
>>> that an extra '\n' was appended to the last line of the msgid in the PO
>>> file causing the translation lookup to fail.  I'm assuming that this is
>>> a bug, but thought I'd ask here first.

Should be fixed in r60268.

Note that multiline strings seem to be always stored with Unix-style
linebreaks in MO file, while in form they can have e.g. Windows-style or
maybe even Mac-style linebreaks. Because of this, these strings should
be normalized with regards to linebreaks before searching in MO file.

Linebreak at the end of multilined string should be present as per PO
format examples, most PO editors add it anyway.

Also note, that loading multiline resourcestrings from MO files probably
won't work in many cases for aforementioned reasons too, because Gettext
unit does not account for platform-specific lineending changes at all.

--
Best regards,
  Maxim Ganetsky                  mailto:[hidden email]
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
On Thu, 31 Jan 2019 at 23:38, Maxim Ganetsky via lazarus
<[hidden email]> wrote:
> 31.01.2019 14:01, Henry Vermaak via lazarus пишет:
> > On Wed, Jan 30, 2019 at 08:19:47PM +0300, Maxim Ganetsky via lazarus wrote:
> >> 30.01.2019 20:10, Henry Vermaak via lazarus пишет:
> >>> I've had a problem with multi-line translations not working and realised
> >>> that an extra '\n' was appended to the last line of the msgid in the PO
> >>> file causing the translation lookup to fail.  I'm assuming that this is
> >>> a bug, but thought I'd ask here first.
>
> Should be fixed in r60268.

No, lazarus still adds an extra newline to the end of the msgid
entries in the PO files.  This is incorrect, the original text does
not include a trailing newline (check the lrj file).  rstconv also
doesn't write spurious newlines to PO files when I feed the lrj to it.

> Linebreak at the end of multilined string should be present as per PO
> format examples, most PO editors add it anyway.

No, this is incorrect.  PO editors have nothing to do with what gets
added to msgid, but they will preserve the newlines that are in the
msgid and make sure that the translation ends in a newline if the
original ends in a newline.  xgettext is responsible for extracting
strings from source and it does not add an extra newline when used
with C or lua source, we've been using multiline strings in C and lua
for years without issues.

Could you explain what was wrong with the patch I sent in the first message?

Henry
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
01.02.2019 12:43, Henry Vermaak пишет:

> On Thu, 31 Jan 2019 at 23:38, Maxim Ganetsky via lazarus
> <[hidden email]> wrote:
>> Should be fixed in r60268.
>
> No, lazarus still adds an extra newline to the end of the msgid
> entries in the PO files.  This is incorrect, the original text does
> not include a trailing newline (check the lrj file).  rstconv also
> doesn't write spurious newlines to PO files when I feed the lrj to it.
>
>> Linebreak at the end of multilined string should be present as per PO
>> format examples, most PO editors add it anyway.

> No, this is incorrect.  PO editors have nothing to do with what gets
> added to msgid, but they will preserve the newlines that are in the
> msgid and make sure that the translation ends in a newline if the
> original ends in a newline.  xgettext is responsible for extracting
> strings from source and it does not add an extra newline when used
> with C or lua source, we've been using multiline strings in C and lua
> for years without issues.

As I remember, it was not the case earlier at least with Poedit. But its
current version behaves exactly as you describe. Maybe it is indeed a
good idea to avoid changing these newlines. I will look into it.

> Could you explain what was wrong with the patch I sent in the first message?

It needlessly changes formatting of PO files and is not sufficient to
correctly solve the problem at hand anyway. But changes along its lines
should be made, yes.

--
Best regards,
 Maxim Ganetsky                  mailto:[hidden email]
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
On Fri, 1 Feb 2019 at 12:46, Maxim Ganetsky via lazarus
<[hidden email]> wrote:
> As I remember, it was not the case earlier at least with Poedit. But its
> current version behaves exactly as you describe. Maybe it is indeed a
> good idea to avoid changing these newlines. I will look into it.

Thanks, much appreciated.

Henry
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
01.02.2019 19:38, Henry Vermaak пишет:
> On Fri, 1 Feb 2019 at 12:46, Maxim Ganetsky via lazarus
> <[hidden email]> wrote:
>> As I remember, it was not the case earlier at least with Poedit. But its
>> current version behaves exactly as you describe. Maybe it is indeed a
>> good idea to avoid changing these newlines. I will look into it.
>
> Thanks, much appreciated.

Fixed in r60343. Please test.

Note that there can be spurious changes of PO files on first IDE
rebuild. You can safely revert them. Also do not forget to rebuild
updatepofiles tool if you use it.

--
Best regards,
  Maxim Ganetsky                  mailto:[hidden email]
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
On Wed, Feb 06, 2019 at 03:58:37AM +0300, Maxim Ganetsky via lazarus wrote:

> 01.02.2019 19:38, Henry Vermaak пишет:
> >On Fri, 1 Feb 2019 at 12:46, Maxim Ganetsky via lazarus
> ><[hidden email]> wrote:
> >>As I remember, it was not the case earlier at least with Poedit. But its
> >>current version behaves exactly as you describe. Maybe it is indeed a
> >>good idea to avoid changing these newlines. I will look into it.
> >
> >Thanks, much appreciated.
>
> Fixed in r60343. Please test.

We've merged them onto fixes_2_0 and everything works well now, thanks!
It's a pity we've missed the release, can you merge this onto fixes_2_0?
Do you still have to strip trailing newlines from translated strings for
lazarus PO files?

Henry
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
06.02.2019 15:42, Henry Vermaak via lazarus пишет:

> On Wed, Feb 06, 2019 at 03:58:37AM +0300, Maxim Ganetsky via lazarus wrote:
>> 01.02.2019 19:38, Henry Vermaak пишет:
>>> On Fri, 1 Feb 2019 at 12:46, Maxim Ganetsky via lazarus
>>> <[hidden email]> wrote:
>>>> As I remember, it was not the case earlier at least with Poedit. But its
>>>> current version behaves exactly as you describe. Maybe it is indeed a
>>>> good idea to avoid changing these newlines. I will look into it.
>>>
>>> Thanks, much appreciated.
>>
>> Fixed in r60343. Please test.
>
> We've merged them onto fixes_2_0 and everything works well now, thanks!

Great.

> It's a pity we've missed the release, can you merge this onto fixes_2_0?

I don't think that this is a good idea at this point of time, the change
is too noisy/invasive. Technically this was not a regression, but rather
a (mis)feature.

> Do you still have to strip trailing newlines from translated strings for
> lazarus PO files?

All such translations will be marked as fuzzy on their regeneration. All
affected Lazarus PO files have already been regenerated.

--
Best regards,
 Maxim Ganetsky                  mailto:[hidden email]
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus
Reply | Threaded
Open this post in threaded view
|

Re: [Lazarus] Multi-line msgid in PO file

Free Pascal - Lazarus mailing list
On Wed, Feb 06, 2019 at 04:16:49PM +0300, Maxim Ganetsky via lazarus
wrote:
> 06.02.2019 15:42, Henry Vermaak via lazarus пишет:
> > It's a pity we've missed the release, can you merge this onto
> > fixes_2_0?
>
> I don't think that this is a good idea at this point of time, the
> change is too noisy/invasive. Technically this was not a regression,
> but rather a (mis)feature.

I'd argue it was a bug (at least with .mo files), but anyway.  We'll
just have to patch all of our lazarus trees until we can move off
fixes_2_0.

> > Do you still have to strip trailing newlines from translated strings
> > for lazarus PO files?
>
> All such translations will be marked as fuzzy on their regeneration.
> All affected Lazarus PO files have already been regenerated.

Great, thanks.

Henry
--
_______________________________________________
lazarus mailing list
[hidden email]
https://lists.lazarus-ide.org/listinfo/lazarus